Scraping a Facebook App for Data

Question

I\'m using a Facebook application that has a rich set of information that I\'d like to get at offline. To do this, I essentially need to read the infromation from the web pa

Answer 1

I think they're using cookies to pass authentication, so first you'll need your app to login to facebook, and keep the cookie in a CookieContainer, then assign that to req.CookieContainer, and only then req.GetResponse();

Answer 2

You will first have to write a script to programmatically log into facebook. Then you will have to save the cookies you get.

I have done something similar with curl and php. (curl has built in cookie handling)

Answer 3

As far as I understood you just need to login to facebook appliction, right? Use any web scraping/crawling framework for it (they support JS, cookies, etc.). They just emulate usuall web browsing. For example, try these:

http://scrapy.org/

http://wwwsearch.sourceforge.net/mechanize/

http://watin.sourceforge.net/

Also see

.Net Screen scraping and session

Answer 4

You need to use the facebook API to get data from facebook. They block screenscaping

Answer 5

Facebook uses REST so you can request the data from the server; it also has the ability to access data from the client-side JavaScript API. You can check the wiki for more information; it uses a rest_server.php?method= to get the data from the appropriate methods.

Check out http://www.facebook.com/developers to get more information about these objects/methods/creating an application so you can query facebook data.