Get URL Contents in Node.js with Express

Question

How would I go about downloading the contents of a URL in Node when using the Express framework? Basically, I need to complete the Facebook authentication flow, but I can\'t

Answer 1

The problem that you will front is: some webpage loads its contents using JavaScript. Thus, you needs a package, like After-Load which simulates browser's behavior, then gives you the HTML content of that URL .

var afterLoad = require('after-load');
afterLoad('https://google.com', function(html){
   console.log(html);
});

Answer 2

Using http way requires way more lines of code for just a simple html page .

Here's an efficient way : Use request

var request = require("request");

request({uri: "http://www.sitepoint.com"}, 
    function(error, response, body) {
    console.log(body);
  });
});

Here is the doc for request : https://github.com/request/request

---

2nd Method using fetch with promises :

    fetch('https://sitepoint.com')
    .then(resp=> resp.text()).then(body => console.log(body)) ; 

Answer 3

var options = {
  host: 'www.google.com',
  port: 80,
  path: '/index.html'
};

http.get(options, function(res) {
  console.log("Got response: " + res.statusCode);
}).on('error', function(e) {
  console.log("Got error: " + e.message);
});

http://nodejs.org/docs/v0.4.11/api/http.html#http.get