What is this Javascript “require”?

Question

I\'m trying to get Javascript to read/write to a PostgreSQL database. I found this project on github. I was able to get the following sample code to run in node.

<         


        

Answer 1

It's used to load modules. Let's use a simple example.

In file circle_object.js:

var Circle = function (radius) {
    this.radius = radius
}
Circle.PI = 3.14

Circle.prototype = {
    area: function () {
        return Circle.PI * this.radius * this.radius;
    }
}

We can use this via require, like:

node> require('circle_object')
{}
node> Circle
{ [Function] PI: 3.14 }
node> var c = new Circle(3)
{ radius: 3 }
node> c.area()

The require() method is used to load and cache JavaScript modules. So, if you want to load a local, relative JavaScript module into a Node.js application, you can simply use the require() method.

Example:

var yourModule = require( "your_module_name" ); //.js file extension is optional

Answer 2

So what is this "require?"

require() is not part of the standard JavaScript API. But in Node.js, it's a built-in function with a special purpose: to load modules.

Modules are a way to split an application into separate files instead of having all of your application in one file. This concept is also present in other languages with minor differences in syntax and behavior, like C's include, Python's import, and so on.

One big difference between Node.js modules and browser JavaScript is how one script's code is accessed from another script's code.

• In browser JavaScript, scripts are added via the <script> element. When they execute, they all have direct access to the global scope, a "shared space" among all scripts. Any script can freely define/modify/remove/call anything on the global scope.

• In Node.js, each module has its own scope. A module cannot directly access things defined in another module unless it chooses to expose them. To expose things from a module, they must be assigned to exports or module.exports. For a module to access another module's exports or module.exports, it must use require().

In your code, var pg = require('pg'); loads the pg module, a PostgreSQL client for Node.js. This allows your code to access functionality of the PostgreSQL client's APIs via the pg variable.

Why does it work in node but not in a webpage?

require(), module.exports and exports are APIs of a module system that is specific to Node.js. Browsers do not implement this module system.

Also, before I got it to work in node, I had to do npm install pg. What's that about?

NPM is a package repository service that hosts published JavaScript modules. npm install is a command that lets you download packages from their repository.

Where did it put it, and how does Javascript find it?

The npm cli puts all the downloaded modules in a node_modules directory where you ran npm install. Node.js has very detailed documentation on how modules find other modules which includes finding a node_modules directory.

Answer 3

You know how when you are running JavaScript in the browser, you have access to variables like "window" or Math? You do not have to declare these variables, they have been written for you to use whenever you want.

Well, when you are running a file in the Node.js environment, there is a variable that you can use. It is called "module" It is an object. It has a property called "exports." And it works like this:

In a file that we will name example.js, you write:

example.js

module.exports = "some code";

Now, you want this string "some code" in another file.

We will name the other file otherFile.js

In this file, you write:

otherFile.js

let str = require('./example.js')

That require() statement goes to the file that you put inside of it, finds whatever data is stored on the module.exports property. The let str = ... part of your code means that whatever that require statement returns is stored to the str variable.

So, in this example, the end-result is that in otherFile.js you now have this:

let string = "some code";

• or -

let str = ('./example.js').module.exports

Note:

the file-name that is written inside of the require statement: If it is a local file, it should be the file-path to example.js. Also, the .js extension is added by default, so I didn't have to write it.

You do something similar when requiring node.js libraries, such as Express. In the express.js file, there is an object named 'module', with a property named 'exports'.

So, it looks something like along these lines, under the hood (I am somewhat of a beginner so some of these details might not be exact, but it's to show the concept:

express.js

module.exports = function() {
    //It returns an object with all of the server methods
    return {
        listen: function(port){},
        get: function(route, function(req, res){}){}
     }
}

If you are requiring a module, it looks like this: const moduleName = require("module-name");

If you are requiring a local file, it looks like this: const localFile = require("./path/to/local-file");

(notice the ./ at the beginning of the file name)

---

Also note that by default, the export is an object .. eg module.exports = {} So, you can write module.exports.myfunction = () => {} before assigning a value to the module.exports. But you can also replace the object by writing module.exports = "I am not an object anymore."

Answer 4

Two flavours of module.exports / require:

(see here)

Flavour 1
export file (misc.js):

var x = 5;
var addX = function(value) {
  return value + x;
};
module.exports.x = x;
module.exports.addX = addX;

other file:

var misc = require('./misc');
console.log("Adding %d to 10 gives us %d", misc.x, misc.addX(10));

Flavour 2
export file (user.js):

var User = function(name, email) {
  this.name = name;
  this.email = email;
};
module.exports = User;

other file:

var user = require('./user');
var u = new user();

Answer 5

Necromancing.
IMHO, the existing answers leave much to be desired.

It's very simple:
Require is simply a (non-standard) function defined at global scope.
(window in browser, global in NodeJS).

Now, as such, to answer the question "what is require", we "simply" need to know what this function does.
This is perhaps best explained with code.

Here's a simple implementation by Michele Nasti, the code you can find on his github page.

Basically, let's call our minimalisc require function myRequire:

function myRequire(name) 
{
    console.log(`Evaluating file ${name}`);
    if (!(name in myRequire.cache)) {
        console.log(`${name} is not in cache; reading from disk`);
        let code = fs.readFileSync(name, 'utf8');
        let module = { exports: {} };
        myRequire.cache[name] = module;
        let wrapper = Function("require, exports, module", code);
        wrapper(myRequire, module.exports, module);
    }
    console.log(`${name} is in cache. Returning it...`);
    return myRequire.cache[name].exports;
}
myRequire.cache = Object.create(null);
window.require = myRequire;
const stuff = window.require('./main.js');
console.log(stuff);

Now you notice, the object "fs" is used here.
For simplicity's sake, Michele just imported the NodeJS fs module:

const fs = require('fs');

Which wouldn't be necessary.
So in the browser, you could make a simple implementation of require with a SYNCHRONOUS XmlHttpRequest:

const fs = {
    file: `
    // module.exports = \"Hello World\";
        
    module.exports = function(){ return 5*3;};
    
    `
    , getFile(fileName: string, encoding: string): string
    {
        // https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Synchronous_and_Asynchronous_Requests
        let client = new XMLHttpRequest();
        // client.setRequestHeader("Content-Type", "text/plain;charset=UTF-8");

        // open(method, url, async)
        client.open("GET", fileName, false);
        client.send();
        if (client.status === 200)
            return client.responseText;

        return null;
    }


    , readFileSync: function (fileName: string, encoding: string): string
    {
        // this.getFile(fileName, encoding);
        return this.file; // Example, getFile would fetch this file 
    }
};

Basically, what require thus does, is download a JavaScript-file, eval it in an anonymous namespace (aka Function), with the global parameters "require", "exports" and "module", and return the exports, meaning an object's public functions and properties.

Note that this evaluation is recursive: you require files, which themselfs can require files.

This way, all "global" variables used in your module are variables in the require-wrapper-function namespace, and don't pollute the global scope with unwanted variables.

Also, this way, you can reuse code without depending on namespaces, so you get "modularity" in JavaScript. "modularity" in quotes, because this is not exactly true, though, because you can still write window.bla, and hence still pollute the global scope... Also, this establishes a separation between private and public functions, the public functions being the exports.

Now instead of saying

module.exports = function(){ return 5*3;};

You can also say:

function privateSomething()
{
    return 42:
}


function privateSomething2()
{
    return 21:
}


module.exports = {
      getRandomNumber: privateSomething
     ,getHalfRandomNumber: privateSomething2
};

and return an object.

Also, because your modules get evaluated in a function with parameters "require", "exports" and "module", your modules can use the undeclared variables "require", "exports" and "module", which might be startling at first. The require parameter there is of course a ByVal pointer to the require function saved into a variable.
Cool, right ?
Seen this way, require looses its magic, and becomes simple.
Now, the real require-function will do a few more checks and quirks, of course, but this is the essence of what that boils down to.

Also, in 2020, you should use the ECMA implementations instead of require:

import defaultExport from "module-name";
import * as name from "module-name";
import { export1 } from "module-name";
import { export1 as alias1 } from "module-name";
import { export1 , export2 } from "module-name";
import { foo , bar } from "module-name/path/to/specific/un-exported/file";
import { export1 , export2 as alias2 , [...] } from "module-name";
import defaultExport, { export1 [ , [...] ] } from "module-name";
import defaultExport, * as name from "module-name";
import "module-name";

And if you need a dynamic non-static import (e.g. load a polyfill based on browser-type), there is the ECMA-import function/keyword:

var promise = import("module-name");

note that import is not synchronous like require.
Instead, import is a promise, so

var something = require("something");

becomes

var something = await import("something");

because import returns a promise (asynchronous).

So basically, unlike require, import replaces fs.readFileSync with fs.readFileAsync.

async readFileAsync(fileName, encoding) 
{
    const textDecoder = new TextDecoder(encoding);
    // textDecoder.ignoreBOM = true;
    const response = await fetch(fileName);
    console.log(response.ok);
    console.log(response.status);
    console.log(response.statusText);
    // let json = await response.json();
    // let txt = await response.text();
    // let blo:Blob = response.blob();
    // let ab:ArrayBuffer = await response.arrayBuffer();
    // let fd = await response.formData()
    // Read file almost by line
    // https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamDefaultReader/read#Example_2_-_handling_text_line_by_line
    let buffer = await response.arrayBuffer();
    let file = textDecoder.decode(buffer);
    return file;
} // End Function readFileAsync

This of course requires the import-function to be async as well.

"use strict";
async function myRequireAsync(name) {
    console.log(`Evaluating file ${name}`);
    if (!(name in myRequireAsync.cache)) {
        console.log(`${name} is not in cache; reading from disk`);
        let code = await fs.readFileAsync(name, 'utf8');
        let module = { exports: {} };
        myRequireAsync.cache[name] = module;
        let wrapper = Function("asyncRequire, exports, module", code);
        await wrapper(myRequireAsync, module.exports, module);
    }
    console.log(`${name} is in cache. Returning it...`);
    return myRequireAsync.cache[name].exports;
}
myRequireAsync.cache = Object.create(null);
window.asyncRequire = myRequireAsync;
async () => {
    const asyncStuff = await window.asyncRequire('./main.js');
    console.log(asyncStuff);
};

Even better, right ?
Well yea, except that there is no ECMA-way to dynamically import synchronously (without promise).

Now, to understand the repercussions, you absolutely might want to read up on promises/async-await here, if you don't know what that is.

But very simply put, if a function returns a promise, it can be "awaited":

function sleep (fn, par) 
{
  return new Promise((resolve) => {
    // wait 3s before calling fn(par)
    setTimeout(() => resolve(fn(par)), 3000)
  })
}


var fileList = await sleep(listFiles, nextPageToken)

Which is nice way to make asynchronous code look synchronous.
Note that if you want to use async await in a function, that function must be declared async.

async function doSomethingAsync()
{
    var fileList = await sleep(listFiles, nextPageToken)
}

And also please note that in JavaScript, there is no way to call an async function (blockingly) from a synchronous one (the ones you know). So if you want to use await (aka ECMA-import), all your code needs to be async, which most likely is a problem, if everything isn't already async...

An example of where this simplified implementation of require fails, is when you require a file that is not valid JavaScript, e.g. when you require css, html, txt, svg and images or other binary files.
And it's easy to see why:
If you e.g. put HTML into a JavaScript function body, you of course rightfully get

SyntaxError: Unexpected token '<'

because of Function("bla", "<doctype...")

Now, if you wanted to extend this to for example include non-modules, you could just check the downloaded file-contents with for code.indexOf("module.exports") == -1, and then e.g. eval("jquery content") instead of Func (which works fine as long as you're in the browser). Since downloads with Fetch/XmlHttpRequests are subject to the same-origin-policy, and integrity is ensured by SSL/TLS, the use of eval here is rather harmless, provided you checked the JS files before you added them to your site, but that much should be standard-operating-procedure.

Note that there are several implementations of require-like functionality:

• the CommonJS (CJS) format, used in Node.js, uses a require function and module.exports to define dependencies and modules. The npm ecosystem is built upon this format. (this is what is implemented above)
• the Asynchronous Module Definition (AMD) format, used in browsers, uses a define function to define modules.
• the ES Module (ESM) format. As of ES6 (ES2015), JavaScript supports a native module format. It uses an export keyword to export a module’s public API and an import keyword to import it.
• the System.register format, designed to support ES6 modules within ES5.
• the Universal Module Definition (UMD) format, compatible to all the above mentioned formats, used both in the browser and in Node.js. It’s especially useful if you write modules that can be used in both NodeJS and the browser.

Answer 6

Alright, so let's first start with making the distinction between Javascript in a web browser, and Javascript on a server (CommonJS and Node).

Javascript is a language traditionally confined to a web browser with a limited global context defined mostly by what came to be known as the Document Object Model (DOM) level 0 (the Netscape Navigator Javascript API).

Server-side Javascript eliminates that restriction and allows Javascript to call into various pieces of native code (like the Postgres library) and open sockets.

Now require() is a special function call defined as part of the CommonJS spec. In node, it resolves libraries and modules in the Node search path, now usually defined as node_modules in the same directory (or the directory of the invoked javascript file) or the system-wide search path.

To try to answer the rest of your question, we need to use a proxy between the code running in the the browser and the database server.

Since we are discussing Node and you are already familiar with how to run a query from there, it would make sense to use Node as that proxy.

As a simple example, we're going to make a URL that returns a few facts about a Beatle, given a name, as JSON.

/* your connection code */

var express = require('express');
var app = express.createServer();
app.get('/beatles/:name', function(req, res) {
    var name = req.params.name || '';
    name = name.replace(/[^a-zA_Z]/, '');
    if (!name.length) {
        res.send({});
    } else {
        var query = client.query('SELECT * FROM BEATLES WHERE name =\''+name+'\' LIMIT 1');
        var data = {};
        query.on('row', function(row) {
            data = row;
            res.send(data);
        });
    };
});
app.listen(80, '127.0.0.1');