Unnesting Node database calls

Question

I have an ordinary

var express = require(\'express\')

Node express www page, using session, pug, etc as usual. My db calls

var         


        

Answer 1

If your database calls returned promises instead of using callbacks, you could:

const cats = await db.cats();
const dogs = await db.dogs();
const budgies = await db.budgies();

res.render('bigScreen.pug', {
  cats : cats,
  k9s: dogs,
  budgies: budgies,
  pretty : true
});


// Or request them all in parallel instead of waiting for each to finish
const [
  cats,
  dogs,
  budgies
] = Promise.all([
  dg.cats(),
  dg.dogs(),
  db.budgies()
]);

Answer 2

You can get rid of nested database calls by using promises.

Since you mentioned that you are using mysql library for interacting with the database, unfortunately, this library doesn't provides a promise-based API. So to get rid of nested database calls in your code, you need to create a promise-based wrapper around the callback version of database calls.

For a general overview of what promises are and how they work, see the following links:

• MDN - Promise.
• MDN - Using Promises

Following is an example of how you can create a promise-based wrapper and then use that wrapper to get rid of nested database calls.

This promise-based wrapper is just a function that returns a promise. It creates a promise instance, wraps the underlying database call and eventually when the database call returns the data, it notifies your code.

function getCats() {
   return new Promise((resolve, reject) => {
       // make the database call
       db.cats((error, cats) => {
           // in case of an error, reject the promise by
           // calling "reject" function
           // Also pass the "error" object to the "reject" function
           // as an argument to get access to the error message 
           // in the code that calls this "getCats" function
           if (error) {
              reject(error);
              return;
           }
           
           // if there was no error, call "resolve" function
           // to resolve the promise. Promise will be resolved 
           // in case of successful database call
           // Also pass the data to "resolve" function
           // to access this data in the code that calls this
           // "getCats" function
           resolve(cats);
       });
   });
}

Now to in your route handler function, instead of calling db.cats(...), call this getCats wrapper function.

There are two ways you can call the function that returns a promise:

• Promise-chaining (For details, visit the links mentioned above)
• async-await syntax (Recommended)

Following code example uses async-await syntax. For this, first mark the route handler function as async by using the async keyword before the function keyword. Doing this, we can use await keyword inside this route handler function.

app.get('/pets', async function(req, res, next) {
    try {
       const cats = await getCats();
       // similar wrappers for other database calls
       const dogs = await getDogs();
       const budgies = await getBudgies();
       
       // render the pub template, passing in the data
       // fetched from the database 
       ...

     catch (error) {
       // catch block will be invoked if the promise returned by
       // the promise-based wrapper function is rejected
       // handle the error appropriately
     }
});

Above code example only shows how to wrap the db.cats(...) database call in a promise-based wrapper and use that wrapper to get the data from the database. Similarly, you can create wrappers for db.dogs(...) and db.budgies(...) calls.

Instead of creating a separate promise-based wrapper for each database call, ideally, you should create a re-usable promise-based wrapper function that takes in a function to call and wraps that function call in a promise just like shown in the above code example, i.e. getCats function.

Parallel Database calls

One important thing to note in the above code in the route handler function

const cats = await getCats();
const dogs = await getDogs();
const budgies = await getBudgies();

is that this will lead to sequential database calls which may or may not what you want.

If these database calls do not depend on each other, then you can call the promise-based wrappers in parallel using Promise.all() method.

Following code example shows how you can call your promise-based wrapper functions in parallel using Promise.all().

app.get('/pets', async function(req, res, next) {
    try {
       // "petsData" will be an array that will contain all the data from 
       // three database calls.
       const petsData = await Promise.all([getCats(), getDogs(), getBudgies()]);
       
       // render the pub template, passing in the data
       // fetched from the database 
       ...
 
     catch (error) {
       ...
     }
 });

I hope this is enough to help you get rid of the nested database calls in your current code and start using promises in your code.

Answer 3

If you're trying to use MySQL with Nodejs, the module you should be looking for is mysql2 rather than mysql.

mysql2 provides a promise based approach and is a much refined version of mysql module for nodejs.

For example, for executing a query,

1. in mysql

con.query(sql_query, (err, rows, field)=>{ //some code here }

2. in mysql2, you can use the async approach as well as promise approach. Also, prepared statements in mysql2 are more easier than mysql.

//async approach
class A {
   static async fn(sql, params){
      const [data] = await con.execute(sql, [params]);
    return data;
    }
}

//promise approach remains same as **mysql** itself.

Here's the documentation for mysql2 & more docs

Answer 4

Promise.all() method seems a more famous and cleaner way to make multiple calls in parallel like your use case.

But there is one more alternate way. : Multiple statement queries

To use this feature you have to enable it for your connection:

var connection = mysql.createConnection({multipleStatements: true});

Once enabled, you can execute multiple statement queries like any other query:

db.query('SELECT cats; SELECT dogs', function (error, results, fields) {
  if (error) throw error;
  // `results` is an array with one element for every statement in the query:
  console.log(results[0]); // [{cat1,cat2}]
  console.log(results[1]); // [{dog1,dog2}]
});

It is technically more efficient as requires less back and forth with MySQL connection.

(However, this feature is disabled by default as it allows for SQL injection attacks if values are not properly escaped). To use this feature you have to enable it for your connection.)

Answer 5

Simply convert the mysql functions into promises using the nodejs standard lib util.promisify

example:

const { promisify } = require('util');

const catsPromise = promisify(db.cats);
const dogsPromise = promisify(db.dogs);
const budgiesPromise = promisify(db.budgies);

async function routeHandler() {
  let err = null;

  try {
    const cats = await catsPromise();
    const dogs = await dogsPromise();
    const budgies = await budgiesPromise();
  } catch(error) {
    err = error;
  }

  if (err) {
    console.log(err);
    // you should res.end() or res.render(someErrorPage) here
    // failure to do so will leave the request open
  } else {
    res.render('bigScreen.pug', {
      'cats' : cats,
      'k9s': dogs,
      'budgies': budgies,
      'pretty' : true
    });
  }
}