Returning asynchronous data then exporting it synchronously in Node.js

Question

Background

I am returning data from AWS Secrets Manager and using the aws-sdk to do so. Earlier I asked a question about how to correct

Answer 1

Because the needed data is gotten asynchronously, there's no way around making everything that depends on it (somehow) asynchronous as well. With asynchronicity involved, one possibility is to usually export functions that can be called on demand, rather than exporting objects:

• an object that depends on the asynchronous data can't be meaningfully exported before the data comes back
• if you export functions rather than objects, you can ensure that control flow starts from your single entry point and heads downstream, rather than every module initializing itself at once (which can be problematic when some modules depend on others to be initialized properly, as you're seeing)

On another note, note that if you have a single Promise that needs to resolve, it's probably easier to call .then on it than use an async function. For example, rather than

const promise = require('../secrets');

(async () => {
  // try/catch is needed to handle rejected promises when using await:
  try {
    const secrets = await promise;
    // use secrets here
  } catch(e) {
    // handle errors
  }
})();

you might consider:

const promise = require('../secrets');

promise
  .then((secrets) => {
    // use secrets here
  })
  .catch((err) => {
    // handle errors
  });

It's less wordy and probably easier to make sense of at a glance - better than a self-invoking async IIFE. IMO, the place to use await is when you have multiple Promises that need to resolve, and chaining .thens and returned Promises together gets too ugly.

A module that depends on secrets to perform has to, in its code, have something that effectively waits for secrets to be populated. Although being able to use your const secrets = require('../secrets'); in your lower code example would be nice, it just isn't possible like that. You can export a function that takes secrets as a parameter rather than as a require, and then (synchronously!) return the instantiated pool:

// note, secrets is *not* imported
function makePool(secrets) {
  const pool = new Pool({
    user: secrets.dbUsername,
    host: secrets.dbHost,
    database: secrets.dbDatabase,
    password: secrets.dbPassword,
    port: secrets.dbPort
  });

  pool.on('error', err => {
    console.error('Unexpected error on idle client', err);
    process.exit(-1);
  });
  return pool;
}

module.exports = makePool;

Then, to use it in another module, once the secrets are created, call makePool with the secrets, and then use / pass around the returned pool:

const secretsProm = require('../secrets');
const makePool = require('./makePool');
secretsProm.then((secrets) => {
  const pool = makePool(secrets);
  doSomethingWithPool(pool);
})
.catch((err) => {
  // handle errors
});

Note that the doSomethingWithPool function can be completely synchronous, as is makePool - the asynchronous nature of secrets, once handled with .then in one module, does not have to be dealt with asynchronously anywhere else, as long as other modules export functions, rather than objects.

Answer 2

I would suggest doing everything in 1 file, and then instead of exporting the object you create, export a function that returns the object. The function will always have access to the must up-to-date version of the object, and you can call it from any file to access the same object.

Example: Create two files in a folder. In the first file, we will do this:

• Define a value.
• Set a timeout to change the value after some time
• Export the value itself
• Export a function that returns the value

values.js

let x = 0 ; // set initial value
setTimeout(() => { x = 5; }, 2000); // sometime later, value will change

const getValueOfX = () => { return x; }; 

module.exports = {
    x: x,
    getValueOfX: getValueOfX
}; 

Now in the other file, we just import the two exports from the previous file (we put them both in an object for easy exporting). We can then log them out, wait for some time to pass, and log them out again.

index.js

let values = require('./values');

console.log(`Single value test. x = ${values.x}`);
console.log(`Function return value test. x = ${values.getValueOfX()}`);
setTimeout(() => { console.log(`Single value test. x = ${values.x}`); }, 4000);
setTimeout(() => { console.log(`Function return value test. x = ${values.getValueOfX()}`); }, 4000);

To run the code, just open your Terminal or Command Prompt and, from the same directory as these two files, run node index.js

You'll see that when just the value (object, array, w/e) is exported, it is exported as-is when the export runs - almost always before the API call is finished.

BUT - If you export a function that returns the value (object, array, w/e), then that function will retrieve the up-to-date version of the value at the time it is called! Great for API calls!

so your code might look like this:

let secrets = { jwtHash: 10 };
const client = new AWS.SecretsManager({
    region: region
});

let pool = null; 

client.getSecretValue({ SecretId: secretName }, async (err, data) => {
    if (err) {
        reject(err);
    } else {
        const res = await JSON.parse(data.SecretString);
        pool = new Pool({
            user: res.username,
            host: res.host
            database: res.dbname
            password: res.password
            port: res.port
        }); 
        pool.on('error', err=> {
            console.error('Unexpected error on idle client', err);
            process.exit(-1);
        }); 
    }
});

module.exports = function(){ return pool; };