Node.JS How to set a variable outside the current scope

Question

I have some code that I cant get my head around, I am trying to return an array of object using a callback, I have a function that is returning the values and then pushing t

Answer 1

You have multiple issues going on here. A core issue is to gain an understanding of how asynchronous responses work and which code executes when. But, in addition to that you also have to learn how to manage multiple async responses in a loop and how to know when all the responses are done and how to get the results in order and what tools can best be used in node.js to do that.

Your core issue is a matter of timing. The createOrUpdateMarket() function is probably asynchronous. That means that it starts its operation when the function is called, then calls its callback sometime in the future. Meanwhile the rest of your code continues to run. Thus, you are trying to access the array BEFORE the callback has been called.

Because you cannot know exactly when that callback will be called, the only place you can reliably use the callback data is inside the callback or in something that is called from within the callback.

You can read more about the details of the async/callback issue here: Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference

To know when a whole series of these createOrUpdateMarket() operations are all done, you will have to code especially to know when all of them are done and you cannot rely on a simple for loop. The modern way to do that is to use promises which offer tools for helping you manage the timing of one or more asynchronous operations.

In addition, if you want to accumulate results from your for loop in marketArray, you have to declare and initialize that before your for loop, not inside your for loop. Here are several solutions:

Manually Coded Solution

var len = res.response.result.length;
var marketArray = new Array(len), cntr = 0;
for (var index = 0, index < len; index++) {
    (function(i) {
        createOrUpdateMarket(res.response.result[i], eventObj , function (err, marketObj) {
            ++cntr;
            if (err) {
                // need error handling here
            }
            marketArray[i] = marketObj;
            // if last response has just finished
            if (cntr === len) {
                // here the marketArray is fully populated and all responses are done
                // put your code to process the marketArray here
            }
        });
    })(index);
}

Standard Promises Built Into Node.js

// make a version of createOrUpdateMarket that returns a promise
function createOrUpdateMarketAsync(a, b) {
    return new Promise(function(resolve, reject) {
        createOrUpdateMarket(a, b, function(err, marketObj) {
            if (err) {
                reject(err);
                return;
            }
            resolve(marketObj);
        });
    });
}

var promises = [];
for (var i = 0; i < res.response.result.length; i++) {
    promises.push(createorUpdateMarketAsync(res.response.result[i], eventObj));
}
Promise.all(promises).then(function(marketArray) {
    // all results done here, results in marketArray
}, function(err) {
    // an error occurred
});

Enhanced Promises with the Bluebird Promise library

The bluebird promise library offers Promise.map() which will iterate over your array of data and produce an array of asynchronously obtained results.

// make a version of createOrUpdateMarket that returns a promise
var Promise = require('bluebird');
var createOrUpdateMarketAsync = Promise.promisify(createOrUpdateMarket);

// iterate the res.response.result array and run an operation on each item
Promise.map(res.response.result, function(item) {
    return createOrUpdateMarketAsync(item, eventObj);
}).then(function(marketArray) {
    // all results done here, results in marketArray
}, function(err) {
    // an error occurred
});

Async Library

You can also use the async library to help manage multiple async operations. In this case, you can use async.map() which will create an array of results.

var async = require('async');
async.map(res.response.result, function(item, done) {
    createOrUpdateMarker(item, eventObj, function(err, marketObj) {
        if (err) {
            done(err);
        } else {
            done(marketObj);
        }
    });
}, function(err, results) {
    if (err) {
        // an error occurred
    } else {
        // results array contains all the async results
    }
});