Asynchronous program execution is the beauty of the Node. We attach a callback and leave the task to complete the execution. Sometimes this can be tricky. We can wait for the results of the asynchronous events but we can not predict the sequence of completion of the events. In this article, I am going to explain the use of Async to limit API calls.

From the Async Documentation,

Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript.

Background

Recently I was integrating the Magento orders API to pull the transaction data. The data was so huge that even the postman hung up. I tried reducing the payload. I divided each day into four sections. So to pull the data for 10 days, I had to make forty requests.The data started to come up but this design introduced one more issue. The no of requests I was making increased drastically which eventually led the server to block my further requests. This is a quite generic issue. There is a rate limit on the number of requests we make per second or per a hundred seconds.

Let’s see how we can use the Async package to control the asynchronous control flow.

Using async.eachLimit

async.eachLimit is like forEach loop with two differences. The execution is asynchronous and accepts one more parameter as a limit.

const request = require("request)";const urls = [
{path: "http://myUrl.com/v1/orders",token: "25s8$"},
{path: "http://myUrl.com/v1/sales",token: "25s8$"},
{path: "http://myUrl.com/v1/revenue/1234",token: "25s8$"},
{path: "http://myUrl.com/v1/transactions",token: "25s8$"}
];
let result = [];async.eachLimit(urls, 2, (url, done) => {request.get(url.path)
.then((response) => { 
  result.push(response);
  return done();
}).catch(error => done(error))}, (error) => {console.log(error);
});

There are four important parts in the above snippet.

1. Array to iterate over.
2. Limit to perform the maximum number of operations at a time.
3. Function to be called on each item of the array.
4. Callback when all the processing is done or an error has occurred.

We have an array of URL’s. The third parameter, a function actually calls the API and returns the callback. The fourth parameter, a callback function is called when all the processing is finished or an error is thrown during the execution. In our case, the callback will be called when the iteratee function is executed for each URL. The most important parameter, the limit parameter is set 2 which conveys that there will be maximum of 2 requests at a given time. So in our scenario orders and sales API will execute in parallel. When either of the requests is completed, revenue API will be called and same will happen for transactions API. In this way, we have controlled the Api’s execution.

The third parameter, iteratee can be a separate function. This function will accept two arguments, element from the array and callback function respectively.

Here is the Pro-tip.

If you don’t want to break the execution flow in case of any errors, do not return any value to the callback function. For example, if an error occurs while calling the second API and we return this error, the callback will be immediately called skipping the rest of the operations.

I have run into a similar issue and took me an hour to realize what was going wrong.

Conclusion

I have used .eachLimit just for the demonstration. If you want to return value from the iteratee function, you can use async.mapLimit. Everything will be the same except the callback function will accept two arguments errors and results respectively. Async has many more important modules such as filter, reject, parallel, reflect, etc.

Have a great day!