Introduction to Node JS asynchronous programming

NodeJS, by outsourcing Node.js development tasks, has a specific way of managing its operations. According to the official documentation, it is an event-driven asynchronous JavaScript runtime. But what does this mean in more direct terms? Here, I'll address this question and explore best practices to follow when writing asynchronous code.

What is asynchronous programming?

Asynchronous programming means that part of the code can be executed while other parts of the code wait for a response. Let's look at a concrete example: As a web developer, you will deal extensively with API calls: you will write code that sends a request to an external server and waits for a response. The issue is that, firstly, you don't even know if you will get the response you expect, you also don't know how long it will take to execute that function, the situation of the server you are calling, and so on.

Therefore, when you make a request for an API call, you are completely at the mercy of the server state, and your application must stop whatever it is doing and wait for the response, just like your user patiently waiting for the page to load. . Right? Well, obviously not. In modern web development, even milliseconds are counted. We need a way to make our application run smoothly. In other words, when we send a request to an API, we should be able to do something else while we wait for the response. That's where asynchronous programming comes in.

There are several benefits of asynchronous programming. The moment you explore the world of asynchronous programming in NodeJS, you will hear terms like event loop, callback functions, promises, scalability, non-blocking, and so on. All of these terms revolve around the idea of ​​asynchronous programming, the ability to execute code while other parts of the code wait for a response.

In the example above, we were making a hypothetical API call to an external server. Because NodeJS is non-blocking, which means our code can continue with other operations while the previous operation waits for a response. This makes NodeJS very scalable because it can handle many concurrent requests. The most important concept to understand is that in asynchronous programming, in layman's terms, you can perform some operations while other operations are already running. Let's see this in action.

Event cycle

In NodeJs, operations are handled in a single threaded manner. This means that only one operation can be performed at a time. But as we saw above, we can still do something while something else is being processed.

How is this possible? NodeJS has a mechanism called event looping that allows NodeJS to perform non-blocking I/O (input/output) operations, such as a loop that checks the event queue and executes the operations in order. If you visit the official NodeJS documentation, you will learn that the event loop consists of the following phases: timers, pending callbacks, sleep/prepare functions, polling, checking, and closing callback.

In the same link, a brief overview of these phases is provided as such:

    timers: this phase executes callbacks scheduled by setTimeout and setInterval . 
pending callbacks: executes I/O callbacks deferred to the next loop iteration.
 idle, prepare: only used internally.
 poll: retrieve new I/O events; execute I/O related callbacks (almost all with the exception of close callbacks, the ones scheduled by timers, and setImmediate ); node will block here when appropriate.
 check: setImmediate callbacks are invoked here.
 close callbacks: some close callbacks, eg socket.on('close', ...).

Although event looping is a rather complex topic that would require an entire post on its own, the most important thing to understand is that event looping is a mechanism that allows NodeJS to perform non-blocking I/O operations. This means that NodeJS can handle many simultaneous requests and that is why it is so scalable. Let's see an example of how the event loop works.

Since all modern browsers have a Javascript engine, we can use the browser console to see how the event loop works. Now, what if we opened the development tools of our browser of choice and wrote the following?

 console.log("one");
 console.log("two");
 console.log("three");

The browser will record them in order, as you would expect: one, two, and then three. But let's consider another example:

 console.log("one");
 setTimeout( => {
  console.log("two");
 }, 1000);
 console.log("three");

Something different will happen here. Due to the setTimeout method, the browser will register one, three, and just a second later, two. See, the setTimeout function is asynchronous here. While the code follows the queue from top to bottom, when it encounters the setTimeout function, it does not sit still waiting for a second to execute the code. No, it will continue its execution, and only after completion of execution will it check the event queue and see if there are any callbacks to be executed.

In this case, the setTimeout function will be executed after one second. This is the most basic example of how the event loop works. This is the same mechanism that allows NodeJS to handle many simultaneous requests, making it very scalable and fast.

Now that we've discussed the event loop, touching on scalability and the non-blocking nature of NodeJS, let's cover the other common terms we described above, like callbacks, promises, and more.

Callback function

Callback functions in NodeJS are functions that are passed as arguments to other functions and are called once the main function completes. One of the most basic examples you can come across is timeouts. Let's look at an example:

 // Example function that performs an asynchronous operation
 function fetchData(callback) {
 // Simulate a delay
  setTimeout( => {
    const data = { name: "John", age: 30 };
 // Call the callback function with the fetched data
    callback(data);
  }, 3000);
 }
 
// Call the fetchData function and pass a callback function
 fetchData((data) => {
 console.log(data); // { name: "John", age: 30 }
 });

Since NodeJS is just Javascript, we can use this same concept in the browser console to easily understand how callbacks work. So if you paste this code into the browser console, the data will be fetched after 3 seconds. While the data is being fetched, the rest of the code will be executed. Once the data is fetched, the callback function will be called and the data will be printed to the console.

The reason we are providing a setTimeout function is to simulate an API call, for example, and since we don't know how long it will take for the API to return, we need to simulate this delay. But callbacks can get out of control. You may have heard of the term callback hell. Let's see what this means.

Callback Hell

Callback hell is a term used to describe a situation where you have too many nested callbacks. There is even a website called callbackhell.com just to explain this concept. If you visit this page, you will be prompted for the following example:

 fs.readdir(source, function (err, files) {
  if (err) {
    console.log("Error finding files: " + err);
  } else {
    files.forEach(function (filename, fileIndex) {
  console.log(filename);
  gm(source + filename).size(function (err, values) {
    if (err) {
      console.log("Error identifying file size: " + err); 
} else {
 console.log(filename + " : " + values);
 aspect = values.width / values.height;
 widths.forEach(
 function (width, widthIndex) {
 height = Math.round(width / aspect);
 console.log(
 "resizing " + filename + "to " + height + "x" + height
 );
 this.resize(width, height).write(
 dest + "w" + width + "_" + filename,
 function (err) {
 if (err) console.log("Error writing file: " + err);
 }
 );
 }.bind(this)
 );
 }
 });
 });
 }
 });

That's a lot of nested callbacks. As you can see, it's not even that readable. We would like to avoid these things in our code for our own good.

Promises

Now, since asynchronicity is such an important concept in NodeJS, there are a few ways to deal with it. While one of them is callbacks, another option is to use promises. Let's rewrite the same setTimeout example above, but with promises :

 // Example function that returns a Promise
 function fetchData {
  return new Promise((resolve, reject) => {
 // Simulate a delay
    setTimeout( => {
  const data = { name: "John", age: 30 };
 // Resolve the Promise with the fetched data
  resolve(date);
    }, 1000);
  });
 }

 // Call the fetchData function and handle the Promise result
 fetchData
  .then((data) => { 
console.log(data); // { name: "John", age: 30 }
 })
 .catch((error) => {
 console.error(error);
 });

This way it looks a little cleaner. At least we can deal with possible errors. Remember that we have no idea what state the server we are making the request to. There is no guarantee that our request will return the response we expect. Therefore, we need a way to handle possible errors. The catch statement here is a good way to do this. But there is also a caveat here: perhaps we are no longer in callback hell, but we can easily fall into the depths of another hell that is called promise hell, or “then hell.” Here, “then hell” gets its name from having many then statements chained together in the code. Let's see what this means.

Promise Hell

Promise hell is a term used to describe a situation where you have too many nested promises. Let's see an example:

 getData
  .then(function (data) {
    return processData(data)
  .then(function (processedData) {
    return saveData(processedData)
      .then(function (savedData) {
        return displayData(savedData);
      })
      .catch(function (err) {
        console.log("Error while saving data:", err);
      });
  })
  .catch(function (err) {
    console.log("Error while processing data:", err);
  });
  })
  .catch(function (err) {
    console.log("Error while getting data:", err);
  });

This is no better than callbacks. So what is the solution? One way is to use async/await. See how this works.

Asynchronous functions (await)

Async/await is a way to handle asynchronous code synchronously. It's by far my favorite and helps avoid the pitfalls of a callback function and promises. Let's see how this works:

 async function fetchData {
  try {
    const res = await fetch("
    const data = await res.json;
    console.log(data);
    return data;
  } catch (error) {
    console.error(error);
  }
 }

Here, instead of setTimeout, we are making an actual API request. For reference, jsonplaceholder is a dummy API that we can call to test API requests. The above request should return a list of users. Now this asynchronous programming method works as follows: The function must be called async so that we can use the await keyword.

The await keyword is the time when we wait for the asynchronous operation to finish. Once you're done, we can continue with the rest of our code. You will also see that we use try/catch statements, which are extremely useful in this specific case: We can handle errors much easier thanks to this method.

Working with asynchronous code in NodeJS

Embracing the power of asynchronous programming in NodeJS not only speeds up our code but also significantly improves the performance and responsiveness of our applications.

Creating a Basic Express Server

Now, see how we can use asynchronous programming in NodeJS with an example. We will be creating a basic express server and sending a get request to the jsonplaceholder API. We will be using the async/await method for this as it is the most convenient way. To proceed, you will need node and npm installed on your system.

First, we create a new folder called 'async-node' and cd into it. Note that we are using the Linux terminal, so we use the mkdir command to create the folder. Once I'm in the folder, I run npm init -y to initialize a new npm project.

The -y flag is used to skip the questions that npm asks us. Once this is done, I will install the express and axios packages via npm. I will also use nodemon in this project, but I will not install it because I have it installed globally. For ease of use, I recommend you do the same. If you don't want to, you can also install it locally as a development dependency.

To install express and axios, we run the following command:

npm i express axios

For nodemon, if you wanted to install it globally, you would write npm i -g nodemon. Also note that here you may need to provide root access by specifying sudo. If I wanted to install nodemon locally as a development dependency, I would run npm i –save-dev nodemon

Now that I have the packages installed, I can create an index.js file with the touch index.js command and start coding. I will open the project with VsCode via code . command. Once I'm there, I'll go to the package.json file and make some changes there. I will add the following line to the scripts section:

 "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node index.js",
    "dev": "nodemon index.js"
  },

Now whenever we run npm run start or npm run dev it will run the index.js file with the specified packages. Now that we're done with the package.json file, let's go to the index.js file and populate it with a basic express server:

 const express = require("express");

 const app = express;

 app.get(" (req, res) => {
  res.send("Hello, world!");
 });

 app.listen(3000, => {
  console.log("Server listening on port 3000");
 });

Here some things are happening. Firstly, I'm importing the express package and initializing it via const app = express;. Next, we are creating a get request for the root route and sending a response with the text “Hello, world!”. Finally, we are starting the server on port 3000. Now, if I run npm run dev and go to localhost:3000, if everything is correct, we should see the text “Hello, world!”.

Send request with Axios and write response to a file

But we want to do more than just create a basic server. We want to send a get request to an external API and write it to a file. So let's do this. Check the following code snippet:

 const axios = require("axios");
 const fs = require("fs");
 const express = require("express");

 const app = express ;

 app.get(" async (req, res) => {
  try {
 // Make API call using Axios
    const response = await axios.get(
  "
    );
    debugger;

 // Write data to file using asynchronous file system method
    await fs.promises.writeFile("response.txt", JSON.stringify(response.data));

    res.send("Data written to file successfully!");
  } catch (error) {
    console.error(error);
    res.status(500).send("An error occurred");
  }
 });

 app.listen(3000, => console.log("Listening on port 3000"));

In this code, we import Axios and fs along with express. Axios allows us to make API requests, while fs gives access to the file system to read and write files.

After initializing the Express application, we add a GET route handler for the root route. Note the async keyword – this means the route contains asynchronous code.

Inside the handler, we use a try/catch block to gracefully handle any errors. In the try part, we make a GET request to the dummy API using axios and wait for a call to fs.promises to create a new file called response.txt. This file will contain the API response data.

When we run the server with npm run dev and visit localhost:3000, we expect to see the message “Data written to file successfully!” message. But instead we get an error.

Clearly there is a bug that prevents the file from being written correctly. Now it's time to debug our asynchronous code to discover and fix the problem. We will use Node's built-in debugger along with other techniques to trace the execution flow and identify problems.

Debugging asynchronous code like this requires some specialized skills, but learning these skills will allow us to build complex Node applications with confidence.

Debugging Nodejs with Chrome Dev Tools

Debugging is a critical skill for developers. When bugs appear, we need tools to identify the root cause. In this example, our code encounters an error instead of writing data as expected. Fortunately, Node.js has a built-in debugger that we can take advantage of.

To use it, first close the server and run nodemon --inspect index.js . This restarts the server in inspection mode. Then open Chrome and find the Node debugger panel using the green Node.js icon.

In the Sources tab, we see row 10 highlighted where our Axios request is made. This indicates that the error occurs here. We can pause execution at line 10 and inspect variables for more context.

Checking the API endpoint, we identified the problem – there is an invalid “/todos/-1” path. I correct this to “/all/1” and resume execution. Now in the debugger we see a 200 status code on the request – success!

The debugger was invaluable for tracking the execution flow and identifying the root cause. Now with the bug fixed, we restart normally with npm run dev . Visiting localhost, we correctly get the “Data written” message and a response.txt file containing the API data.

Being able to debug asynchronous Node code is crucial for any developer. Mastering the built-in debugger and other debugging workflows will give you the skills to efficiently eliminate bugs in your applications.

Conclusion

Here we discuss asynchronous programming in NodeJs. We've covered a few ways to deal with asynchronicity, such as promises, callback functions, and the relatively new async/await block. We also looked at how to debug our code using nodeJS's built-in debugger. So now it's up to you to incorporate and write asynchronous code in your future projects. However, if you need more assistance or want to scale your project, consider hiring a reputable Node.js development company.

Common questions

What is the difference between asynchronous and synchronous programming?

Synchronous programming is a linear approach where tasks are executed one after another, meaning that one task must be completed before the next can begin. This can make your program easier to understand, but it also means that it may hang or become unresponsive if a task takes too long to complete.

An asynchronous function, on the other hand, allows tasks to be executed simultaneously. If a task takes too long to complete (such as reading a file from a disk or fetching data from the network), the program may continue with other tasks. After the long task is completed, a callback function is typically invoked to handle the result. The non-blocking nature of asynchronous operations makes them particularly suitable for performing tasks within JavaScript code that require waiting for external resources or need to run in the background. This approach can lead to the creation of potentially more efficient and responsive programs.

How does the event loop work in Node.js to handle asynchronous operations?

The event loop in Node.js is the mechanism that handles asynchronous operations. When you hire Node.js developers, they take advantage of the event loop by starting asynchronous tasks like reading files or making database requests. These tasks run outside the event loop, allowing you to continue processing other JavaScript code. When an asynchronous task completes, its callback function is added to a queue, or 'callback queue'. The event loop checks this queue and processes the callbacks one by one, thus handling the results of asynchronous operations efficiently.