How to Execute Shell Commands With Node.Js

This tutorial will teach you how to execute shell commands, and even your own custom shell scripts with Node.js.

We will learn how to create a program in Node.js to run common commands like ls, mkdir or even npm on your Unix system (Mac or Linux).

If you just want to see the code, you can view it on Github

The Child Process Module

When we execute shell commands, we are running code outside of our Node.js application. In order to do this, we need to run these commands in a child process.

A child process is code that is run in a different thread (and process id) outside your Node.js application. However, we can still send and receive information from the child process using standard I/O streams.

In Node.js, we can use the child_process standard library to start these processes to run shell commands.

Running Basic Shell Commands

To run a simple command and read its output, we can use the exec function.

In this example, let’s list the files in our current directory using ls, and print the output from our Node.js code:

const { exec } = require('node:child_process')

// run the `ls` command using exec
exec('ls ./', (err, output) => {
    // once the command has completed, the callback function is called
    if (err) {
        // log and return if we encounter an error
        console.error("could not execute command: ", err)
        return
    }
    // log the output received from the command
    console.log("Output: \n", output)
})

Since I am running this code within the example repository, it prints the files in the project root:

Output: 
LICENSE
README.md
index.js
package.json

Note that when we run exec, our application spawns a shell ('/bin/sh' by default) and runs the given command on that shell. This means that the command is first processed by the shell and then executed.

So for example, when we run ls ./*.md , the shell processes ./*.md into README.md and then runs the command ls README.md:

We are also limited to 1MB by default as the maximum size of output generated by the command.

Executing Long Running Commands

The previous example executed the ls command that returned its output immediately. What about commands whose output is continuous, or takes a long time to retrieve?

For example, when we run the ping command, we get continuous output at periodic intervals:

➜  ~ ping google.com
PING google.com (142.250.77.110): 56 data bytes
64 bytes from 142.250.77.110: icmp_seq=0 ttl=116 time=11.397 ms
64 bytes from 142.250.77.110: icmp_seq=1 ttl=116 time=17.646 ms  ## this is received after 1 second
64 bytes from 142.250.77.110: icmp_seq=2 ttl=116 time=10.036 ms  ## this is received after 2 seconds
64 bytes from 142.250.77.110: icmp_seq=3 ttl=116 time=9.656 ms   ## and so on
# ...

If we tried executing this type of command using exec, we are limited by the maximum output constraint. Additionally, the exec function callback waits for the command to complete and returns the output after that.

Instead, we can use the spawn function to read output continuously, and without limit:

const { spawn } = require('node:child_process')

// start the `ping google.com` command
const command = spawn('ping', ["google.com"])

// the `data` event is fired every time data is
// output from the command
command.stdout.on('data', output => {
    // the output data is captured and printed in the callback
    console.log("Output: ", output.toString())
})

Output:

Output:  PING google.com (142.250.77.110): 56 data bytes
64 bytes from 142.250.77.110: icmp_seq=0 ttl=116 time=9.550 ms

Output:  64 bytes from 142.250.77.110: icmp_seq=1 ttl=116 time=17.892 ms

Output:  64 bytes from 142.250.77.110: icmp_seq=2 ttl=116 time=9.466 ms

Output:  64 bytes from 142.250.77.110: icmp_seq=3 ttl=116 time=18.592 ms

Output:  64 bytes from 142.250.77.110: icmp_seq=4 ttl=116 time=9.712 ms

By taking an event driven approach, we can capture the output throughout the commands lifecycle, and process it as soon as it is received.

Note that unlike exec, the spawn function does not create a new shell to process the command, and executes it directly.

Another example of a long-running command is npm install - which gives continuous output during the module installation process. In this case as well, it makes sense to give the user continuous feedback as soon as output is received

Passing Input To Commands With STDIN

In the previous examples, we executed commands without giving any input (or providing limited inputs as arguments). In most cases, input is given through the STDIN stream.

One popular example of this is the grep command, where we can pipe the input from another command:

➜  ~ echo "1. pear\n2. grapes\n3. apple\n4. banana\n" | grep apple
3. apple

Here, the input is passed to the grep command through STDIN. In this case the input is a list of fruit, and grep filters the line that contains "apple"

The child process object returned by the spawn function provides us with an input stream which we can write into. Let’s use it to pass input to a grep child process:

const { spawn } = require('node:child_process')

// run the grep command
const command = spawn('grep', ["apple"])

// use the stdin stream from the command to
// send data to the spawned command
command.stdin.write("1. pear\n")
command.stdin.write("2. grapes\n")
command.stdin.write("3. apple\n")
command.stdin.write("4. banana\n")
// once we're done sending input, call the `end` method
command.stdin.end()

// similar to the previous example, print the output whenever it's 
// received
command.stdout.on('data', output => {
    console.log("Output: ", output.toString())
})

Output:

Output:  3. apple

Killing a Child Process

There are several commands that run indefinitely, or need an explicit signal to stop.

For example, if we start a web server using python3 -m http.server or execute sleep 10000 the resulting child processes will run for a very long time (or indefinitely).

To stop these processes, we need to send a kill signal from our Node.js application. We can do this using the kill method of any child process object.

const { exec } = require('node:child_process')

// execute the sleep command - this will
// wait for 100 seconds before exiting
const command = exec("sleep 100")

// when a child process exits, it fires
// the "close" event
command.on('close', (code) => {
    console.log('process has exited')
})

// Since we don't want to wait for 100 seconds,
// we can send a kill command after a 1 second timeout
setTimeout(() => {
    command.kill()
}, 1000)

This will give the following output after 1 second has elapsed:

process has exited

Terminating child processes is useful when you want to limit the time spent in running a command or want to create a fallback incase a command doesn’t return a result on time.

Conclusion

So far, we learned multiple ways to execute and interact with unix shell commands. Here are some things to keep in mind when using the child_process module:

• Use exec when you want to execute simple commands that don’t usually give too much output
• For functions with continuous or long-running output, you should use spawn instead. However, note that you won’t get any of the shell processing (like wildcards or command chaining) that you would get with exec
• In production applications, its useful to keep a timeout and kill a process if it isn’t responding for a given time. We can send termination commands using the kill method to achieve this

If you want to read more about the different functions and configuration options, you can view the official documentation page.

You can view the working code for all examples on Github.