With the fanfare surrounding the release of chokidar v3.0.0, I created an innocuous task for myself to update one of my modules to use the brand new versions. And little did I know, this is where a new adventure began.

The update went just fine, as chokidar v3.0.0 is pretty much a drop-in replacement. However, my macOS build started failing. Chalking it up to a sub-optimal design of my module, I began what turned out to be a complete rewrite, far more streamlined than the first version. However, at no point did the macOS build start passing. Curious, I began to investigate and ran across a bug. While it seems chokidar is perfectly happy to watch file when left on its own, as soon as you start doing other work in your application, it no longer fired the ready event. A Windows crashing bug and a Linux CPU bug solidified the issues for me.

Seeing as how watching files is only a feature in my otherwise not file-watching-centric application, I would like the luxury of doing other work while files are being watched in the background. I had to trust that my file watcher has my back, even if I decide to use a few CPU cycles on something else. So like any good engineer, I started wondering "🤔 what goes into watching files?"

Why is watching files hard?

We've all heard it before... watching files is hard and we should lean on the battle-tested libraries to handle it for us. But have you wondered where this story began? It turns out the answer is 2012. Node was still at version 0.6 (version 0.8 to be released later that year), the community was a wild west, and anyone using it was a rebel. Those were the days. (For those wondering, gaze got started that same year as well.)

With such a young ecosystem came various shortcomings. In the world of file watching, support was rather patchy and inconsistent. It didn't report filenames on MacOS (née OSX), events were sometimes (often) reported twice, only a non-useful rename event is present, and recursive watching wasn't really a thing. And that all came from the README from 2012. The story hasn't changed much today, but you would be wrong in assuming that the implementation has not changed either. Since then, Node has started relying on FSEvents for watching files on MacOS, started reporting filenames, added recursive watching on Windows, and gotten many other improvements. Seems that a mere mortal developer like me could even get something done.

Watching files, revisited

Here we go again... someone is trying to reinvent the wheel, or... I guess... watching the wheel?

Hey now! I thought we liked each other. I wouldn't waste your time on just another version of the same old thing. I am talking about watchboy, and he's magnificent!

You see, with 7 years of fixed issues and gained knowledge, we can get some good things done. If a library is going to earn its spot in the ring, it doesn't only need to solve the problem at hand (that is, watching files), but it needs to do it faster, do it with less memory usage, be smaller, be simpler to use, and be backed by more straightforward and easy to understand code. Watchboy is all that and more. That's right, it also shows you a picture of a dog!

Show me the numbers

Watchboy wasn't just built to watch files. It was built for speed, built for low memory usage, built for code simplicity, all right from the very first line of code. Those are all things that are difficult to retrofit after years of only "solving the problem". So let's look at some numbers.

In this first chart, we'll take a look at watching the source code of a pretty small library, React.

Before we jump to conclusions, let's look at watching a larger repository of source code, VSCode itself (which ironically uses chokidar for watching files).

Hmm... this image looks familiar. But let's do one more before we talk about it. Let's look at the source code of a completely diabolical project, Babel. It comes in at 16,847 files in 8,276 directories.

Discussion

Okay, let's get the elephant in the room out of the way. MacOS is so fast! That's gotta be the winner of this benchmark, right? Right? Leave me a comment about this if you want... but be nice, I am sensitive.

Now some real talk: we can see an issue pretty quickly. Gaze does okay on Linux, and chokidar does well on MacOS, but Windows is basically a neglected case. The operating system used by half of developers in 2019 is a second rate citizen (with consumer numbers even higher), up to 5x slower than MacOS, with Linux not holding up much better. Now, based on the results before looking at watchboy, it's easy to think "Windows just sucks... I've even heard people saying that before". But the fact of the matter is that you can't just neglect it. For that matter, you can't neglect Linux either, considering that it has just as many developers using it as MacOS. When you build from the ground-up for speed and consistency, you can make your software behave the same on all operating systems. Looking at the watchboy results, we see far more consistency across all operating systems in time to watch files, with consistently low memory usage.

You may have also noticed that gaze was left out of the last chart. That is because watching that many files results in Maximum call stack size exceeded error.

Show me the code

I must have rigged these results, right? I mean, I am biased after all. I welcome you to take a look at the code and the numbers more closely, and to add your own submissions to the benchmark. The hardware for these tests is standardized to the CI environment, so we can all reproduce the same results.

If you'd like to help make watchboy even better, send a PR my way!