Learn why you’ve probably been sharing code modules the hard way.

Fernando Doglio

·

Follow

Published in

Bits and Pieces

·

11 min read

·

Aug 19, 2019

--

5

Listen

Share

We all know how Node’s asynchronous I/O has made it one of the de-facto tools for developing services (normally REST APIs, although other types, not as popular, are quite common too). And if you’ve worked on any mid-sized (or bigger) project, you’ve probably experienced the process of splitting functionality between individual services.

This practice is great to help keep code untangled, it’s fantastic to allow for partial deployments and lowers the risks of massive catastrophic failure when updating old code (because the amount of damage you can cause is limited to one particular service).

That being said, if you’ve experienced the above, you’ve probably had the issue of having inner custom libraries that need to be shared between your project’s services. For instance: the logging wrapper that you wrote around Winston, or that communication library you wrote to send data between services.

Either way, having to share common code between modules of the same project requires some logistics because what you do not do, is to keep three different versions of the same code, in parallel, inside three different modules. That would be the worst-case scenario and something you can avoid easily.

Also read:

The current ways to solve this problem

I’m sure anyone can come up with alternative ways to handle this problem, but two very common methods of doing it are:

#1 Sharing your code on NPM

Sure, you can take the time to create a separate project for your common code and share it on NPM. At first glance, this might sound like a great idea and the perfect solution, after all, it really fits with the job description of NPM, doesn’t it?

However, if we look at it from the perspective of this being a tiny piece of code from a larger codebase, and you suddenly need to share it with others, there are a couple of reasons why this can be a bit of a problematic approach. Let me elaborate:

1. You might not be using a private registry. You are, after all, sharing your custom code with other co-workers developing the other services for your project, you don’t really want others from around the world to use your code (even less see it, since it can contain custom, proprietary bits that need to remain secret!). And as you probably know, NPM allows you to use private repositories, where you can publish your custom code without publishing it to the world. Setting one up takes time and resources, so even though this might be a solution for this case, it might not be cost-effective if you only have a couple of custom libraries that you’re trying to share among modules.
2. Even if the previous point wasn’t a problem for you, having to extract the code for your common code into a separate project and then properly create and publish the module requires effort. Mind you, it’s not overly complicated to share your modules, but some re-factoring is required, no matter how little or big it might be and you might not have the time or the manpower to do it.
3. The code you extracted is no longer available to you. Yes, it’s inside the node_modules folder, which is a place where very few developers dare to enter (or even know where to look). The point here is that you’ve literally removed the code from your codebase and made it a generic, external entity. This essentially makes it harder to maintain, since it is now a new project, with its own codebase. And that is only if we’re talking about one library, what if you’re extracting three? Or four? Who’s maintaining it and when are they rolling the updates?

Although at first glance this solution can seem like the way to go, it can become cumbersome and awkward to keep on the long run.

#2 Dealing with it using GIT

There are two ways GIT can potentially help you here (and maybe other version control systems can do the same, but I’m not very familiar with them, so I’ll stick to what I know):

1. Moving the generic code into a different repository, essentially doing something similar to what you did with NPM, but now you need to have submodules in your folder. That can’t go wrong, can it?! I mean, raise your hand if you’ve never had issues where one of your teammates (or even you) screwed up the project’s repo using a normal GIT repository, let alone adding submodules to the mix. They’re an elegant solution that GIT provides to solving the issue I’m talking about, but let’s be honest, the tool itself remains too much of a hustle for a lot of devs to even think about adding the extra complexity of dealing with this approach.
2. Instead of adding a new repository, you can bring everything together into a monorepo. After all, everyone’s doing these days! So instead of having your services divided into individual repositories, and your generic modules into yet even more repos, you’ll create a single one, and add everything into it. Depending on your setup, it might be a good idea for you, but think about it: how many individual mini-projects do you need to maintain? And add to them the generic modules, how many is that? To have everything inside a single repository the orchestration needs to be amazingly well executed. And trust me, I’ve done it in the past, it can be done, but I would only recommend it as a last resort.

So, for the time being, let’s leave GIT out of the picture shall we? Or at least, let’s leave it out of the solution and don’t touch if it’s already working for you.

Enter Bit: The new way to share components

Bit is a new service that allows you to share components between projects. This at a first glance, is very much alike NPM, I know, but stay with me for a second, would you?

The concept of components is essentially anything you want to share, be it a single file with a class definition, a set of functions or an entire folder filled with generic libraries. Whatever it is you’re working on and suddenly realized that it can be shared, you can export it to others. Then what is the difference with NPM?

1. For starters, you’re not removing the code from your codebase. I consider this a huge advantage because suddenly, you’re dealing with shared content without having to separate it from the rest of your project. You keep being the maintainer of it, because you created it after all, but at the same time, you’re able to share it as an npm module (I’ll show you how in a second).
2. Your shared code (no matter how many different individual components you’re sharing) remains inside your code repository. You don’t need to add extra orchestration to your code versioning solution in order to have a portion of it shared. If you need to keep updating it, just update the code and have Bit’s CLI tool pull the latest version on the destination project.
3. Unlike NPM, Bit will check the dependency tree of your component. What this means is that if you only choose to share one file, but it requires other local files as dependencies, bit will tell you and allow you to add them as part of the component.

Essentially, bit allows you to solve the problem with a similar approach to #1, but it provides you with the resources you need to actually do it right:

• The private registry is provided by Bit, so you don’t need to worry about that part.
• You don’t need to set up a new npm module to be published, with just a few steps, you can share the code without needed to perform any kind of refactoring.
• Again, you’re not extracting the code from the main project where it was originated. It remains in the same place, and inside the same repository. The impact on the project is almost none.

For an updated version of this tutorial, click here

Using Bit to share your components

How do we share our code then? They have a very detailed documentation page where you can check in more detail, but for the sake of simplicity, I’ll show you how to share a simple component from one project to another with quick steps.

The component I’m going to share is a logger object, which is simply an instance of Winston:

const winston = require("winston")
const config = require("../config.json")const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: config.logging.output_files.error, level: 'error' })
  ]
});
if (process.env.NODE_ENV !== 'production') {
  logger.add(new winston.transports.Console({
    format: winston.format.simple()
  }));
}module.exports = logger

Step 1: Install bit

You can install bit in different ways, but the easiest and most OS generic would be to use npm like so:

$ npm install bit-bin --global

Step 2: Login

Once installed, you need to login or sign-up, this is extremely simple to do, especially if you already have a Github account. From the command line, just type:

$ bit login

That command will launch your browser and open the login page for Bit. Once in there and once you’ve completed the sign-up / login actions, you’ll be able to start setting up what to share.

Note as well that by logging in, you’re adding a new scope to the NPM configuration file. Now you’ll have access to the @bit scope, which will allow you to install components as if they were the classic NPM modules (more on that in a second).

Step 3: Initializing the Workspace

Bit uses the concept of workspace to group collections (which are groups of components). The first thing you need to do, is to initialize your workspace, and you can do that simply typing:

$ bit init

That’s it, once that is done, you’ll be able to start deciding what to share.

Step 4: Configure a compiler

Bit offers various pre-configured compilers for components managed by the workspace. In our case, we will use the Babel compiler component. This will allow us to create a distributable code for our ‘logger’ component, without relying on any specific setup in the authoring environment.

Our component will be easily maintainable by others, in other contexts, thanks to the standardized Babel compiler component used by it.

$ bit import bit.envs/compilers/babel --compiler

Step 5: Adding file & Checking the status of the components

Adding the files you want to share is quite simple. Assuming a project structure like the following:

In order to add files, you can simply do:

$ bit add lib/logger.js

That’ll add the file and create a new component called “logger”. By default, the add command will name the component using the filename, you can check out the full docs for this command to see everything you can do with it.

You can now perform a status check in order to understand if you have everything you need:

$ bit status

The screenshot above shows you the output from the checks performed by bit. This is where the CLI creates and checks the dependency tree for our module. If you look at the code from before, you’ll notice I’m requiring a JSON file which I haven’t added yet. This is one of the benefits of using bit instead of going the npm route, we can avoid missing out important files.

After adding the other file, you can perform a new status check and you’ll get a better-looking response.

Step 6: Versioning

Before uploading the files, you’ll need to tag the component version. This will tag all components, so it’s a great way to initialize them all at the same time:

$ bit tag --all 0.0.1 --message "initial version for the component"

This step is mandatory, you won’t be able to commit anything until you tag the first version.

Step 7: Exporting the component

Once everything above is ready, exporting the components requires you to create the collection where they’re going to live. You do that from their website and once you’ve created it, you can simply do:

$ bit export <account-name>.<collection-name>

In my case, I called the collection “custom-logger” and my account name is “deleteman”, so my command would look like:

$ bit export deleteman.custom-logger

That will upload the file up to the custom registry, without doing anything to your code nor your repo.

Step 7: Using it someplace else (optional)

If you happen to need to use your component on another project, and I’m guessing you’ll need otherwise, what the heck are you doing here? You’ll be able to install it using NPM simply by writing:

$ npm install @bit/<account-name>.<collection-name>.<component-name>

So, in our case:

$ npm install @bit/deleteman.custom-logger.logger

Although I’m referencing one of the components, all dependencies will also be installed, so in order to use it, just require it like this:

const logger = require("@bit/deleteman.custom-logger.logger")logger.info("Testing test!")

Of course, the box where you’re running the later npm install command will have to first be logged in to bit, like I showed you in step #2 (in order to have the bit scope setup for npm and also, to have access to the components since they can also be private).

Conclusion

With these simple steps, you’ve managed to share code from your project, without having to take it out of it, and managed to use it in another one, with minimum effort.

Because this article is already quite long, I decided to leave out optional steps, such as showing you how the interaction between GIT and Bit goes, or how to update the content of a component and its version and how that can be translated to all the related projects. There is a lot more behind these basic steps, but hopefully, this has been enough to show you the benefits of using such a service to share common code between related projects with minimum effort.

Leave a comment down below if you’ve ever used Bit before or if you’ve managed to solve this problem with another approach, I’d love to read about it!

Otherwise, see you on the next one!

Learn More