Let’s extend continuous integration to our open source dependencies

Over the past 5-10 years, the software development world has fully embraced the idea of continuous integration.

It’s not a new idea at all—continuous integration can easily be tracked back twenty years to Kent Beck’s book Extreme Programming where it was one of the twelve practices to help a software development team develop higher quality software while also having a good quality of life. 

The general idea behind continuous integration is that you want to set up systems where you are constantly testing changes immediately as they are introduced into the build. Daily builds, which had been the previous Best You Could Hope For, were no longer a fast enough feedback loop to allow for the development of quality software. 

Prior to Kent Beck’s book, developers avoided doing this integration because of the conflicts and problems that were uncovered. Extreme Programming instead suggested that “if it hurts, do it more often.” This approach leads to an environment where you find problems faster and also get a lot better at identifying problems and being able to fix them quickly. 

But with our move to the cloud and increased focus on developer productivity, it’s become a practice adopted not just by niche early adopters but instead a defacto part of software engineering best practices. By continuously integrating our code, we are forced to do things such as having our code in a version control system, automating our builds, having automated testing as part of our builds, and having everyone commit early and often. So if you’re in a company and have a change pending, you’re regularly pulling in the changes from the rest of your team as you do your development and ensuring that the code continues to work.

That’s well and good while we’re talking about the code being built inside your organization. But as we have covered previously, the application code being developed within your organization is only 20% of the application code. If you’re like most development teams, there are a wide swath of frameworks and dependencies that you’re using which are open source. And we treat these libraries as precious and unable to be upgraded. An upgrade of these components is treated with a fear that is far beyond that which we attribute to the rest of the code being built by our team.

Why is that? 🤔🤔🤔

Some of it just stems from history. When you purchase proprietary software, often you purchase a specific version of that software. Upgrading to a new version requires you to work with procurement to buy the new version. That friction and the relative lack of frequency of updates have meant that we just aren’t used to doing these sorts of upgrades on a regular basis for code that is not our own.

Another—and perhaps larger part—is just a matter of comfort. We don’t know when the changes happen to code we don’t control. We don’t have a good way to understand why there are changes. We don’t know the people who made the changes. So the default assumption is that they aren’t making changes that will benefit us.

It’s time for us to work differently 

Let’s start upgrading our dependencies early and often. When an open source package we depend on is updated, we should update our application to use the new version. By bringing in the changes more often, the cost of integrating each change will be lower, just like the cost of integrating the code from other parts of your organization is lower. 

And then, when there is a major security vulnerability, the cost will be far lower to your organization. First because the amount of change will be reduced and thus the amount of work to validate that there aren’t problems for you. But just as importantly, because as an organization, you will be more used to the process of integrating the updates.

I’d be remiss if I didn’t point out that Tidelift helps to make this process even easier. With a Tidelift Subscription, you will get information directly from the maintainers about what has changed in each new version of the packages that you depend on. 

We also work with the maintainers to help provide recommendations about what version you should upgrade to given what you currently use. And in cases where there are incompatibilities between versions of different software that you depend on, we will work with the maintainers to resolve the incompatibility. 

The result? Open source libraries you can depend on the same way you depend on commercial software. If you are interested in learning more about how Tidelift can make your continuous integration of open source software run more smoothly, check out our free dependency analyzer. We’ll give you some stats on the current health of your dependencies as well as three actionable things you can do to improve them right away.