An intro to Libraries.io, the universal catalog of open source

Havoc here! I’m one of the co-founders of Tidelift.

Last week we announced that Andrew Nesbitt and Ben Nickolls have joined Tidelift, bringing their Libraries.io project with them.

Libraries.io started as a side project, but Andrew and Ben spent the past year working on it full time thanks to grants from the Ford and Alfred P. Sloan Foundations. We were fortunate to get to know them and talk them into Tidelift as a long-term home for Libraries.io.

What is Libraries.io?

Simply put, Libraries.io is awesome. It’s a catalog of open source, but unlike many catalog-style sites, Libraries.io is completely language and ecosystem-agnostic. It strives to cover open source in its entirety. This means work on Libraries.io can benefit every ecosystem at once, and it means we can map relationships and make comparisons across ecosystems.

What’s in the catalog?

• 28 million code repositories
• 2.5 million packages from 34 package managers
• Every release of all those packages
• The dependency relationships between packages
• All sorts of other details about each package and repo

How can you access it?

• There’s a running web frontend to the database, which gets quite a bit of traffic
• There’s an API
• There are periodic releases of the full data set
• There’s an open source codebase if you’d like to run on your own server

What can you do with Libraries.io today?

With all that rich, networked data there’s a lot of scope for developers, teams, researchers, and open source maintainers to create useful tools and uncover interesting facts. Libraries.io developed some and offers them for free so you can:

• Discover packages you didn’t know about
• Keep track of new package releases via email (if you want to know every time a new Vue.js release comes out, for example)
• Use the periodic data release to run your own analyses of open source projects (We’ve been doing some of that ourselves. Get the data here.)
• Use the API to search for (or grab information about) packages, contributors, and repositories. If you have an idea for an app to explore packages or repositories, you might be able to build it with this. (If you need new APIs or a higher rate limit, get in touch!)
• Look at some of these pre-built queries to find projects that need help:
  • Projects that underpin many others
  • Projects with few contributors
  • Underhyped projects that underpin many others but have few stars on GitHub 

Who is using it?

A number of organizations and companies are building on Libraries.io. Some of them are public, while others are not.

• Bower outsources its search functionality to Libraries.io through the API
• MIcrosoft’s Visual Studio developer environment uses Libraries.io to power its package search
• Glitch uses Libraries.io to power its ‘add package’ system

And how might someone contribute to Libraries.io?

I’m glad you asked! (You were about to ask right?)

There’s a lovely contributor’s handbook with the details you’ll need, including a carefully considered labeling policy that will help you find something meaningful and impactful to work on quickly. But as with any open source project, it probably boils down to scratching your own itch or exploring your own interests.

Personally, I’m hoping there’s interest in adding more package manager support. Most of the big ones are covered, but there are some gaps which you can explore in the issue tracker.

Libraries.io isn’t package-manager-specific; as a side effect of cataloging ALL the open source, Libraries.io has to define a cross-package-manager schema for describing packages.

34 package managers are already supported, and new ones can be added by following these instructions.

If that doesn’t sound exciting to you, it’s just one from a huge backlog of ideas. Andrew and Ben have been fanatical about running the Libraries.io project in the open, so there’s quite a bit of valuable discussion to be found in the GitHub repository.

By the way: consider joining us on Slack to talk about the project.

How does Libraries.io fit in to Tidelift?

We want to connect open source users with open source developers. This happens by way of the package graph, so Tidelift will be built on the Libraries.io data set. More details to come, I promise!

That stuff aside, this is an opportunity to immediately give something back to the open source world by participating in the Libraries.io project and funding the canonical instance.

Perhaps most of all, we’re happy to have Andrew and Ben join us. We want to build a company with a social mission, and they’ve been walking that walk. They’ve also demonstrated their ability to get stuff done and think through big ideas. It’s great to have them here adding insights, perspectives, and of course, running code!

We hope you’ll dig in on Libraries.io, see the promise that it has, and maybe even build some cool stuff around it. Let us know how we can help you.

BTW, Andrew loves poodles—follow him on Twitter to get open source + Poodles in your feed. You can follow Ben too, good open source stuff also, Salukis over Poodles though—fair warning.

Finally, if you’d like regular updates from Tidelift on things like this, sign up here.