Last week, in a response to the ever-growing list of software supply chain attacks (SolarWinds and Log4Shell specifically), the U.S. National Security Agency (NSA) partnered with the Cybersecurity and Infrastructure Security Agency (CISA) and the Office of the Director of National Intelligence (ODNI) to release a report entitled Securing the Software Supply Chain for Developers.
Here are some of the report’s specific recommendations relating to developing applications with open source:
- Organizations should maintain a central repository of approved third-party software that tracks vulnerabilities and updates in components, and notifies developers using components when there are issues (section 2.1 “Secure Product Criteria and Management” and section 2.2.4 “Code Integration”)
- Organizations should employ dedicated systems that perform recurring checks of open source libraries for new versions, updates, and known or new vulnerabilities (section 2.2.2 “Open Source Management Practices”)
- Organizations should consider not just a scan of third-party software artifacts, but also, importantly, “the development process, quality, and security” practices of the software supplier (section 2.3.3 “Obtain Components from a Known and Trusted Supplier”)
While the report highlights several roles for software-only point solutions, the bigger picture it paints is that organizations fundamentally need a comprehensive strategy and process for managing open source, including active engagement of software suppliers—which in the case of open source, means the independent community maintainers behind so many of the projects we rely on.
In this post, we highlight some of the report’s key insights, with a Tidelift point of view on each.
#1 Release criteria should include stable versions of all open source code
Section 2.1 of the report on “Secure product criteria and management” says that open source software incorporated into applications should be validated to meet defined standards, and that organizations should carefully manage the release cycles of third-party projects:
“All shipping of open source meets company-wide standards including vulnerability assessment of the source and this information is made available to development groups. Ship the latest stable versions of open source, removing or providing a support plan for any open source software that has reached end of life, and ensuring licensing, if any, is fully understood and compliant with the open source usage policy.”
An important point here is that in the case of open source packages, assessing vulnerabilities and having a plan for future support and maintenance requires active partnership with the independent creators behind them. We have previously discussed this in more detail in the post: Software bills of materials are important—but they won't work at scale if we don't pay the maintainers.
#2 A well-balanced authenticated source control check-in process must be implemented
In section 2.2.1 “Modification or Exploitation of Source Code by Insiders,” the report recommends the use of a secure software repository for managing third-party open source components:
“As an example, the acquisition processes for free and open source software, commercial off the shelf software source components, and the management of a secure software repository are outlined in Figure 3.”
“The secure repository should initially and continuously look for new vulnerabilities and updates within the added components. A log of all developers and the components they download should be kept. If a component becomes flagged due to a new vulnerability or update in the future, the developers who have downloaded the component should be automatically notified to address the issue. In this manner, when new vulnerabilities arise, it will also be evident which programs/projects are affected.”
On this point too, we strongly agree. The Tidelift Subscription allows you to set up such a secure repository, in the form of a catalog for your organization. The catalog represents all of the open source packages and package versions approved and denied for use in your organization’s production environment.
#3 Open source management best practices are necessary
In section 2.2.2 “Open Source Management Practices,” the report further emphasizes the need for dedicated methods to validate the standards that open source components meet:
“Developers commonly use open source code in application development, with projects potentially having multiple dependencies on open source libraries which may contain vulnerabilities.
“Development organizations should employ dedicated systems that download, scan, and perform recurring checks of open source libraries for new versions, updates, and known or new vulnerabilities (see Figure 3 above). As with all software, we strongly recommend educating developers on considerations for the use of open source software, close-source software, and evolving best-practice mitigations.”
An important point to take away from this section is that organizations need not just notifications about new security vulnerabilities, but also a closed-loop process for their development teams to effectively respond to zero day vulnerabilities. We’ve covered our recommended best practices for managing open source in The Tidelift guide to managing open source for application development teams.
#4 All third-party modules should be tested for known vulnerabilities
In section 2.2.4 “Code Integration,” the report appropriately calls for careful monitoring of known security vulnerabilities to be incorporated into the previously discussed trusted repository:
“All third-party modules should be tested for known vulnerabilities against the Common Vulnerabilities and Exposures (CVEs) that are listed in the National Vulnerability Database (NVD). A software composition analysis tool can help automate this process. The development team should also subscribe to alerts and reports from the National Cyber Awareness System and other sources for the latest software vulnerability information. Once modules are reviewed for vulnerabilities, they can be added to a developer or Open Source Review Board (OSRB) repository for all approved downloaded modules. This trusted repository should continue ongoing testing to identify new vulnerabilities that are reported within the modules. It should also incorporate a process to provide vulnerability updates and/or patches to the end-user. Applications include code from other sources, sometimes slightly modifying or adding integration code for their specific use purpose.”
Some good news on this front: Tidelift’s research has found that the best practice of centrally managing a repository of approved open source components is growing rapidly. IDC also reports that 67% of organizations are using internally curated repositories today (moderate, extensive, or strategic use), growing to 82% in 12 months.
#5 Developers must verify third-party components
In section 2.3 “Verify Third-Party Components,” the report highlights the role of open source components and calls for various kinds of evaluation prior to including third-party packages in applications:
“Developers routinely incorporate third-party commercial software components as an aspect of their activities to leverage existing Application Programming Interface (API) capabilities. These components may be Free Open Source Software (FOSS) or Commercial Off-the-Shelf Software (COTS). When sourcing these components, developers will typically make their selection based on criteria including the capabilities the component enables and the sustaining engineering support model for the component. Prior to the incorporation of third-party components by engineers, an organization may require an approval process as outlined in Section 2.2.4, Code Integration. This process may include vulnerability database analysis, secure composition analysis, vulnerability analysis, risk assessment, and source code evaluation on the components under consideration, the results of which indicate whether the specific components identified are allowed or not. Once selected, the identified components are continually monitored, if possible, by using an automatic vulnerability tracking service that prioritizes and fixes identified vulnerabilities within open source components.”
This recommendation to verify third-party components is well advised. However, the workload involved in checking all of these boxes—vulnerability analysis and risk assessment, source code evaluation, continual monitoring—is non-trivial and requires active partnership with the upstream creators behind open source projects, as we explored in Log4Shell highlights the need to proactively cooperate with open source maintainers at scale.
#6 Components must be obtained from a known and trusted supplier
In section 2.3.3 “Obtain Components from a Known and Trusted Supplier,” the report calls for attestation of software development process behind open source components:
“When the organization makes decisions concerning selection, use, changes, or updates of third-party or open source software for its products, it should perform a risk assessment and ensure the residual risks are acceptable […] Suppliers should produce artifacts attesting to the development process, quality, and security aspects of the component being considered for inclusion in an organization’s software product.”
This mirrors guidance from the White House cybersecurity executive order 14028 on application development teams that requires “ensuring and attesting, to the extent practicable, to the integrity and provenance of open source software used within any portion of a product.”
According to a recent Tidelift survey, only 15% of organizations are extremely confident that the open source components they are using are up-to-date, secured, and well maintained.
This raises an important question: if you are part of the 85% of organizations that aren’t extremely confident, would you be willing to personally attest to the development process, quality, and security aspects of the open source components in your supply chain?
#7 Software bills of materials (or SBOMs) can help validate approved components
In section 2.3.5 “Software Bill of Materials (SBOM),” the report discusses the important role of SBOMs:
“The details of an integrated third-party component should be reported in an SBOM for the product developed to easily validate approved components and identify the presence of vulnerable components when defects are discovered.
However, it’s somewhat at odds with the recent U.S. Cyber Safety Review Board Report on the Log4Shell vulnerability, which included the surprising finding that organizations using SBOMs didn’t have a significant advantage in their response to Log4Shell.
Our opinion: SBOMs are necessary, but not sufficient, to ensure open source software supply chain security. Dive deeper on this topic in Tidelift’s take on the CSRB report.
Conclusion:
The new Securing the Software Supply Chain for Developers report from NSA, CISA, and ODNI is another important contribution to the conversation around software supply chain security broadly, with a healthy focus on the role of open source software in particular.
We commend the report for its focus on three key strategies for ensuring open source software supply chain security:
- Central repositories of approved third-party software
- Dedicated systems to perform recurring checks that open source components comply with objective standards
- Active partnership with suppliers to validate secure development practices
At the end of the day, the report makes it clear that software tools alone are not enough. Effective open source software supply chain security requires a combination of tools, organizational processes, and most importantly people—including the independent upstream maintainers—to have a real impact on improving the security of the open source we all rely on.
50 Milk St, 16th Floor, Boston, MA 02109