OpenSSF Scorecards

Is the project free of checked-in binaries?

The project should not have generated executable (binary) artifacts in the source repository.

Reference: OSSF Scorecard - binary artifacts.

Including generated executables in the source repository increases user risk. Many programming language systems can generate executables from source code (e.g., C/C++ generated machine code, Java .class files, Python .pyc files, and minified JavaScript). Users will often directly use executables if they are included in the source repository, leading to many dangerous behaviors.

Problems with generated executable (binary) artifacts:

The checkpoint compliance fails when there is a single binary executable file in the source repository for the project.

Many language platforms directly run source code (using interpreters or just-in-time compilers): the source code is executable but reviewable, so it is not considered by this check. This includes shell scripts.

Generated source code, like parsers from parser-generation tools from a grammar specification, or annotated source code that is "transpiled" into generated code, is not considered "binary artifact". Generated source code could be more difficult to review than human-written code (some may argue that there are obfuscation-loving programmers out there), and it is recommended to have the tools for re-generating the code from the initial specification.

Generated documentation in source repositories is also ignored. Generated documentation is intended for use by humans (not computers) who can evaluate the context. Thus, generated documentation doesn’t pose the same level of risk.

Non-executable binary files, like images, videos, and other content, are not considered binary executables for this check.

Does the project use branch protection?

This check determines whether a project’s default and release branches are protected with source code repository’s branch protection settings.

Reference: Scorecard Checks page.

Branch protection allows maintainers to define rules that enforce certain workflows for branches, such as requiring review or passing certain status checks before acceptance into a main branch, or preventing rewriting of public history.

Different types of branch protection protect against different risks:

To avoid security issues, branch protection rules could be added. These rules provide the following benefits:

The configuration specifies the minimal protection rules that must be enabled on the configured branches to pass this checkpoint:

Follow the instructions to add protected branches rules in GitHub, GitLab or BitBucket.

Please note that in certain special cases the rules may need to be suspended. For example, if a past commit includes illegal or critical content, it may be necessary to use a force push to rewrite the history rather than simply hide the commit.

For fetching certain protection rules for branches, this checkpoint may need administrative access to the target repository. If the access token provided does not have administrator role, and the associated protecting rule is required for the checkpoint, the checkpoint will not be given a "pass" state, as the status of protection rule cannot be determined.

This check tries to determine if the project runs tests before pull requests are merged.

Reference: OpenSSF Scorecard - CI Tests.

Running tests helps developers catch mistakes early on, which can reduce the number of vulnerabilities that find their way into a project.

The check works by looking for a set of CI-system names in GitHub CheckRuns and Statuses among the recent commits. A CI-system is considered well-known if its name contains any of the following: appveyor, buildkite, circleci, e2e, github-actions, jenkins, mergeable, test, travis-ci.

Current implementation is limited to repositories hosted on GitHub and Bitbucket, with other systems under development.

Does the project have an OpenSSF Best Practices badge?

The Open Source Security Foundation (OpenSSF) Best Practices badge is a way to show that software projects follow recommended best practices, many of them to prevent security issues related to the software supply-chain.

This check determines whether the project has earned a OpenSSF Best Practices Badge, which indicates that the project uses a set of security-focused development best practices for open source software.

Reference: OpenSSF Scorecard - CII-Best-Practices.

The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give full credit to projects that meet the passing criteria, which is a significant achievement for many projects. Lower scores represent a project that is at least working to achieve a badge, with increasingly more points awarded as more criteria are met.

The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold, with increasing security criteria.

To earn the passing badge, the project MUST:

The check uses the URL for the Git repo and the OpenSSF API.

Highest compliance level (and PASS compliance) is given to projects that meet the passing criteria, which is a significant achievement for many projects.

Lower levels (PARTIAL compliance) is given to a project that is at least working to achieve a badge, with increasingly more points awarded as more criteria are met.

FAIL compliance is given to projects that had not enrolled yet with the badge program.

The Core Infrastructure Initiative (CII) was replaced by the Open Source Security Foundation (OpenSSF).

Please note that many criteria in the OpenSSF Best Practices badge program overlap with checkpoints in the OpenSSF Scorecard standard.

Does the project require code review before code is merged?

This check determines whether the project requires code review before pull requests (merge requests) are merged.

Reference: OpenSSF Scorecard - Code Review.

Reviews detect various unintentional problems, including vulnerabilities that can be fixed immediately before they are merged, which improves the quality of the code.

Reviews may also detect or deter an attacker trying to insert malicious code (either as a malicious contributor or as an attacker who has subverted a contributor’s account), because a reviewer might either detect the subversion, including any attempts made by the attacker to obfuscate malicious code or implant an evil dependency.

Lack of code review increase the risk of unintentional vulnerabilities or possible injection of malicious code.

The check first tries to detect whether Branch-Protection is enabled on the default branch with at least one required reviewer or by requiring Approvals from Code Owners. If this fails, the check determines whether the most recent commits have an approved review or if the merger is different from the committer (implicit review). It also performs a similar check for reviews using Prow (labels "lgtm" or "approved") and Gerrit ("Reviewed-on" and "Reviewed-by").

Does the project have contributors from at least two different organizations?

This check tries to determine if the project has recent contributors from multiple organizations.

Reference: OpenSSF Scorecard - Contributors.

This check provides insight into which organizations have contributed, so that a trust-based decision based on that information can be made.

The check looks at the Company field on the GitHub user profile for authors of recent commits. To receive the highest score, the project must have had contributors from at least 3 different companies in the last 30 commits; each of those contributors must have had at least 5 commits in the last 30 commits.

Contributors should join their respective organizations, if they have not already. Otherwise, there is no remediation for this check.

It is currently limited to repositories hosted on GitHub, and does not support other source hosting repositories.

Does the project avoid dangerous coding patterns in CI workflows?

This check determines whether the project’s GitHub Action workflows has dangerous code patterns. Some examples of these patterns are untrusted code checkouts, logging github context and secrets, or use of potentially untrusted inputs in scripts.

Reference: OpenSSF Scorecard - Dangerous Workflow.

Using dangerous coding in CI workflows make the repository vulnerable to compromise. Attackers might access repository secrets or run build scripts controlled by the author of a PR.

The following patterns are checked:

Avoid the dangerous workflow patterns. See this post for information on avoiding untrusted code checkouts. See this document for information on avoiding and mitigating the risk of script injections.

Current implementation is limited to repositories hosted on GitHub, with other systems under development.

Does the project use tools to help update its dependencies?

This check tries to determine if the project uses a dependency update tool. These tools automate the process of updating dependencies by scanning for outdated or insecure requirements, and opening a pull request to update them if found.

Reference: OpenSSF Scorecard - Dependency Update Tool.

Out-of-date dependencies make a project vulnerable to known flaws and prone to attacks.

There are both open-source and commercial tools available for Software Composition Analysis (SCA) for the process of identifying potential areas of risk from the use of third-party software components.

Dependency Update tools automate part of the dependency-update process by integrating with the SCM, detecting out-of-date or vulnerable dependencies, and creating a pull request with the change in project dependencies descriptors, that could be accepted for merge.

The check looks for well-known tools in use, specifically dependabot or renovatebot.

For GitHub, see instructions for dependabot or renovatebot.

There are many ways to implement dependency updates, and it is challenging for an automated tool to detect them all. A FAIL status is therefore not a definitive indication that the project is at risk.

Does the project use fuzzing tools?

This check tries to determine if the project uses fuzzing.

Fuzzing tools inject semi-random data into a program to detect bugs. They often use known-to-be-dangerous values (fuzz vectors like random long sequences, extreme values, out-of-bound values, or metacharacters) for each input type, and check the program response.

Reference: OpenSSF Scorecard - Fuzzing.

Fuzzing, or fuzz testing, is the practice of feeding unexpected or random data into a program to expose bugs. Regular fuzzing is important to detect vulnerabilities that may be exploited by others, especially since attackers can also use fuzzing to find the same flaws.

For certain languages (C is the archetype) where the memory is managed by the developer, fuzzing can uncover memory-handling and type-mismatch bugs like stack- and heap- overruns, integer overflows, attacks on format strings, off-by-one vulnerabilities…​

This check tries to determine if the project uses fuzzing by checking:

As fuzzers are more relevant for certain languages, the set of languages that should be considered by the check could be configured.

Example: for OSS-Fuzz over GitHub follow the instructions here.

There are many fuzzer tools and ways to run in the build and CI tools, and it is challenging for an automated tool to detect them all. A FAIL status is therefore not a definitive indication that the project is at risk.

Does the project declare a license?

This check tries to determine if the project has published a license.

Reference: OpenSSF Scorecard - License.

A license can give users information about how the source code may or may not be used. The lack of a license will impede any kind of security review or audit, and creates a legal risk for potential users.

The check looks for a file named according to common conventions for licenses, under standard locations.

This check will detect files in the top-level directory with any combination of the following names and extensions: LICENSE, LICENCE, COPYING, COPYRIGHT and .html, .txt, .md. It will also detect these files in a directory named LICENSES. (Files in a LICENSES directory are typically named as their SPDX license identifier followed by an appropriate file extension, as described in the REUSE Specification.)

Is the project maintained?

This check determines whether the project is actively maintained.

Reference: OpenSSF Scorecard - Maintained.

A project which is not active might not be patched, have its dependencies patched, or be actively tested and used. It might hold possibly unpatched vulnerabilities.

However, a lack of active maintenance is not necessarily always a problem. Some software, especially smaller utility functions, does not normally need to be maintained. For example, a library that determines if an integer is even would not normally need maintenance unless an underlying implementation language definition changed.

A lack of active maintenance should signal that potential users should investigate further to judge the situation.

If the project is archived, it receives the lowest score with FAIL compliance.

The activity considered on the project during the previous period (of 90 days by default) is:

If the activity per week (number of commits or issue changes) exceeds the minimum activity threshold per week, the project receives a PASS with maximum score.

If the activity is below this threshold, the project receives a PARTIAL compliance.

There is no remediation work needed from projects not passing this checkpoint. The check simply provides insight into the project activity and maintenance commitment.

External users should determine whether the software is the type that would not normally need active maintenance.

Does the project build and publish official packages from CI/CD?

This check tries to determine if the project is published as a 'package' or 'deployment unit'.

You can create a 'package' or 'deployment unit' in several ways:

Reference: OpenSSF Scorecard - Packaging.

Packages give users of a project an easy way to download, install, update, and uninstall the software by a package manager. In particular, they make it easy for users to receive security patches as updates.

When a project does not distribute updates via a package manager, users may miss security updates.

The check currently looks for CI workflows and language-specific commands that upload the package to a corresponding hub, like Npm or PyPI.

Some SCM platforms have package hubs often used for internal components, but foss components are typically deployed into public language-specific repositories like Npm, PyPI, Maven Central…​ There are also commercial tools for enterprise software repositories. like the popular SonaType Nexus or JFrog Artifactory.

Example: if the project is hosted on GitHub, use GitHub’s mechanisms for publishing a package. In case you need to publish a package in GitHub package registry, follow the instructions here.

There are many ways to package software, and it is challenging for an automated tool to detect them all. A FAIL status is therefore not a definitive indication that the project is at risk.

Does the project declare and pin dependencies?

This check tries to determine if the project pins its dependencies. A pinned dependency is a dependency that is explicitly set to a specific one instead of allowing a mutable version or range of versions.

Reference: OpenSSF Scorecard - Pinned Dependencies.

Pinned dependencies reduce several security risks:

However, pinning dependencies can inhibit software updates, either because of a security vulnerability or because the pinned version is compromised.

Mitigate this risk by:

The check works by looking for unpinned dependencies in dependency descriptors.

Does the project use static code analysis tools?

This check tries to determine if the project uses Static Application Security Testing (SAST), also known as static code analysis.

Reference: OpenSSF Scorecard - SAST.

SAST is analyzing source code before the software is run. This means that the source code can be checked for bugs before it is integrated or complete and ready for delivery.

Using SAST tools can prevent known classes of bugs from being inadvertently introduced in the codebase. Many SAST tools are specialized in catching security flaws; while others are more generic and look for generic defects in source code and configurations.

Lack of SAST increase the risk of unknown bugs (and security vulnerabilities for the security-focused tools) in the delivered software.

The check looks for execution of known SAST tools in the recent merged PRs, or the usage of these tools in CI workflows.

Follow the instructions of the particular SAST tool for invocation in the project’s CI workflows.

There are many SAST tools and ways of invoking them, and it is challenging for an automated tool to detect them all. A FAIL result is therefore not a definitive indication that the project is at risk.

Does the project contain a security policy?

This check tries to determine if the project has published a security policy.

Reference: OpenSSF Scorecard - Security Policy.

At some point in the life of any software project, someone (a user, a contributor, or a security researcher) will find a vulnerability that affects the safety and usefulness of the software.

A security policy (typically a SECURITY.md file) can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible.

Such security policy should document at least:

This check works by looking for a file named SECURITY.md (case-insensitive) in a few well-known directories.

Does the project cryptographically sign releases?

This check tries to determine if the project cryptographically signs release artifacts.

A Release Signature is a cryptographic digital signature of a digest of the release contents, that could be verified by consumers of the release (e.g. a tarball) by using the publicly available public key of the signer.

OpenPGP and variants like GnuPG (GPG) are common public key frameworks used for this purpose. Other tools, like minisign, offer popular alternatives for signing the tarball of a release, either in source or in binary form.

There are other tools for digital signatures, like openssl, or code signing certificates (where public keys are signed by a trusted Certificate Authority).

Reference: OpenSSF Scorecard - Signed Releases.

Signed releases attest to the provenance of the artifact.

This check looks for the following filenames in the project’s last (five) releases: *.minisig, *.asc (pgp), *.sig, *.sign.

Current implementation checks signatures for releases in GitHub only. Support for additional SCM platforms is under development.

Does the project declare tokens in CI/CD as read-only?

This check determines whether the project automated workflows' tokens are set to read-only by default.

Reference: OpenSSF Scorecard - Token Permissions.

Setting token permissions to read-only follows the principle of least privilege. This is important because attackers may use a compromised token with write access to push malicious code into the project.

The highest compliance level is awarded when the permissions definitions in each workflow’s yaml file are set as read-only at the top level and the required write permissions are declared at the run-level. One point is reduced from the level if all jobs have their permissions defined but the top level permissions are not defined. This configuration is secure, but there is a chance that when a new job is added to the workflow, its job permissions could be left undefined because of human error.

Additionally, points are reduced if certain write permissions are defined for a job.

Write permissions causing a small reduction to compliance level:

Write permissions causing a large reduction to compliance level:

For GitHub:

Current implementation checks signatures for releases in GitHub only. Support for additional SCM platforms is under development.

Does the project have unfixed known vulnerabilities?

This check determines whether the project has open, unfixed vulnerabilities.

Reference: OpenSSF Scorecard - Vulnerabilities.

Having unfixed known vulnerabilities increases the risk, proportionally to how much work and expertise are required for a successful exploit of the vulnerability, and its impact on the systems where the software is deployed.

An open vulnerability can be readily exploited by attackers and should be fixed as soon as possible.

The default configuration for the check uses the Open Source Vulnerabilities (OSV) or NIST National Vulnerabilities Database (NVD) databases. This is relevant for open-source projects.

As private projects typically do not publish detected vulnerabilities in OSV, the check could be configured with a URL pattern plus a 'unfixed vulnerabilities found' response pattern that could help with custom security advisories. But this may not work to detect unfixed vulnerabilities if your security advisories registry does not follow a structured format.

It is not easy, in a generic way, to detect unfixed vulnerabilities in non-standard vulnerability databases. This check may have 'false negatives': vulnerabilities that are unknown or known and unfixed but that cannot be found in the configured database(s).

Does repository webhooks have a token for authenticating the origin of requests?

Webhooks allow you to build or set up integrations that subscribe to certain events in the project repository. When one of those events is triggered, the webhook’s configured URL will receive an HTTP request with information about the event.

This check determines whether the webhook is secure or not, by taking into account:

If not authenticated, the HTTP payload that your integration will receive could be a faked one generated by a bad actor.

A common way to perform such authentication is to share a secret token that could be registered with the webhook. When the SCM invokes the webhook URL, it sends a header with an message authentication code (MAC) computed over the HTTP payload with the secret token. Your integration should compute the same MAC and compare with the received header. This is documented here for GitHub.

The behaviour of this detector varies depending on the SCM/ CI/CD system which declares the webhook (- means not supported, x means supported):

Current implementation checks signatures for releases in GitHub only. Support for additional SCM platforms is under development.