CI Tests: Does the project run tests in CI?
ID |
openssf_scorecard/ci_tests |
Severity |
low |
Category |
|
Levels |
|
Optional |
false |
Tags |
supply-chain, testing |
Description
This check tries to determine if the project runs tests before pull requests are merged.
Reference: OpenSSF Scorecard - CI Tests.
Rationale
Running tests helps developers catch mistakes early on, which can reduce the number of vulnerabilities that find their way into a project.
Verification
The check works by looking for a set of CI-system names in GitHub CheckRuns and Statuses among the recent commits. A CI-system is considered well-known if its name contains any of the following: appveyor, buildkite, circleci, e2e, github-actions, jenkins, mergeable, test, travis-ci.
Remediation
-
Check-in scripts that run all the tests in your repository. Build tools often provide direct support for running tests developed under unit testing frameworks.
-
Integrate those scripts with a CI/CD platform that runs it on every pull request (e.g. if hosted on GitHub, GitHub Actions, Prow, etc).
Small Print
Current implementation is limited to repositories hosted on GitHub and Bitbucket, with other systems under development.
| A compliant project using other tools may still receive a low score on this check. There are many ways to implement CI testing, and it is challenging for an automated tool to detect them all. A low score is therefore not a definitive indication that the project is at risk. |