Audit-grade CI for safety-critical ROS2 robotics

Why does safety-critical robotics need different CI?

Ordinary CI answers one question: did the build pass? Teams building cobots, autonomous mobile robots, ADAS, surgical systems or eVTOL also have to prove which requirements were verified, by which tests, on which version of the code, and show that the record has not been altered since. Functional safety standards such as IEC 61508, ISO 26262 and IEC 62304 all expect that kind of evidence in some form.

In many teams that proof lives in spreadsheets. A verification matrix is updated by hand before a release, it drifts the moment the code changes, and the audit turns into a scramble to reconstruct what was actually tested. The fix is to make the pipeline produce the evidence as a side effect of normal development.

What should run on every pull request?

Verification is cheapest when it happens before a change merges, so the pull request is the natural unit of evidence. Each PR should trigger the same checks a release would, and report them where reviewers already look.

• Unit and integration tests for ROS2 nodes, in both Python and C++, with results captured as JUnit XML.
• Static analysis whose findings are exported as SARIF, the standard interchange format for analysis results.
• Simulation scenarios on GPU-backed runners in Gazebo or Webots, recording topics to MCAP so failures can be replayed.
• A traceability check that fails the PR if a requirement loses its covering test or a test points at a requirement that no longer exists.
• Status reported back to the code host, for example as GitHub Check Runs, so a failing safety check blocks the merge like any other.

How do you trace safety requirements to tests?

Traceability holds up best when the link lives in the code, next to the test. A common pattern is to annotate each test with the requirement IDs it confirms, for example with a decorator such as @confirms, so the trace is versioned, reviewed and diffed with the code it describes.

Requirements themselves usually live in an application lifecycle management tool such as Polarion, Jama Connect or IBM DOORS. ReqIF, the Requirements Interchange Format, lets CI import the current requirement set and export verification status back. With a round-trip in place, the requirements team sees test coverage in their own tool, and engineers never edit a matrix by hand.

What makes test evidence audit-grade?

An auditor needs to trust that the evidence is complete, belongs to a specific build and has not been edited afterwards. Logs scattered across a CI dashboard do not meet that bar, especially once retention policies start deleting them.

1. Bundle everything for a run into one evidence pack: JUnit results, SARIF findings, the ReqIF trace and MCAP simulation recordings.
2. Record the exact commit, dependency versions, simulator version and runner environment that produced it.
3. Hash each artifact and chain each hash to the previous one, so changing or removing any item breaks the chain.
4. Make verification possible offline, so an auditor can check the pack with the pack itself, without access to your CI system or cloud account.

Hash chaining does not prove that a test is a good test; that is still the job of reviews and the safety case. What it proves is integrity: the evidence you hand over is exactly what the pipeline produced.

Where does simulation fit in?

Many safety behaviors, such as stopping for a person, recovering from sensor dropout or respecting a speed limit near humans, cannot be tested meaningfully with unit tests alone. Running scenarios in Gazebo Harmonic or Webots on every change catches regressions long before hardware-in-the-loop testing. GPU-backed runners keep those scenarios fast enough to run on every PR, and self-hosted compute helps where models, maps or data must stay on premises.

How did SigmaBit approach this with Roboticks?

SigmaBit helped build Roboticks, an audit-grade Test CI platform for ROS2 robotics that turns safety compliance from a spreadsheet chore into an automated, provable part of the pipeline. It is built in three parts, then rounded out with simulation and SDKs.

• Automated test execution on every pull request through GitHub Check Runs.
• Requirements traceability that maps safety requirements to tests with @confirms decorators, with ReqIF round-trips into Polarion, Jama and DOORS.
• Hash-chained, offline-verifiable Evidence Packs that bundle JUnit, SARIF, ReqIF and MCAP data for regulators.
• Hosted GPU simulation in Gazebo Harmonic and Webots, self-hosted compute, and Python and C++ SDKs.

The result is traceability that cannot go stale and coverage across 29 tier-1 safety standards, with no spreadsheets required. We bring the same rigor to MLOps and DevOps work where models and pipelines have to be provably correct.