ReleaseGuard
A release safety workbench that makes the evidence for advancing, holding, or stopping a canary inspectable.
- My contribution
- Product requirements, canary workflow, statistical policy, fault scenarios, and evidence design; implementation and verification developed with Codex assistance.
- Status
- Verified local application
- Built with
- TypeScript · Node.js · React · SQLite

The problem
A release can look healthy on average while making one group of users worse off. Sparse measurements can also look reassuring when they actually provide too little evidence. Stopping the rollout adds another question: did new traffic really stop reaching the candidate?
ReleaseGuard makes those decisions inspectable in a local experiment. It compares a fresh baseline and candidate, checks each required cohort, and verifies withdrawal through the router and subsequent requests.
The workflow
Start with a healthy catalogue service or choose a fault such as a mobile-only slowdown, a configuration error, lost telemetry, or a candidate crash. Inspect the policy before starting. The generator sends real HTTP requests to separate stable, baseline, candidate, and router processes.
The exposure stages are 5% baseline / 5% candidate, then 25% / 25%, then 50% / 50%. The stable service handles the remainder. A stage may become eligible, detect a regression, or remain inconclusive. Advancement always requires an explicit action. The final stage completes the local experiment; it does not promote a production release.
Decisions that matter
- A pass needs evidence. The upper bound on degradation must fit within the practical tolerance, the baseline must be healthy, and every required cohort must have enough complete observations.
- Charts do not authorize extra tests. The policy freezes the sample checkpoints. Conservative binomial intervals account for both outcomes, cohorts, arms, interval tails, stages and checkpoints. p95 is descriptive context.
- Missing measurements hold the release. Lost telemetry cannot become zero errors or zero latency. After the declared grace period, the router withdraws candidate traffic.
- A stop must be observed. Generation checks fence stale commands. Lease expiry protects new traffic if the controller disappears. Withdrawal is confirmed through router acknowledgement and version-tagged baseline requests.
- Earlier evidence stays intact. SQLite history retains the policy, input prefix, judgment and control events. Exported raw requests can reproduce the exact judgment without reproducing live timing.
Measured evidence
The frozen synthetic evaluation includes 200 no-regression simulations and 100 prespecified large regressions, plus a separate boundary suite and correlated-workload stress suite. It produced 0/200 false regression flags, 100/100 large-regression detections, and 0/100 false advancements. Of the healthy simulations, 135/200 completed all stages and 65/200 remained inconclusive. All 60 near-boundary simulations were inconclusive. These are finite synthetic measurements, not a universal safety guarantee.
The statistical record links independent SciPy reference calculations, the frozen manifests, raw observations, uncertainty intervals and initial failed evaluations. The conservative method’s delays are reported alongside its useful detections. An overall-threshold baseline is evaluated against the same captured inputs.
The runtime verification record includes real process tests, repeated live fault scenarios, controller termination during a stop, stale commands, and observed withdrawal timings. The browser workflow was checked at 360, 768, and 1440 pixels, including keyboard operation and automated accessibility checks. Automated testing is not a human usability study.
Watch and reproduce
This 79.72-second recording shows the real local workflow captured on 17 November 2026. It is a recording, not a live hosted application. Captions are visible in the video and available as a separate track.
git clone https://github.com/ngyathin16/releaseguard.git
cd releaseguard
npm ci
npm run build
npm start
Use Node 24.14 or later in the 24.x line, then open http://127.0.0.1:4318. No API key is needed. Verification and evidence-reanalysis commands are in the README.
Scope and limits
The project routes only to its own local services and uses synthetic data. The traffic shares one host; it does not recreate production resource isolation or independent user behavior. The statistical assumptions and conservative inconclusive outcomes remain visible. Rollback moves new requests to the baseline; requests already in flight can finish or time out. There is no external chaos testing, full production promotion, hosted backend, or human participant study.
The problem framing draws on Netflix’s Kayenta canary analysis, Netflix’s discussion of repeated testing, and Meta’s configuration-canary episode. ReleaseGuard does not reproduce their platforms or imply affiliation or endorsement.