12 min read

Translating Health Scores into SLAs

A composite health score is a diagnostic number until someone attaches a consequence to it. This page is part of Aligning Audit Goals with Business KPIs and shows the mechanical step most teams skip: turning a score into a Service Level Objective with a real error budget, then wiring the budget's burn rate to a decision a release pipeline can actually enforce. Without that wiring, a health score dashboard is just another chart nobody is accountable to.

Health Score to Release Gate Flow Diagram showing daily score history flowing into SLI computation, then burn-rate calculation, then a gate decision box that branches into a Ship path when burn is under budget and a Block path when the error budget is exhausted. Score History daily snapshots Compute SLI vs SLO target Burn Rate budget consumed Gate Decision window-level check burn < budget burn ≥ budget Ship budget remaining Block budget exhausted

Environment Isolation and Dependency Declaration

Pin the SLO parameters as environment variables, not literals buried in a script. The same config must be readable by the CI pipeline that gates a release and by whatever dashboard reports remaining budget to stakeholders — if the two read from different copies of the number, the SLA is not enforceable.

# /opt/audit/sla_gate — absolute working directory
export PYTHONPATH=/opt/audit/sla_gate
export SLO_TARGET_SCORE=90          # composite health score a day must meet or exceed
export SLO_OBJECTIVE_PCT=95         # percent of days in the window that must be compliant
export SLO_WINDOW_DAYS=28           # rolling measurement window
export SLA_GATE_FAIL_ON_EXHAUST=1   # exit non-zero when budget is exhausted
set -euo pipefail
cd /opt/audit/sla_gate
python3 -m venv .venv
.venv/bin/pip install --quiet pandas==2.2.2 numpy==1.26.4

SLO_OBJECTIVE_PCT and SLO_WINDOW_DAYS together fix the error budget size: at 95 percent over 28 days, 1.4 days are allowed to fail before the budget is exhausted. Changing either value mid-window invalidates the current budget calculation — treat both as versioned config, reviewed the same way you would review a threshold change in configuring alert thresholds and routing.

Implementation

The script below reads a daily score-history table, computes compliance against the SLO target, converts non-compliant days into error-budget consumption, calculates the burn rate for the current window, and emits a gate decision a CI job can act on directly through its exit code.

#!/usr/bin/env python3
# /opt/audit/sla_gate/error_budget_gate.py
"""
Convert daily composite health-score history into an SLO compliance check,
an error-budget burn rate, and a release gate decision.
Reads SLO_TARGET_SCORE, SLO_OBJECTIVE_PCT, SLO_WINDOW_DAYS from environment.
"""

import os
import sys
import json
import pandas as pd

# ── 1. load environment-injected SLO parameters ──────────────────────────────
SLO_TARGET_SCORE = float(os.environ["SLO_TARGET_SCORE"])   # e.g. 90.0
SLO_OBJECTIVE_PCT = float(os.environ["SLO_OBJECTIVE_PCT"])  # e.g. 95.0
SLO_WINDOW_DAYS = int(os.environ["SLO_WINDOW_DAYS"])        # e.g. 28
FAIL_ON_EXHAUST = os.environ.get("SLA_GATE_FAIL_ON_EXHAUST", "1") == "1"

# ── 2. load daily score history, most recent SLO_WINDOW_DAYS only ───────────
# expected columns: date (ISO), composite_score (0-100)
history = pd.read_parquet("/opt/audit/sla_gate/input/score_history.parquet")
history["date"] = pd.to_datetime(history["date"])
history = history.sort_values("date")

window = history.tail(SLO_WINDOW_DAYS).copy()
if len(window) < SLO_WINDOW_DAYS:
    # a partial window cannot support a defensible compliance percentage —
    # fail loudly rather than silently reporting on fewer days than promised
    print(f"WARN: only {len(window)}/{SLO_WINDOW_DAYS} days of history available")

# ── 3. mark each day compliant or not against the SLO target ────────────────
window["compliant"] = window["composite_score"] >= SLO_TARGET_SCORE
days_total = len(window)
days_compliant = int(window["compliant"].sum())
days_noncompliant = days_total - days_compliant

# ── 4. size the error budget and compute consumption ─────────────────────────
# allowed non-compliant days = (1 - objective) * window length
allowed_noncompliant_days = (1 - SLO_OBJECTIVE_PCT / 100) * SLO_WINDOW_DAYS
budget_consumed_days = days_noncompliant
budget_remaining_days = allowed_noncompliant_days - budget_consumed_days

# ── 5. burn rate: consumption rate relative to elapsed time in the window ───
# a burn_rate of 1.0 means budget will be exhausted exactly at window end;
# above 1.0 means the budget will run out before the window closes
elapsed_fraction = days_total / SLO_WINDOW_DAYS
consumed_fraction = (budget_consumed_days / allowed_noncompliant_days
                     if allowed_noncompliant_days > 0 else float("inf"))
burn_rate = consumed_fraction / elapsed_fraction if elapsed_fraction > 0 else 0.0

# ── 6. gate decision ──────────────────────────────────────────────────────────
gate_status = "BLOCK" if budget_remaining_days <= 0 else "SHIP"

decision = {
    "window_days": SLO_WINDOW_DAYS,
    "slo_target_score": SLO_TARGET_SCORE,
    "slo_objective_pct": SLO_OBJECTIVE_PCT,
    "days_compliant": days_compliant,
    "days_noncompliant": days_noncompliant,
    "allowed_noncompliant_days": round(allowed_noncompliant_days, 2),
    "budget_remaining_days": round(budget_remaining_days, 2),
    "burn_rate": round(burn_rate, 2),
    "gate_status": gate_status,
}

out_path = "/opt/audit/sla_gate/output/gate_decision.json"
with open(out_path, "w") as f:
    json.dump(decision, f, indent=2)

print(json.dumps(decision, indent=2))

# ── 7. exit non-zero on exhaustion so a CI job blocks the release ───────────
if gate_status == "BLOCK" and FAIL_ON_EXHAUST:
    print("FAIL: error budget exhausted — release gate BLOCK", file=sys.stderr)
    sys.exit(1)

Lines 18–25 restrict the calculation to exactly SLO_WINDOW_DAYS of history — an SLA measured over a shifting or ambiguous window is not defensible, so the script prints a warning rather than silently averaging over whatever rows happen to exist. Lines 28–31 do the actual compliance check: a day either meets SLO_TARGET_SCORE or it does not, there is no partial credit. Lines 34–37 convert the objective percentage into an absolute day count, which is the number stakeholders should see quoted in the SLA document, not the raw percentage. Lines 41–45 compute burn rate the same way a multiwindow SRE alert would — as a ratio of consumption rate to elapsed-time rate, so a value above 1.0 is an early warning that the budget will run out before the window closes even if today looks fine. The exit code on line 62 is what makes this a gate rather than a report: a CI step that runs this script and checks $? blocks the deploy automatically.

Verification and Smoke Test

Run the gate against a known-good and a known-bad window before wiring it into a real pipeline, so a broken threshold config fails in a test run and not in production.

set -euo pipefail
cd /opt/audit/sla_gate

# 1. run against a synthetic all-compliant window — must SHIP
.venv/bin/python -c "
import pandas as pd
dates = pd.date_range('2026-06-01', periods=28)
pd.DataFrame({'date': dates, 'composite_score': 96}).to_parquet('input/score_history.parquet')
"
.venv/bin/python error_budget_gate.py && echo "PASS: all-compliant window ships"

# 2. run against a synthetic exhausted-budget window — must exit 1 (BLOCK)
.venv/bin/python -c "
import pandas as pd
dates = pd.date_range('2026-06-01', periods=28)
scores = [96]*24 + [70]*4   # 4 non-compliant days > 1.4 day allowance
pd.DataFrame({'date': dates, 'composite_score': scores}).to_parquet('input/score_history.parquet')
"
if .venv/bin/python error_budget_gate.py; then
  echo "FAIL: expected BLOCK, gate returned SHIP"
else
  echo "PASS: exhausted budget correctly blocks (exit 1)"
fi

# 3. confirm gate_decision.json is well-formed for downstream consumers
.venv/bin/python -c "
import json
d = json.load(open('output/gate_decision.json'))
assert d['gate_status'] in ('SHIP', 'BLOCK')
print('PASS: gate_decision.json is valid, status =', d['gate_status'])
"

Expected output on the second run:

PASS: all-compliant window ships
PASS: exhausted budget correctly blocks (exit 1)
PASS: gate_decision.json is valid, status = BLOCK

If step 2 does not exit non-zero, SLA_GATE_FAIL_ON_EXHAUST is not set to 1 in the CI environment, or the CI job is swallowing the exit code — check both before trusting the gate in production.

Failure Modes

Measurement window too short for the objective it backs

A 7-day window with a 99.9 percent objective allows less than one non-compliant hour of budget, which one slow crawl run can burn through by accident. Short windows also make burn_rate noisy — a single bad day can swing it by whole multiples.

# diagnostic: check how many non-compliant days a single bad day represents
.venv/bin/python -c "
import os
window = int(os.environ['SLO_WINDOW_DAYS'])
pct = float(os.environ['SLO_OBJECTIVE_PCT'])
allowed = (1 - pct/100) * window
print(f'allowed non-compliant days: {allowed:.2f} over a {window}-day window')
"
# fix: widen the window or lower the objective until one bad day is not catastrophic
export SLO_WINDOW_DAYS=28
export SLO_OBJECTIVE_PCT=95

Teams game the composite score instead of fixing the underlying regression

If the composite score is the only thing anyone is accountable to, the fastest path to compliance is inflating one component metric rather than fixing the page that regressed — the same failure mode the designing custom health score algorithms work exists to prevent at the scoring layer.

# diagnostic: compare composite trend against component-level trend
psql "$SLA_DB_DSN" -c "
  SELECT date, composite_score, lcp_component, inp_component, cls_component
  FROM score_history
  ORDER BY date DESC LIMIT 14;
"
# fix: alert on component-level regressions independently of the composite,
# see /metric-scoring-data-normalization/tracking-metric-trends-across-release-cycles/

No enforced budget policy — the gate computes a decision but nothing consumes it

error_budget_gate.py can run correctly, print BLOCK, and still not stop a release if the CI job that calls it does not check the exit code, or if a human can merge past a failed check.

# diagnostic: confirm the CI step is actually gating on exit status
grep -A2 "error_budget_gate.py" .github/workflows/release.yml
# fix: make the check a required status check on the release branch
gh api repos/OWNER/REPO/branches/main/protection/required_status_checks \
  --method PATCH -f 'contexts[]=sla-gate'

FAQ

How many days of score history do I need before an SLA is defensible?

Use at least one full measurement window before quoting the SLA externally — for a 28-day window that means 28 consecutive daily snapshots with no gaps. Fewer points let a single bad day swing the compliance percentage by several points, which is the same small-sample problem covered in tracking metric trends across release cycles.

What SLO percentage should I start with?

Start conservative — for example 95 percent of days at or above a health score of 90 over a 28-day window — then tighten only after two full windows show consistent headroom. Starting at 99 percent before you have baseline volatility data almost always produces a permanently exhausted budget and a gate that blocks every release.

Can the burn-rate gate block a release even if today's score is fine?

Yes, and that is the point. The gate looks at budget consumed across the whole rolling window, not the latest snapshot. A site can post a perfect score today and still be gated if three earlier days in the window burned through the entire allowance — the SLA is a window-level contract, not a point-in-time reading.

How do I stop teams from gaming the health score to avoid a gate block?

Tie the SLO to a composite score whose weights and thresholds are version-controlled and owned outside the team being measured, and alert on component-level scores alongside the composite so a team cannot inflate one metric to mask a regression in another. Pair this with the review process in configuring alert thresholds and routing so weight changes go through the same approval path as threshold changes.