10 min read

Adjusting Score Thresholds for E-commerce vs Blogs

Applying a uniform health score model to a mixed-architecture site creates two simultaneous failures: checkout and product pages generate constant noise from third-party payment scripts and cart hydration, while blog content silently regresses through missed LCP or CLS drift. The fix is per-section error budgets applied before any alert fires. This page is part of the Calibrating Error Thresholds for Different Site Sections workflow, which sits under the broader Metric Scoring & Data Normalization pipeline.

Environment Isolation and Dependency Declaration

Pin tool versions and export the required variables before running any segmentation logic. Using unpinned library versions causes statistics.variance behaviour differences between Python 3.9 and 3.11 that shift computed baselines by up to 8%.

export CRAWL_EXPORT_PATH="/var/data/crawl/latest.json"
export MONITORING_CONFIG_PATH="/etc/monitoring/thresholds.yaml"
export STAGING_BASE_URL="https://staging.example.com"
export MONITORING_WEBHOOK="https://monitoring.internal/api"

# Python 3.11.9, pyyaml 6.0.1, requests 2.31.0
python3 -m venv /opt/venv/threshold-tuning
source /opt/venv/threshold-tuning/bin/activate
pip install --quiet pyyaml==6.0.1 requests==2.31.0

Implementation

The classifier, baseline calculator, and config writer below form a single repeatable run. Execute them in order after each deployment or crawl export.

Step 1 — Classify URLs and compute per-section baselines 

#!/usr/bin/env python3
"""
threshold_segmenter.py
Classifies crawl records by template type, computes per-section
LCP/CLS/INP baselines, and writes a threshold-override YAML.

Usage:
    python3 threshold_segmenter.py \
        --input  /var/data/crawl/latest.json \
        --output /etc/monitoring/thresholds.yaml
"""

import re
import json
import statistics
import argparse
from collections import defaultdict
from pathlib import Path
import yaml  # pyyaml==6.0.1

# ------------------------------------------------------------------
# URL → section mapping
# Prefer CMS-driven template tags when available; fall back to regex.
# ------------------------------------------------------------------
SECTION_PATTERNS = {
    "ecommerce": re.compile(
        r"^/(product|products|cart|checkout|account|order|wishlist)/", re.I
    ),
    "blog": re.compile(
        r"^/(blog|articles|guides|news|insights)/", re.I
    ),
    "static": re.compile(
        r"^/(about|terms|privacy|contact|faq)(/|$)", re.I
    ),
}

def classify_url(url: str) -> str:
    for section, pattern in SECTION_PATTERNS.items():
        if pattern.match(url):
            return section
    return "unknown"


def load_crawl_export(path: str) -> list[dict]:
    with open(path, "r") as fh:
        data = json.load(fh)
    # Normalise: accept both top-level list and {"records": [...]}
    return data if isinstance(data, list) else data.get("records", [])


def compute_baselines(records: list[dict]) -> dict:
    """
    Returns per-section stats used to derive threshold tolerances.
    Requires ≥2 non-null samples per metric to compute variance;
    falls back to 0 for single-sample sections.
    """
    buckets: dict[str, list[dict]] = defaultdict(list)
    for rec in records:
        section = classify_url(rec.get("url", ""))
        buckets[section].append(rec)

    baselines = {}
    for section, recs in buckets.items():
        total = len(recs)
        errors = sum(1 for r in recs if r.get("status_code", 200) >= 400)

        def safe_values(metric: str) -> list[float]:
            return [
                r["metrics"][metric]
                for r in recs
                if r.get("metrics", {}).get(metric) is not None
            ]

        lcp_vals  = safe_values("lcp")
        cls_vals  = safe_values("cls")
        inp_vals  = safe_values("inp")

        baselines[section] = {
            "sample_size":    total,
            "error_rate":     round(errors / total, 4) if total else 0,
            "lcp_median_ms":  round(statistics.median(lcp_vals), 1) if lcp_vals else None,
            "lcp_variance":   round(statistics.variance(lcp_vals), 2) if len(lcp_vals) > 1 else 0,
            "cls_median":     round(statistics.median(cls_vals), 4) if cls_vals else None,
            "inp_median_ms":  round(statistics.median(inp_vals), 1) if inp_vals else None,
        }
    return baselines


def write_threshold_config(baselines: dict, output_path: str) -> None:
    """
    Writes the YAML threshold-override file consumed by the
    monitoring platform (see Step 2).  Section weights are
    hardcoded here and should be adjusted after 30-day calibration.
    """
    config = {"threshold_overrides": {}}

    SECTION_WEIGHTS = {
        "ecommerce": {
            "path_regex": r"^/(product|products|cart|checkout|account|order|wishlist)/.*",
            "lcp_critical_ms": 2500,
            "cls_critical": 0.10,
            "inp_critical_ms": 200,
            "baseline_deviation_tolerance": 0.04,
            "suppress": [
                {"type": "third_party_script_timeout", "condition": "duration < 2000ms"},
                {"type": "dynamic_cart_update",        "condition": "status == 200"},
            ],
        },
        "blog": {
            "path_regex": r"^/(blog|articles|guides|news|insights)/.*",
            "lcp_critical_ms": 4000,
            "cls_critical": 0.25,
            "inp_critical_ms": 500,
            "baseline_deviation_tolerance": 0.15,
            "suppress": [
                {"type": "render_blocking_css", "condition": "count <= 3"},
                {"type": "lazy_load_defer",     "condition": "impact == low"},
            ],
        },
        "static": {
            "path_regex": r"^/(about|terms|privacy|contact|faq)(/.*)?",
            "lcp_critical_ms": 5000,
            "cls_critical": 0.30,
            "inp_critical_ms": 800,
            "baseline_deviation_tolerance": 0.20,
            "suppress": [],
        },
    }

    for section, weights in SECTION_WEIGHTS.items():
        baseline = baselines.get(section, {})
        entry = dict(weights)
        entry["observed_lcp_median_ms"] = baseline.get("lcp_median_ms")
        entry["observed_error_rate"]    = baseline.get("error_rate")
        entry["sample_size"]            = baseline.get("sample_size", 0)
        config["threshold_overrides"][section] = entry

    Path(output_path).parent.mkdir(parents=True, exist_ok=True)
    with open(output_path, "w") as fh:
        yaml.dump(config, fh, default_flow_style=False, sort_keys=False)
    print(f"[OK] Threshold config written → {output_path}")


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--input",  required=True, help="Path to crawl export JSON")
    parser.add_argument("--output", required=True, help="Destination YAML config path")
    args = parser.parse_args()

    records   = load_crawl_export(args.input)
    baselines = compute_baselines(records)
    write_threshold_config(baselines, args.output)

The diagram below shows how sections map to severity tiers and where suppression rules intercept the alert path.

Per-section threshold routing Crawl records are classified by URL pattern into ecommerce, blog, or static buckets. Each bucket passes through its own severity weight matrix and suppression rule layer before alerts reach the monitoring platform. Crawl records URL classifier (regex patterns) E-commerce LCP ≤ 2500 ms CLS ≤ 0.10 Blog LCP ≤ 4000 ms CLS ≤ 0.25 Static LCP ≤ 5000 ms CLS ≤ 0.30 Alerting platform (suppression applied)

Step 2 — Apply the generated YAML to your monitoring platform

The file written by threshold_segmenter.py is consumed directly by most monitoring platforms that accept file-based config (Grafana, Prometheus Alertmanager, or a custom webhook router). For Alertmanager, mount it at /etc/alertmanager/threshold_overrides.yaml and reference it in the main alertmanager.yml:

# alertmanager.yml (excerpt) — reference the generated override file
route:
  receiver: "default"
  routes:
    - matchers:
        - section="ecommerce"
      receiver: "pagerduty-critical"
      continue: false
    - matchers:
        - section="blog"
      receiver: "slack-warning"
      continue: false

inhibit_rules:
  - source_matchers:   [ 'alertname="third_party_script_timeout"', 'section="ecommerce"' ]
    target_matchers:   [ 'section="ecommerce"' ]
    equal:             [ 'url' ]

For platforms that consume JSON, pipe the YAML output through yq -o=json /etc/monitoring/thresholds.yaml.

Verification and Smoke Test

Run this against staging before applying thresholds to production. Expected output for each assert_* function is shown in comments.

#!/usr/bin/env bash
set -euo pipefail

STAGING="${STAGING_BASE_URL:-https://staging.example.com}"
WEBHOOK="${MONITORING_WEBHOOK:-https://monitoring.internal/api}"

# 1. Inject a synthetic slow checkout (latency header)
assert_ecommerce_alert() {
    local status
    status=$(curl -s -o /dev/null -w "%{http_code}" \
        -H "X-Simulate-LCP: 3200" \
        "${STAGING}/checkout/step-1/")
    # Expect: monitoring platform fires a CRITICAL alert for /checkout/
    echo "Checkout probe HTTP status: ${status}"  # expected: 200
}

# 2. Inject a synthetic slow blog post — should NOT fire CRITICAL
assert_blog_tolerance() {
    local status
    status=$(curl -s -o /dev/null -w "%{http_code}" \
        -H "X-Simulate-LCP: 3800" \
        "${STAGING}/blog/example-post/")
    echo "Blog probe HTTP status: ${status}"  # expected: 200, no CRITICAL alert
}

# 3. Poll the webhook endpoint for alert severity breakdown
fetch_alert_summary() {
    curl -s -X POST "${WEBHOOK}/simulate/summary" \
        -H "Content-Type: application/json" \
        -d '{"window_seconds": 30}' \
    | jq -r '.alerts[] | "\(.section)\t\(.severity)\t\(.metric)"'
    # Expected lines:
    #   ecommerce   CRITICAL   lcp
    #   blog        WARNING    lcp   (or no alert if within tolerance)
}

assert_ecommerce_alert
assert_blog_tolerance
fetch_alert_summary

Pass criteria:

  • /checkout/ probe triggers a CRITICAL LCP alert within 30 seconds of injection.
  • /blog/ probe with LCP 3800 ms triggers at most a WARNING — never CRITICAL.
  • No 5xx status codes are suppressed for either section (hard floor).

Failure Modes

1. All blog URLs classified as unknown — no relaxed tolerance applied

Cause: regex pattern does not match the site's actual URL structure (e.g., locale prefix /en/blog/).

Diagnostic:

python3 -c "
import re, sys
pattern = re.compile(r'^/(blog|articles|guides|news|insights)/', re.I)
urls = ['/en/blog/post-123', '/blog/post-456', '/articles/guide-1']
for u in urls:
    print(u, '->', 'match' if pattern.match(u) else 'NO MATCH')
"

Fix: Prepend an optional locale segment to the pattern:

"blog": re.compile(
    r"^(/[a-z]{2}(-[a-z]{2})?)?(/(blog|articles|guides|news|insights))/", re.I
),

2. E-commerce suppression rules silencing legitimate 5xx errors

Cause: suppression conditions are too broad — the status == 200 guard was removed and duration < 2000ms is matching slow-but-failing requests.

Diagnostic: check alert suppression logs:

monitoring-cli alerts list \
    --section ecommerce \
    --suppressed true \
    --since "2h" \
| jq -r '.[] | select(.status_code >= 500) | "\(.url) \(.status_code) \(.suppressed_by)"'

Fix: add an explicit status_code < 400 guard to every suppression rule so HTTP error states are never silenced regardless of response time.

3. statistics.variance raises StatisticsError: variance requires at least two data points

Cause: a section has fewer than two LCP samples in the crawl export (common for staging environments or newly added URL templates).

Fix: guard with a length check before calling variance:

lcp_variance = statistics.variance(lcp_vals) if len(lcp_vals) > 1 else 0

This is already in the reference implementation above but can be dropped accidentally during copy-paste edits.

FAQ

Can I use the same LCP threshold for product pages and blog posts?

No. Product pages carry third-party payment scripts and hydration overhead that inflate LCP by 400–800 ms on median hardware. A single site-wide threshold either generates constant noise on PDPs or misses genuine regressions on content pages. Segment first, then apply separate budgets.

How do I prevent blog threshold relaxation from masking real outages?

Set a hard floor on suppression rules: never suppress 5xx status codes or WCAG contrast failures regardless of section. Relaxed tolerances should only affect metric variance (CLS, INP jitter), not error-state HTTP responses.

What `baseline_deviation_tolerance` value is right for a high-traffic checkout?

Start at 0.03–0.05 (3–5%). Checkout flows have narrow acceptable variance because even a 150 ms LCP shift correlates measurably with add-to-cart drop-off. Widen only after 30 days of clean baseline data.

How do threshold adjustments interact with device-type normalization?

Apply device-type normalization before section segmentation. Standardize raw metric values to a common baseline per device class first, then apply section-specific weights to the normalized scores. Mixing the order causes mobile variance to inflate desktop health scores for checkout paths.