Operational
metric definitions.

Model 0.3.1-draft.1 · bundle SHA-256 003b50c9d7f852a15f84d91f517430045203cff3402f790089b304f15e506947

[E0] Operational definitions and observation windows are design choices, not observed results. Read review and privacy rules.

This English reference is generated from the canonical OpenKedge model bundle. Register the definition version, local parameters, cohort, task/risk tier, observation window, severity rubric, missing-data rule and collection coverage before intervention. Report undefined ratios and incomplete observations explicitly. These measures are organizational diagnostics, not individual performance targets.

Download the metric appendix (Markdown) · Choose a change profile

Contract authoring and maintenance hours

CE-METRIC-01 · design principle

Human effort to create, review, reconcile, amend, retire, and regenerate views of contracts and ADR links.

Counter-metrics: Contract drift, Intent-Ready-to-production lead time

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Unlogged edits or moving contract work into coding categories. Include rejected attempts; report coverage.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: sum(person_minutes for category=contract)/60; report initial vs upkeep and record kind separately
Unit: person-hours per initiative/change tier and reporting period
Source and collection: Append-only session start/pause/resume/done timestamps linked to change, owner and tier; reconcile with editor/review history and report idle/untracked coverage. No content or keystroke capture. Manual corrections retain provenance.
Cadence: Weekly; baseline and intervention
Gaming vector: Unlogged edits or moving contract work into coding categories. Include rejected attempts; report coverage.
Interpretation limits: This is a subset of coordination effort and total labor, not an extra charge to add twice.

Total cost per verified outcome

CE-METRIC-01 · design principle

All observed cost of work assigned to a cohort, including failed or abandoned attempts, divided by independently verified outcomes in that cohort.

Counter-metrics: Realized outcome rate, Escaped-defect severity

Owner: Engineering Leader
Metric type: output
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Excluding failures, optimistic future amortization, or redefining an outcome. Keep the cohort and outcome rule fixed.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: (sum(person_hours × declared_loaded_hourly_rate) + model + compute + infrastructure + integration_cost) / verified_outcome_count
Unit: declared currency/outcome; report total costs when count=0
Source and collection: Ledger, billing exports, CI usage, and independently approved outcome records. Allocate shared setup using a preregistered rule; disclose raw costs and allocation.
Cadence: Monthly and at each outcome horizon
Gaming vector: Excluding failures, optimistic future amortization, or redefining an outcome. Keep the cohort and outcome rule fixed.
Interpretation limits: Do not pool tiers with different horizons. Contract time is included once, within total labor.

Discovery latency

CE-METRIC-01 · design principle

Time from logged, evidence-backed customer problem to accepted initial Intent revision.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Product Manager
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Delaying problem entry or excluding unresolved problems. Audit intake timestamps and show incomplete cases.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: accepted_intent_at − problem_logged_at
Unit: calendar hours; report median, p90/p95 when supported and open ages
Source and collection: Product research/evidence log and immutable contract acceptance event; log entry when evidence is received, not at drafting completion.
Cadence: Weekly
Gaming vector: Delaying problem entry or excluding unresolved problems. Audit intake timestamps and show incomplete cases.
Interpretation limits: Unlike full delivery lead time, this ends at intent agreement.

Realized outcome rate

CE-METRIC-01 · design principle

Share of due outcome commitments meeting their preregistered customer criterion at their contracted horizon; aliases include 90-Day Realized Value Rate and Outcome Realization.

Counter-metrics: Total cost per verified outcome, Escaped-defect severity

Owner: Product Manager
Metric type: output
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Choosing short horizons, moving due dates or dropping failures. Freeze scope/horizon; report withdrawals and missing evidence.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × commitments meeting criterion / all commitments whose horizon elapsed in the cohort
Unit: percent, with horizon, numerator and denominator
Source and collection: Outcome registry with frozen criteria, telemetry queries and independent review; missing due evidence counts as not demonstrated. Pending windows are reported separately.
Cadence: At the registered horizon; monthly cohort review
Gaming vector: Choosing short horizons, moving due dates or dropping failures. Freeze scope/horizon; report withdrawals and missing evidence.
Interpretation limits: 90 days applies to longer product bets, not every bug fix; compare only like horizons and risk tiers.

Intent-Ready-to-production lead time

CE-METRIC-01 · design principle

Elapsed time from the accepted Intent Ready transition to first actual production exposure linked to that initiative; admission alone does not end the clock.

Counter-metrics: Escaped-defect severity, Organizational rework tax

Owner: Engineering Leader
Metric type: output
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Restarting the clock after rework or counting merge as production. Keep original and amendment timestamps.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: p50 and p85(production_exposed_at - intent_ready_at), by risk/complexity cohort
Unit: calendar hours; full distributions and open ages
Source and collection: Versioned Intent Ready event, admission record and actual deployment/feature-flag exposure telemetry; retain original initiative ID across supersession.
Cadence: Collect continuously; monthly output review with weekly input diagnostics
Gaming vector: Restarting the clock after rework or counting merge as production. Keep original and amendment timestamps.
Interpretation limits: Report time to full rollout separately and include incomplete/abandoned work. Schema 0.2 changes the start from approval to Intent Ready and the endpoint from admission to actual production exposure; do not pool incompatible definition versions.

Review and verification latency

CE-METRIC-01 · design principle

Elapsed time from a candidate becoming review-ready under the registered rule to a final independent evidence/admission decision.

Counter-metrics: Escaped-defect severity, Mutation score

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Marking ready late, splitting PRs, or hiding failed checks. Fix readiness criteria and report queue/retry distributions.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: decision_at − review_ready_at; separately sum active_reviewer_minutes/60
Unit: calendar hours and active reviewer person-hours
Source and collection: PR events, protected CI/evidence receipts and time ledger, joined by candidate ID; record retries and rejected candidates.
Cadence: Weekly
Gaming vector: Marking ready late, splitting PRs, or hiding failed checks. Fix readiness criteria and report queue/retry distributions.
Interpretation limits: Queue elapsed time and human verification burden are different measures.

Decision latency

CE-METRIC-01 · design principle

Time from a complete trade-off request entering the decision queue to a recorded owner decision.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Declaring requests incomplete to hide waiting or rushing decisions. Audit resets and subsequent reversal reasons.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: decision_recorded_at − request_logged_at
Unit: calendar hours, by risk and decision type
Source and collection: Timestamped decision-request log and accepted decision/waiver records. Preserve incomplete-request and completion events.
Cadence: Weekly
Gaming vector: Declaring requests incomplete to hide waiting or rushing decisions. Audit resets and subsequent reversal reasons.
Interpretation limits: Compare Type 1 and Type 2 decisions separately; do not reward speed alone.

Organizational rework tax

CE-METRIC-01 · design principle

Share of human effort repeating or discarding prior work because recorded intent, dependencies, or decisions were misunderstood or changed.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Relabeling rework as discovery or blaming individuals. Publish cause rules and category coverage.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × classified_rework_hours / all_in_scope_human_hours
Unit: percent and person-hours
Source and collection: Exclusive-category time ledger with linked cause codes; weekly peer sample reviews classify necessary learning separately.
Cadence: Weekly
Gaming vector: Relabeling rework as discovery or blaming individuals. Publish cause rules and category coverage.
Interpretation limits: Rework is not every refactor; maintenance with planned value is not automatically waste.

Coordination overhead

CE-METRIC-01 · design principle

Human effort spent on synchronization, contract upkeep, handoffs, approvals, and reconstructing context.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Skipping necessary conversations or moving coordination to unlogged chat. Sample logs; pair with defects and developer feedback.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × sum(exclusive coordination categories hours) / all_in_scope_human_hours
Unit: percent and person-hours
Source and collection: Consented calendar samples with linked-work tags and observed participation, reconciled with review/session traces. Count each participant once and distinguish active discussion from waiting; record corrections and sampling coverage. No individual ranking or sole reliance on retrospective self-report.
Cadence: Weekly
Gaming vector: Skipping necessary conversations or moving coordination to unlogged chat. Sample logs; pair with defects and developer feedback.
Interpretation limits: Useful collaboration is still counted; the objective is net benefit, not zero coordination.

Coordination-to-output ratio

CE-METRIC-01 · design principle

Coordination effort required for one independently verified customer or maintenance outcome in a fixed cohort.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: output
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Splitting one outcome into many or omitting abandoned work. Freeze outcome definitions and preserve full cohort.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: coordination_hours / verified_outcome_count
Unit: person-hours/outcome; undefined when count=0
Source and collection: Coordination ledger joined to outcome registry; use the same cohort and horizon as total cost.
Cadence: Monthly at the outcome horizon
Gaming vector: Splitting one outcome into many or omitting abandoned work. Freeze outcome definitions and preserve full cohort.
Interpretation limits: This is a diagnostic decomposition of total cost, not a separate score to optimize.

Metric-induced engineering toil

CE-METRIC-01 · design principle

Effort collecting, correcting, and presenting measurement data that would not otherwise be needed to deliver or operate the service.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Treating reporting as free or hiding manual corrections. Audit export jobs and recurring reports.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: sum(person_minutes with measurement_only tag)/60; also 100 × hours / total_human_hours
Unit: person-hours and percent
Source and collection: Ledger and measurement pipeline issue log; distinguish automated collection setup from recurring corrections and reporting.
Cadence: Weekly
Gaming vector: Treating reporting as free or hiding manual corrections. Audit export jobs and recurring reports.
Interpretation limits: A subset of total human effort; one primary work category plus tags avoids double-counting.

Priority stability

CE-METRIC-01 · design principle

Share of initiatives in a frozen weekly commitment list with no unplanned cancellation, goal change, or drop below their committed priority band.

Counter-metrics: Realized outcome rate, Decision latency

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Choosing a vacuous baseline or avoiding justified reprioritization. Record rationale, new urgent demand, and customer effects.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × unchanged_baseline_initiatives / all_baseline_initiatives
Unit: percent per week; report interruptions separately
Source and collection: Append-only weekly priority snapshot and change log with reason/owner; new urgent items are logged as interruptions, not silently substituted.
Cadence: Weekly
Gaming vector: Choosing a vacuous baseline or avoiding justified reprioritization. Record rationale, new urgent demand, and customer effects.
Interpretation limits: Adaptation can be correct. This diagnoses churn rather than rewarding rigidity.

ADR durability

CE-METRIC-01 · design principle

Elapsed accepted life of a decision before explicit supersession, with still-active decisions treated as right-censored observations.

Counter-metrics: Decision latency, Escaped-defect severity

Owner: Engineering Leader
Metric type: output
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Keeping bad decisions alive or omitting supersessions. Include reversal quality and changed assumptions.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: superseded_at − accepted_at; active age = cutoff_at − accepted_at
Unit: calendar days; lifetimes, active ages and reasons
Source and collection: Versioned ADR log with acceptance/supersession events and reason codes; audit modified history.
Cadence: Monthly
Gaming vector: Keeping bad decisions alive or omitting supersessions. Include reversal quality and changed assumptions.
Interpretation limits: Report all active/censored decisions; a mean of completed lifetimes alone is biased. Longer is not inherently better.

Stakeholder transition recovery

CE-METRIC-01 · design principle

Time from a declared ownership handover to the successor passing a predefined context-and-authority readiness review.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: output
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Easy checklists or delaying handover entry. Register the checklist and preserve failed attempts.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: readiness_review_passed_at − handover_at
Unit: calendar days, with review checklist revision
Source and collection: Ownership registry, access grant log, handover checklist and independent reviewer event.
Cadence: Per transition
Gaming vector: Easy checklists or delaying handover entry. Register the checklist and preserve failed attempts.
Interpretation limits: Do not infer retention or delivery improvement from this measure alone.

Deep focus time ratio

CE-METRIC-01 · design principle

Share of observed working time in uninterrupted blocks of at least the registered duration; duration is declared before collection.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Blocking calendars without focused work or avoiding needed collaboration. Pair with voluntary feedback; no individual ranking.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × qualifying_block_minutes / observed_work_minutes
Unit: percent; disclose duration and observation coverage
Source and collection: Voluntary diary or consented calendar sampling; exclude leave and off-hours using the same rules for both groups.
Cadence: Weekly aggregate
Gaming vector: Blocking calendars without focused work or avoiding needed collaboration. Pair with voluntary feedback; no individual ranking.
Interpretation limits: The earlier >3-hour example is not a universal threshold; protect privacy and report aggregate coverage.

Voluntary talent retention

CE-METRIC-01 · design principle

Share of the starting eligible employee cohort that has not voluntarily left by the review date.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: output
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Reclassifying exits or changing the starting cohort. Disclose exclusions and organizational changes.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × (starting_cohort − voluntary_leavers) / starting_cohort
Unit: percent; cohort count and observation duration
Source and collection: Authorized aggregate HR records with fixed eligibility and departure categories; no personal data in public reports.
Cadence: Six and twelve months
Gaming vector: Reclassifying exits or changing the starting cohort. Disclose exclusions and organizational changes.
Interpretation limits: Compensation, management changes and labor markets confound attribution.

Time to restore service

CE-METRIC-01 · design principle

Time from customer-impact incident start to restoration of the registered service objective; this is the defined MTTR variant here.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Engineering Leader
Metric type: input
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Moving incident start or declaring restoration early. Preserve detection time and original timeline.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: service_restored_at − customer_impact_started_at
Unit: calendar minutes by severity; distributions and unresolved ages
Source and collection: Incident timeline and SLO telemetry, reconciled by the incident owner after review.
Cadence: Per incident; monthly summary
Gaming vector: Moving incident start or declaring restoration early. Preserve detection time and original timeline.
Interpretation limits: Do not mix time to detect, repair, resolve, and restore under one MTTR label.

Escaped-defect severity

CE-METRIC-01 · design principle

Production defects attributed to the release cohort after admission.

Counter-metrics: Intent-Ready-to-production lead time, Mutation score

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Downgrading severity, missing slow defects or changing attribution. Audit samples and publish exposure windows.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: count(defects) by preregistered severity; optionally count/100 deployments within the same tier
Unit: counts and defects/100 deployments; no undisclosed severity weights
Source and collection: Incident/defect tracker joined to deployment IDs; independent triage against a versioned severity rubric.
Cadence: Per event; weekly and six-/twelve-month follow-up
Gaming vector: Downgrading severity, missing slow defects or changing attribution. Audit samples and publish exposure windows.
Interpretation limits: Report severity strata and unresolved attribution instead of one favorable weighted average.

Rollback severity

CE-METRIC-01 · design principle

Rollbacks and failed recoveries attributed to a release cohort, classified by customer impact and recovery difficulty.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Avoiding a necessary rollback or relabeling it a hotfix. Include forward fixes and failed recovery attempts.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: count(rollbacks) by severity; count/100 deployments within the same tier
Unit: counts and rollbacks/100 deployments, with impact duration
Source and collection: Deployment/rollback events joined to incident records and the registered impact rubric.
Cadence: Per event; monthly
Gaming vector: Avoiding a necessary rollback or relabeling it a hotfix. Include forward fixes and failed recovery attempts.
Interpretation limits: A timely low-impact rollback can be a healthy control; interpret with customer impact.

Error-budget burn

CE-METRIC-01 · design principle

Observed bad-event fraction relative to the bad-event fraction allowed by the registered SLO.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Changing eligibility or SLO targets after seeing results. Version queries and thresholds before intervention.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: (bad_events / eligible_events) / (1 − SLO_target)
Unit: dimensionless burn ratio per declared time window
Source and collection: Versioned SLO and telemetry query; retain traffic exclusions, window, and event counts.
Cadence: Continuous alerts; weekly review
Gaming vector: Changing eligibility or SLO targets after seeing results. Version queries and thresholds before intervention.
Interpretation limits: Undefined if eligible_events=0 or SLO_target=1. Report missing data; do not treat it as zero burn.

Two-week code survival

CE-METRIC-01 · design principle

Fraction of eligible added executable lines from a release still present after 14 days under a fixed diff/move-detection rule.

Counter-metrics: Organizational rework tax, Realized outcome rate

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Preserving useless code or avoiding healthy deletions. Inspect deletion reasons and maintenance cost.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × surviving_eligible_added_lines / eligible_added_lines
Unit: percent with diff algorithm and eligible-line count
Source and collection: Git history joined to deployed commits; exclude generated/vendor code under preregistered rules.
Cadence: 14 days after deployment
Gaming vector: Preserving useless code or avoiding healthy deletions. Inspect deletion reasons and maintenance cost.
Interpretation limits: Diagnostic only; deletion can be good and code movement changes the result.

Mutation score

CE-METRIC-01 · design principle

Fraction of non-equivalent, successfully executed mutants killed by a protected test suite.

Counter-metrics: Escaped-defect severity, Contract authoring and maintenance hours

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Choosing weak operators or excluding survivors. Freeze operator policy and audit exclusions.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × killed_mutants / (killed_mutants + surviving_mutants)
Unit: percent plus counts of invalid, timed-out, excluded and equivalent mutants
Source and collection: Protected mutation runner with operator, harness and test-input versions; equivalence exclusions require owner review.
Cadence: Per applicable candidate; weekly trend
Gaming vector: Choosing weak operators or excluding survivors. Freeze operator policy and audit exclusions.
Interpretation limits: Context-specific coverage signal, not a universal quality score or proof of correctness.

Contract drift

CE-METRIC-01 · design principle

Required record links or derived views inconsistent with the exact governing revisions at inspection time.

Counter-metrics: Escaped-defect severity, Contract cost share

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Reducing the inventory or checking only syntax. Freeze required link types and report semantic-review coverage.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: 100 × stale_or_missing_required_links / inspected_required_links
Unit: percent plus unresolved drift count and age
Source and collection: Automated reference/digest validation and sampled human semantic review against a registered link inventory.
Cadence: Per candidate and daily deployed-state review
Gaming vector: Reducing the inventory or checking only syntax. Freeze required link types and report semantic-review coverage.
Interpretation limits: Digest equality detects version drift; it does not establish that a requirement is correct.

Policy exception volume

CE-METRIC-01 · design principle

Requested, accepted, rejected, expired and unresolved waivers/emergency exceptions for the release cohort.

Counter-metrics: Escaped-defect severity, Review and verification latency

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Using informal overrides or closing exceptions without checks. Reconcile privileged-access events against the exception log.
Publication evidence floor: E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats.
Formula: count(exception_records) by status and rule; accepted_count/100 admitted_deployments
Unit: counts, exceptions/100 deployments and unresolved age
Source and collection: Append-only waiver, incident and admission logs joined by exact artifact and environment.
Cadence: Per admission; weekly
Gaming vector: Using informal overrides or closing exceptions without checks. Reconcile privileged-access events against the exception log.
Interpretation limits: Rising counts may reflect better reporting; investigate scope and severity, not count alone.

Verification wait

CE-METRIC-01 · design principle

Elapsed queue time from candidate-ready event to independent verification start, distinct from active check/review duration.

Counter-metrics: Escaped-defect severity, Mutation score

Owner: Verification Owner
Metric type: input
Anti-gaming rule: Report missing observations, denominators and the listed gaming vectors; never use for individual evaluation.
Publication evidence floor: E2 for published local observations; E3 for preregistered comparative claims with caveats.
Formula: p50 and p85(verification_started_at - candidate_ready_at), by tier
Unit: hours
Source and collection: Candidate and protected verifier events; preserve rejected and incomplete candidates.
Cadence: Weekly
Gaming vector: Delaying ready events or omitting rejected candidates.
Interpretation limits: Right-censor unfinished waits; report sample size and clock rules.

Developer experience pulse

CE-METRIC-01 · design principle

Team aggregate responses on focus time, intent clarity, trust in checks, toil and intent to stay.

Counter-metrics: Metric-induced engineering toil, Voluntary talent retention

Owner: Engineering Leader
Metric type: guardrail
Anti-gaming rule: Report missing observations, denominators and the listed gaming vectors; never use for individual evaluation.
Publication evidence floor: E2 for published local observations; E3 for preregistered comparative claims with caveats.
Formula: per-question response distribution; no individual composite ranking
Unit: ordinal distribution and response count
Source and collection: Voluntary five-question pulse; suppress small groups under the preregistered privacy rule.
Cadence: Quarterly
Gaming vector: Coerced responses, nonresponse bias and identifying small groups.
Interpretation limits: No individual evaluation; report response coverage and never infer causality from the pulse alone.

Check flakiness

CE-METRIC-01 · design principle

Inconsistent verdicts on identical immutable inputs under the same declared harness/environment.

Counter-metrics: Review and verification latency, Escaped-defect severity

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking.
Publication evidence floor: E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats.
Formula: inconsistent rerun verdict sets / all eligible rerun sets
Unit: fraction; undefined without eligible reruns
Source and collection: Protected runner repeated-run events joined to exact artifact/harness/environment; report sampling policy.
Cadence: Per calibration/record; weekly aggregate review
Gaming vector: Selecting easy seeds/controls or drafts; omitting failed attempts or unknown results.
Interpretation limits: Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone.

Seeded-defect catch rate

CE-METRIC-01 · design principle

Known realistic sandboxed defects rejected by a specific verifier configuration; report clean-control false positives separately.

Counter-metrics: Verifier false-positive rate, Escaped-defect severity

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking.
Publication evidence floor: E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats.
Formula: caught_seed_ids / all_valid_seed_ids, by defect class and configuration
Unit: fraction and raw counts
Source and collection: Independently curated seed catalog, run receipts, catch/miss outcomes and representative-sampling limits.
Cadence: Per calibration/record; weekly aggregate review
Gaming vector: Selecting easy seeds/controls or drafts; omitting failed attempts or unknown results.
Interpretation limits: Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone.

Verifier false-positive rate

CE-METRIC-01 · design principle

Known clean calibration controls rejected by a verifier configuration.

Counter-metrics: Seeded-defect catch rate, Review and verification latency

Owner: Verification Owner
Metric type: guardrail
Anti-gaming rule: Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking.
Publication evidence floor: E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats.
Formula: rejected_clean_controls / all_valid_clean_controls
Unit: fraction and raw counts
Source and collection: Independent clean-control catalog and protected calibration results.
Cadence: Per calibration/record; weekly aggregate review
Gaming vector: Selecting easy seeds/controls or drafts; omitting failed attempts or unknown results.
Interpretation limits: Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone.

Draft edit survival

CE-METRIC-01 · design principle

Text similarity between an agent draft and accepted owner-edited record; a data-quality diagnostic, never acceptance evidence.

Counter-metrics: Contract authoring and maintenance hours, Contract drift

Owner: Software Engineer
Metric type: input
Anti-gaming rule: Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking.
Publication evidence floor: E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats.
Formula: 1 - codepoint_levenshtein(draft, accepted) / max(codepoint_lengths); identical empty pair = 1
Unit: fraction; register normalization and artifact versions
Source and collection: Local draft-quality command emits only digests, counts and edit distance; record drafting session time separately.
Cadence: Per calibration/record; weekly aggregate review
Gaming vector: Selecting easy seeds/controls or drafts; omitting failed attempts or unknown results.
Interpretation limits: Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone.

Change failure rate

CE-METRIC-01 · design principle

Admitted changes causing a registered service-impacting failure within the fixed follow-up window.

Counter-metrics: Intent-Ready-to-production lead time, Review and verification latency

Owner: Operations Owner
Metric type: guardrail
Anti-gaming rule: Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation.
Publication evidence floor: E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits.
Formula: failed_admitted_change_ids / all_eligible_admitted_change_ids
Unit: fraction and raw counts
Source and collection: Deployment/incident links using preregistered failure/severity definitions; keep unresolved attribution explicit.
Cadence: Collect per event/exercise; review weekly inputs and at registered output horizon
Gaming vector: Changing cohort/window or omitting incomplete, adverse or unlinked observations.
Interpretation limits: Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends.

Rollback frequency

CE-METRIC-01 · design principle

Admitted deployments requiring rollback within the registered window, distinct from severity.

Counter-metrics: Rollback severity, Realized outcome rate

Owner: Operations Owner
Metric type: guardrail
Anti-gaming rule: Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation.
Publication evidence floor: E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits.
Formula: rolled_back_deployment_ids / eligible_deployment_ids
Unit: fraction and raw counts
Source and collection: Deployment controller recovery events and incident attribution; include aborted exposure under declared rules.
Cadence: Collect per event/exercise; review weekly inputs and at registered output horizon
Gaming vector: Changing cohort/window or omitting incomplete, adverse or unlinked observations.
Interpretation limits: Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends.

Contract change frequency

CE-METRIC-01 · design principle

Material accepted revisions per linked initiative and observation window.

Counter-metrics: Contract authoring and maintenance hours, Contract drift

Owner: Software Engineer
Metric type: input
Anti-gaming rule: Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation.
Publication evidence floor: E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits.
Formula: material_revision_count / eligible_initiative_count
Unit: revisions per initiative; undefined with no eligible initiatives
Source and collection: Immutable record supersession events; separate justified discovery revisions from reconciliation-only changes.
Cadence: Collect per event/exercise; review weekly inputs and at registered output horizon
Gaming vector: Changing cohort/window or omitting incomplete, adverse or unlinked observations.
Interpretation limits: Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends.

Evidence invalidation frequency

CE-METRIC-01 · design principle

Previously accepted bundles invalidated by material input/policy/identity/environment changes.

Counter-metrics: Escaped-defect severity, Review and verification latency

Owner: Verification Owner
Metric type: input
Anti-gaming rule: Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation.
Publication evidence floor: E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits.
Formula: invalidated_bundle_ids / eligible_accepted_bundle_ids
Unit: fraction and raw counts
Source and collection: Versioned invalidation events with trigger and downstream affected-artifact links.
Cadence: Collect per event/exercise; review weekly inputs and at registered output horizon
Gaming vector: Changing cohort/window or omitting incomplete, adverse or unlinked observations.
Interpretation limits: Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends.

Maintainer comprehension

CE-METRIC-01 · design principle

Team-level ability to reconstruct rationale, allowed effects and failure response from a fixed maintenance task.

Counter-metrics: Contract authoring and maintenance hours, Organizational rework tax

Owner: Engineering Leader
Metric type: guardrail
Anti-gaming rule: Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation.
Publication evidence floor: E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits.
Formula: distribution of blinded rubric scores and completion time; register rubric before intervention
Unit: rubric distribution, task count and elapsed time
Source and collection: Consented team-level maintenance exercise with independent rubric scoring and comparable tasks.
Cadence: Collect per event/exercise; review weekly inputs and at registered output horizon
Gaming vector: Changing cohort/window or omitting incomplete, adverse or unlinked observations.
Interpretation limits: Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends.