# Concerted Engineering: operational metric dictionary

[claim:CE-METRIC-01] Model version: 0.3.1-draft.1 · Experimental definitions · Updated 2026-10-20

[claim:CE-METRIC-01] Definitions below are diagnostic measures, not universal performance targets. Register local parameters, unit of analysis, cohort, task/risk tier, horizon, data access, missingness and collection coverage before intervention. Keep denominators and severity rubrics fixed; report undefined ratios and incomplete observations explicitly. Comparison teams use the same ledger categories. Nested metrics must not be added twice.

[claim:CE-METRIC-01] The versioned model bundle in the OpenKedge repository is the canonical source. This appendix and the companion metric page are generated views of its metric records.

| Metric | Type / owner | Counter-metrics | Operational definition | Formula | Unit | Source and collection | Cadence | Anti-gaming / evidence floor |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| [claim:CE-METRIC-01] Contract authoring and maintenance hours | input / Engineering Leader | contract-drift, delivery-lead-time | Human effort to create, review, reconcile, amend, retire, and regenerate views of contracts and ADR links. | sum(person_minutes for category=contract)/60; report initial vs upkeep and record kind separately | person-hours per initiative/change tier and reporting period | Append-only session start/pause/resume/done timestamps linked to change, owner and tier; reconcile with editor/review history and report idle/untracked coverage. No content or keystroke capture. Manual corrections retain provenance. | Weekly; baseline and intervention | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Unlogged edits or moving contract work into coding categories. Include rejected attempts; report coverage. This is a subset of coordination effort and total labor, not an extra charge to add twice. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Contract cost share | input / Engineering Leader | escaped-defects, outcome-rate | Share of total human effort spent on contract work. | 100 × contract_hours / all_in_scope_human_hours | percent; undefined when denominator is zero | Same exclusive-category time ledger, including discovery, execution, review, operations, and maintenance. | Weekly by task/risk tier | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Omitting upkeep or comparison-team documentation work. Apply the same categories in both groups. A lower share alone is not success; inspect outcomes and absolute hours. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Total cost per verified outcome | output / Engineering Leader | outcome-rate, escaped-defects | All observed cost of work assigned to a cohort, including failed or abandoned attempts, divided by independently verified outcomes in that cohort. | (sum(person_hours × declared_loaded_hourly_rate) + model + compute + infrastructure + integration_cost) / verified_outcome_count | declared currency/outcome; report total costs when count=0 | Ledger, billing exports, CI usage, and independently approved outcome records. Allocate shared setup using a preregistered rule; disclose raw costs and allocation. | Monthly and at each outcome horizon | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Excluding failures, optimistic future amortization, or redefining an outcome. Keep the cohort and outcome rule fixed. Do not pool tiers with different horizons. Contract time is included once, within total labor. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Discovery latency | input / Product Manager | escaped-defects, contract-cost-share | Time from logged, evidence-backed customer problem to accepted initial Intent revision. | accepted_intent_at − problem_logged_at | calendar hours; report median, p90/p95 when supported and open ages | Product research/evidence log and immutable contract acceptance event; log entry when evidence is received, not at drafting completion. | Weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Delaying problem entry or excluding unresolved problems. Audit intake timestamps and show incomplete cases. Unlike full delivery lead time, this ends at intent agreement. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [E0] [claim:CE-METRIC-01] Realized outcome rate | output / Product Manager | verified-outcome-cost, escaped-defects | Share of due outcome commitments meeting their preregistered customer criterion at their contracted horizon; aliases include 90-Day Realized Value Rate and Outcome Realization. | 100 × commitments meeting criterion / all commitments whose horizon elapsed in the cohort | percent, with horizon, numerator and denominator | Outcome registry with frozen criteria, telemetry queries and independent review; missing due evidence counts as not demonstrated. Pending windows are reported separately. | At the registered horizon; monthly cohort review | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Choosing short horizons, moving due dates or dropping failures. Freeze scope/horizon; report withdrawals and missing evidence. 90 days applies to longer product bets, not every bug fix; compare only like horizons and risk tiers. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Intent-Ready-to-production lead time | output / Engineering Leader | escaped-defects, rework-tax | Elapsed time from the accepted Intent Ready transition to first actual production exposure linked to that initiative; admission alone does not end the clock. | p50 and p85(production_exposed_at - intent_ready_at), by risk/complexity cohort | calendar hours; full distributions and open ages | Versioned Intent Ready event, admission record and actual deployment/feature-flag exposure telemetry; retain original initiative ID across supersession. | Collect continuously; monthly output review with weekly input diagnostics | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Restarting the clock after rework or counting merge as production. Keep original and amendment timestamps. Report time to full rollout separately and include incomplete/abandoned work. Schema 0.2 changes the start from approval to Intent Ready and the endpoint from admission to actual production exposure; do not pool incompatible definition versions. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Review and verification latency | input / Engineering Leader | escaped-defects, mutation-score | Elapsed time from a candidate becoming review-ready under the registered rule to a final independent evidence/admission decision. | decision_at − review_ready_at; separately sum active_reviewer_minutes/60 | calendar hours and active reviewer person-hours | PR events, protected CI/evidence receipts and time ledger, joined by candidate ID; record retries and rejected candidates. | Weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Marking ready late, splitting PRs, or hiding failed checks. Fix readiness criteria and report queue/retry distributions. Queue elapsed time and human verification burden are different measures. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Decision latency | input / Engineering Leader | escaped-defects, contract-cost-share | Time from a complete trade-off request entering the decision queue to a recorded owner decision. | decision_recorded_at − request_logged_at | calendar hours, by risk and decision type | Timestamped decision-request log and accepted decision/waiver records. Preserve incomplete-request and completion events. | Weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Declaring requests incomplete to hide waiting or rushing decisions. Audit resets and subsequent reversal reasons. Compare Type 1 and Type 2 decisions separately; do not reward speed alone. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Organizational rework tax | input / Engineering Leader | escaped-defects, contract-cost-share | Share of human effort repeating or discarding prior work because recorded intent, dependencies, or decisions were misunderstood or changed. | 100 × classified_rework_hours / all_in_scope_human_hours | percent and person-hours | Exclusive-category time ledger with linked cause codes; weekly peer sample reviews classify necessary learning separately. | Weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Relabeling rework as discovery or blaming individuals. Publish cause rules and category coverage. Rework is not every refactor; maintenance with planned value is not automatically waste. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Coordination overhead | input / Engineering Leader | escaped-defects, contract-cost-share | Human effort spent on synchronization, contract upkeep, handoffs, approvals, and reconstructing context. | 100 × sum(exclusive coordination categories hours) / all_in_scope_human_hours | percent and person-hours | Consented calendar samples with linked-work tags and observed participation, reconciled with review/session traces. Count each participant once and distinguish active discussion from waiting; record corrections and sampling coverage. No individual ranking or sole reliance on retrospective self-report. | Weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Skipping necessary conversations or moving coordination to unlogged chat. Sample logs; pair with defects and developer feedback. Useful collaboration is still counted; the objective is net benefit, not zero coordination. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Coordination-to-output ratio | output / Engineering Leader | escaped-defects, contract-cost-share | Coordination effort required for one independently verified customer or maintenance outcome in a fixed cohort. | coordination_hours / verified_outcome_count | person-hours/outcome; undefined when count=0 | Coordination ledger joined to outcome registry; use the same cohort and horizon as total cost. | Monthly at the outcome horizon | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Splitting one outcome into many or omitting abandoned work. Freeze outcome definitions and preserve full cohort. This is a diagnostic decomposition of total cost, not a separate score to optimize. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Metric-induced engineering toil | guardrail / Verification Owner | escaped-defects, contract-cost-share | Effort collecting, correcting, and presenting measurement data that would not otherwise be needed to deliver or operate the service. | sum(person_minutes with measurement_only tag)/60; also 100 × hours / total_human_hours | person-hours and percent | Ledger and measurement pipeline issue log; distinguish automated collection setup from recurring corrections and reporting. | Weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Treating reporting as free or hiding manual corrections. Audit export jobs and recurring reports. A subset of total human effort; one primary work category plus tags avoids double-counting. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Priority stability | input / Engineering Leader | outcome-rate, decision-latency | Share of initiatives in a frozen weekly commitment list with no unplanned cancellation, goal change, or drop below their committed priority band. | 100 × unchanged_baseline_initiatives / all_baseline_initiatives | percent per week; report interruptions separately | Append-only weekly priority snapshot and change log with reason/owner; new urgent items are logged as interruptions, not silently substituted. | Weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Choosing a vacuous baseline or avoiding justified reprioritization. Record rationale, new urgent demand, and customer effects. Adaptation can be correct. This diagnoses churn rather than rewarding rigidity. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] ADR durability | output / Engineering Leader | decision-latency, escaped-defects | Elapsed accepted life of a decision before explicit supersession, with still-active decisions treated as right-censored observations. | superseded_at − accepted_at; active age = cutoff_at − accepted_at | calendar days; lifetimes, active ages and reasons | Versioned ADR log with acceptance/supersession events and reason codes; audit modified history. | Monthly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Keeping bad decisions alive or omitting supersessions. Include reversal quality and changed assumptions. Report all active/censored decisions; a mean of completed lifetimes alone is biased. Longer is not inherently better. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Stakeholder transition recovery | output / Engineering Leader | escaped-defects, contract-cost-share | Time from a declared ownership handover to the successor passing a predefined context-and-authority readiness review. | readiness_review_passed_at − handover_at | calendar days, with review checklist revision | Ownership registry, access grant log, handover checklist and independent reviewer event. | Per transition | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Easy checklists or delaying handover entry. Register the checklist and preserve failed attempts. Do not infer retention or delivery improvement from this measure alone. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [E0] [claim:CE-METRIC-01] Deep focus time ratio | input / Engineering Leader | escaped-defects, contract-cost-share | Share of observed working time in uninterrupted blocks of at least the registered duration; duration is declared before collection. | 100 × qualifying_block_minutes / observed_work_minutes | percent; disclose duration and observation coverage | Voluntary diary or consented calendar sampling; exclude leave and off-hours using the same rules for both groups. | Weekly aggregate | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Blocking calendars without focused work or avoiding needed collaboration. Pair with voluntary feedback; no individual ranking. The earlier >3-hour example is not a universal threshold; protect privacy and report aggregate coverage. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Voluntary talent retention | output / Engineering Leader | escaped-defects, contract-cost-share | Share of the starting eligible employee cohort that has not voluntarily left by the review date. | 100 × (starting_cohort − voluntary_leavers) / starting_cohort | percent; cohort count and observation duration | Authorized aggregate HR records with fixed eligibility and departure categories; no personal data in public reports. | Six and twelve months | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Reclassifying exits or changing the starting cohort. Disclose exclusions and organizational changes. Compensation, management changes and labor markets confound attribution. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Time to restore service | input / Engineering Leader | escaped-defects, contract-cost-share | Time from customer-impact incident start to restoration of the registered service objective; this is the defined MTTR variant here. | service_restored_at − customer_impact_started_at | calendar minutes by severity; distributions and unresolved ages | Incident timeline and SLO telemetry, reconciled by the incident owner after review. | Per incident; monthly summary | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Moving incident start or declaring restoration early. Preserve detection time and original timeline. Do not mix time to detect, repair, resolve, and restore under one MTTR label. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Escaped-defect severity | guardrail / Verification Owner | delivery-lead-time, mutation-score | Production defects attributed to the release cohort after admission. | count(defects) by preregistered severity; optionally count/100 deployments within the same tier | counts and defects/100 deployments; no undisclosed severity weights | Incident/defect tracker joined to deployment IDs; independent triage against a versioned severity rubric. | Per event; weekly and six-/twelve-month follow-up | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Downgrading severity, missing slow defects or changing attribution. Audit samples and publish exposure windows. Report severity strata and unresolved attribution instead of one favorable weighted average. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Rollback severity | guardrail / Verification Owner | escaped-defects, contract-cost-share | Rollbacks and failed recoveries attributed to a release cohort, classified by customer impact and recovery difficulty. | count(rollbacks) by severity; count/100 deployments within the same tier | counts and rollbacks/100 deployments, with impact duration | Deployment/rollback events joined to incident records and the registered impact rubric. | Per event; monthly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Avoiding a necessary rollback or relabeling it a hotfix. Include forward fixes and failed recovery attempts. A timely low-impact rollback can be a healthy control; interpret with customer impact. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Error-budget burn | guardrail / Verification Owner | escaped-defects, contract-cost-share | Observed bad-event fraction relative to the bad-event fraction allowed by the registered SLO. | (bad_events / eligible_events) / (1 − SLO_target) | dimensionless burn ratio per declared time window | Versioned SLO and telemetry query; retain traffic exclusions, window, and event counts. | Continuous alerts; weekly review | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Changing eligibility or SLO targets after seeing results. Version queries and thresholds before intervention. Undefined if eligible_events=0 or SLO_target=1. Report missing data; do not treat it as zero burn. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [E0] [claim:CE-METRIC-01] Two-week code survival | guardrail / Verification Owner | rework-tax, outcome-rate | Fraction of eligible added executable lines from a release still present after 14 days under a fixed diff/move-detection rule. | 100 × surviving_eligible_added_lines / eligible_added_lines | percent with diff algorithm and eligible-line count | Git history joined to deployed commits; exclude generated/vendor code under preregistered rules. | 14 days after deployment | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Preserving useless code or avoiding healthy deletions. Inspect deletion reasons and maintenance cost. Diagnostic only; deletion can be good and code movement changes the result. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Mutation score | guardrail / Verification Owner | escaped-defects, contract-hours | Fraction of non-equivalent, successfully executed mutants killed by a protected test suite. | 100 × killed_mutants / (killed_mutants + surviving_mutants) | percent plus counts of invalid, timed-out, excluded and equivalent mutants | Protected mutation runner with operator, harness and test-input versions; equivalence exclusions require owner review. | Per applicable candidate; weekly trend | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Choosing weak operators or excluding survivors. Freeze operator policy and audit exclusions. Context-specific coverage signal, not a universal quality score or proof of correctness. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Contract drift | guardrail / Verification Owner | escaped-defects, contract-cost-share | Required record links or derived views inconsistent with the exact governing revisions at inspection time. | 100 × stale_or_missing_required_links / inspected_required_links | percent plus unresolved drift count and age | Automated reference/digest validation and sampled human semantic review against a registered link inventory. | Per candidate and daily deployed-state review | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Reducing the inventory or checking only syntax. Freeze required link types and report semantic-review coverage. Digest equality detects version drift; it does not establish that a requirement is correct. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Policy exception volume | guardrail / Verification Owner | escaped-defects, review-latency | Requested, accepted, rejected, expired and unresolved waivers/emergency exceptions for the release cohort. | count(exception_records) by status and rule; accepted_count/100 admitted_deployments | counts, exceptions/100 deployments and unresolved age | Append-only waiver, incident and admission logs joined by exact artifact and environment. | Per admission; weekly | Report coverage, raw denominators and workload/tier mix; investigate unexplained changes. Using informal overrides or closing exceptions without checks. Reconcile privileged-access events against the exception log. Rising counts may reflect better reporting; investigate scope and severity, not count alone. E2 for a published local observed result; E3 requires a preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Translation share of lead time | input / Engineering Leader | delivery-lead-time, escaped-defects | Share of delivery lead time spent reconstructing or reconciling inconsistent/missing shared meaning across artifacts. Useful audience-specific rendering and analysis are not automatically waste. | 100 × union_duration(translation_intervals) / (production_exposed_at - intent_ready_at) | percent; undefined for nonpositive lead time | Workflow state transitions joined to the approved Intent and first admission; merge overlapping intervals before summing. Tag corrective reconciliation separately from purposeful projection/analysis work; retain both in total labor, and disclose coding disagreements. | Weekly | Report missing observations, denominators and the listed gaming vectors; never use for individual evaluation. Report state coverage, intent count/size and parallel-work attribution. Schema 0.2 aligns its window to Intent Ready through actual production exposure; do not pool earlier approval/admission cohorts. The numerator excludes useful projection work unless a registered review establishes corrective reconstruction; retain the definition version. E2 for published local observations; E3 for preregistered comparative claims with caveats. |
| [claim:CE-METRIC-01] Verification wait | input / Verification Owner | escaped-defects, mutation-score | Elapsed queue time from candidate-ready event to independent verification start, distinct from active check/review duration. | p50 and p85(verification_started_at - candidate_ready_at), by tier | hours | Candidate and protected verifier events; preserve rejected and incomplete candidates. | Weekly | Report missing observations, denominators and the listed gaming vectors; never use for individual evaluation. Right-censor unfinished waits; report sample size and clock rules. E2 for published local observations; E3 for preregistered comparative claims with caveats. |
| [claim:CE-METRIC-01] Developer experience pulse | guardrail / Engineering Leader | metric-toil, retention | Team aggregate responses on focus time, intent clarity, trust in checks, toil and intent to stay. | per-question response distribution; no individual composite ranking | ordinal distribution and response count | Voluntary five-question pulse; suppress small groups under the preregistered privacy rule. | Quarterly | Report missing observations, denominators and the listed gaming vectors; never use for individual evaluation. No individual evaluation; report response coverage and never infer causality from the pulse alone. E2 for published local observations; E3 for preregistered comparative claims with caveats. |
| [claim:CE-METRIC-01] Check flakiness | guardrail / Verification Owner | review-latency, escaped-defects | Inconsistent verdicts on identical immutable inputs under the same declared harness/environment. | inconsistent rerun verdict sets / all eligible rerun sets | fraction; undefined without eligible reruns | Protected runner repeated-run events joined to exact artifact/harness/environment; report sampling policy. | Per calibration/record; weekly aggregate review | Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking. Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone. E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Seeded-defect catch rate | guardrail / Verification Owner | verifier-false-positive, escaped-defects | Known realistic sandboxed defects rejected by a specific verifier configuration; report clean-control false positives separately. | caught_seed_ids / all_valid_seed_ids, by defect class and configuration | fraction and raw counts | Independently curated seed catalog, run receipts, catch/miss outcomes and representative-sampling limits. | Per calibration/record; weekly aggregate review | Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking. Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone. E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Verifier false-positive rate | guardrail / Verification Owner | seeded-catch-rate, review-latency | Known clean calibration controls rejected by a verifier configuration. | rejected_clean_controls / all_valid_clean_controls | fraction and raw counts | Independent clean-control catalog and protected calibration results. | Per calibration/record; weekly aggregate review | Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking. Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone. E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Draft edit survival | input / Software Engineer | contract-hours, contract-drift | Text similarity between an agent draft and accepted owner-edited record; a data-quality diagnostic, never acceptance evidence. | 1 - codepoint_levenshtein(draft, accepted) / max(codepoint_lengths); identical empty pair = 1 | fraction; register normalization and artifact versions | Local draft-quality command emits only digests, counts and edit distance; record drafting session time separately. | Per calibration/record; weekly aggregate review | Freeze versioned inclusion rules; report all eligible cases, missingness and source coverage; no individual ranking. Sample- and configuration-specific. No statistical independence, general correctness or causal benefit follows from this metric alone. E0 for synthetic exercises; E2 requires a recorded local baseline; E3 requires preregistered comparison and causal caveats. |
| [claim:CE-METRIC-01] Waiting share of delivery lead time | input / Engineering Leader | delivery-lead-time, escaped-defects | Share of lead time in any registered waiting state; translation share is a subset. | 100 × union_duration(waiting_intervals) / (production_exposed_at - intent_ready_at) | percent; undefined for nonpositive lead time | Workflow transitions joined to original Intent Ready and actual production exposure; merge overlapping intervals. | Collect per event/exercise; review weekly inputs and at registered output horizon | Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation. Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends. E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits. |
| [claim:CE-METRIC-01] Change failure rate | guardrail / Operations Owner | delivery-lead-time, review-latency | Admitted changes causing a registered service-impacting failure within the fixed follow-up window. | failed_admitted_change_ids / all_eligible_admitted_change_ids | fraction and raw counts | Deployment/incident links using preregistered failure/severity definitions; keep unresolved attribution explicit. | Collect per event/exercise; review weekly inputs and at registered output horizon | Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation. Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends. E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits. |
| [claim:CE-METRIC-01] Rollback frequency | guardrail / Operations Owner | rollback-severity, outcome-rate | Admitted deployments requiring rollback within the registered window, distinct from severity. | rolled_back_deployment_ids / eligible_deployment_ids | fraction and raw counts | Deployment controller recovery events and incident attribution; include aborted exposure under declared rules. | Collect per event/exercise; review weekly inputs and at registered output horizon | Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation. Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends. E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits. |
| [claim:CE-METRIC-01] Contract change frequency | input / Software Engineer | contract-hours, contract-drift | Material accepted revisions per linked initiative and observation window. | material_revision_count / eligible_initiative_count | revisions per initiative; undefined with no eligible initiatives | Immutable record supersession events; separate justified discovery revisions from reconciliation-only changes. | Collect per event/exercise; review weekly inputs and at registered output horizon | Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation. Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends. E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits. |
| [claim:CE-METRIC-01] Evidence invalidation frequency | input / Verification Owner | escaped-defects, review-latency | Previously accepted bundles invalidated by material input/policy/identity/environment changes. | invalidated_bundle_ids / eligible_accepted_bundle_ids | fraction and raw counts | Versioned invalidation events with trigger and downstream affected-artifact links. | Collect per event/exercise; review weekly inputs and at registered output horizon | Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation. Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends. E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits. |
| [claim:CE-METRIC-01] Maintainer comprehension | guardrail / Engineering Leader | contract-hours, rework-tax | Team-level ability to reconstruct rationale, allowed effects and failure response from a fixed maintenance task. | distribution of blinded rubric scores and completion time; register rubric before intervention | rubric distribution, task count and elapsed time | Consented team-level maintenance exercise with independent rubric scoring and comparable tasks. | Collect per event/exercise; review weekly inputs and at registered output horizon | Freeze definitions/denominators; report missingness, severity and all eligible cases; no individual evaluation. Context- and cohort-specific; distinguish description from causal claims and follow maintenance after special pilot support ends. E0 for definitions and synthetic examples; E2 needs recorded local baseline, E3 a preregistered comparison and explicit causal limits. |