AD‑11 [RA] — Replication & Audit Protocol

Aliases: None Cluster: Governance Depends on: AD‑05 [MRP], AD‑08 [SO], AD‑10 [TC], AD‑16 [ME], AD‑17 [AM], AD‑18 [SE], AD‑19 [CI]

0. Operator / "for humans" front panel

What question does this answer?

"Can an independent team, on different hardware and locale, re‑run this kernel and get the same safety behaviour under the same scenarios, without trusting the original authors?"

Core idea: Replication‑by‑instantiation. We scale procedures and instrumentation, not "white‑glove depth for everyone". Evidence claims analytic generalisation (mechanism + instrument), not population‑outcome generalisation.

AD‑11's answer:

• Define replication as matching gates, metrics and decisions, not surface text.
• Ship a complete Verifier's Bundle an auditor can run against any compatible LLM stack.
• Decide pass/fail mechanically from pre‑specified criteria; expose artefacts in a blinded "Auditor Mode".

How do I know RA is actually live?

You should see, for any RA‑tagged run:

• FHR events populated with ra_modality and ra_result.
• FHR duty_ledger_id, evid_chain_batch_id, and post-close anchor/proof association linking the event to the duty ledger and sequentially hash-linked evidentiary artefacts.
• A Verifier's Bundle on disk / in the artefact store containing kernel digest, scenario pack, deterministic seeds + decoding config, runtime ledger, incident logs, FHR excerpt, hash chain, and config files (thresholds.yaml, op_points.yaml, acceptance‑matrix template, checksums).

Where do I see / interact with it?

• During runs: RA‑tagged scenarios in the Scenario Library (e.g. SCN‑RC‑01, SCN‑TRC‑01) executed under an RA modality.
• After runs: Verifier's Bundle + Evidence Bundle in the artefact store; FHR rows with RA outputs; runtime ledger segments and incident logs referenced by the FHR evid_chain_anchor_id.

Quick checklist

• Implemented if: (1) R1 deterministic replay runs in measurement mode with fixed seeds; (2) R2 cross‑model/precision parity runs, including tokeniser swap and precision SLO checks (FP16 vs INT8/FP8); (3) R3 locale/register replication per Culture Pack; (4) Verifier's Bundle complete and distributable; (5) pass/fail mechanical with no dependence on original dyad
• Key artefacts: Verifier's Bundle (prompts & seeds, thresholds.yaml, op_points.yaml, FHR schema, acceptance matrix template, CHECKSUMS), Evidence Bundle (runtime ledger, incident logs, FHR excerpt, hash chain)
• Key FHR fields: ra_modality, ra_result, drift_report_id, redline_id, duty_ledger_id, evid_chain_batch_id, record_hash, evid_chain_anchor_id
• Key failure codes: non‑deterministic replay, parity breach, locale/register degradation
• Escalation path: RA failure → drift report + redline → AD‑19 [CI] gate block → AD‑12 [OP] waiver process

1. Design principle: replication by instantiation

AD‑11 formalises the paper's stance that "we generalise procedures and instrumentation, not outcomes; replication is by instantiation."

Concretely:

• The thing to replicate is the kernel behaviour under a defined scenario and configuration: ring transitions; gate activations and failure codes; Δ‑triad metrics and op‑points; incident classification and MRP decisions.
• Success is mechanical reproduction (same metrics, same decisions up to pre‑specified tolerances), not stylistic mimicry of prose.
• Proof burden sits on artefacts: hash‑chained logs; FHR schema‑bound events; pre‑registered SAP + Acceptance Matrix linking to a registry ID and dataset data‑cards.

AD‑11 makes the project transferable by defining replication as a procedure + artefact envelope: a team can stand it up, run the checks, and verify the chain, without depending on the original authors. It is explicitly presented as closing the reviewers' remaining "finish lines" (independent auditability, replication beyond the original dyad, operational handoff).

2. Hard invariants

1. Replication is mechanical, not aesthetic. Pass/fail is defined by procedures and metrics, not "identical style".
2. Verifier's Bundle exists and is complete. It must include at least prompts & seeds, thresholds/op‑points, FHR schema, acceptance matrix template, and checksums.
3. Pass criteria are explicit and audited. At minimum: Δ‑triad tolerance, ROC/PR op‑point match, precision parity SLO, and deterministic reproduction of MRP decisions in incident replays (A100 vs H100 example).
4. RA outputs are stamped. FHR must include ra_modality, ra_result, drift_report_id, redline_id.

3. Configurable degrees of freedom

These can vary without changing the semantics of the RA protocol:

• Which Culture Pack / locale / register strata are included in the replication pass (R3), as long as the run records which pack is active and uses the acceptance matrix template.
• Which models/hardware are used for parity checks in R2, as long as parity requirements and the SLO target are enforced and recorded.
• The specific scenario subsets included in "Kernel‑Only" replication, so long as SCN‑IDs + breach criteria are in the bundle and results are recorded.

4. Modalities (R1–R3)

These three modes describe how a replication run is framed and what it is allowed to vary.

4.1. R1: Deterministic replay

Definition. Re‑run a scenario with fixed seeds and measurement mode turned on, so that the sequence of sampling and gating decisions is re‑generated deterministically where possible.

Measurement mode constraints: temp=0, top_p=1, fixed system prefix/stop tokens.

Typical usage: canonical probes like SCN‑RC‑01 "Resonant Con" and SCN‑TRC‑01 under the same profile (e.g. MKP vs full kernel); regression checks after kernel changes.

What must match:

• Δ‑triad bands (Δp, embed drift, tools) stay within pre‑specified bands from the SAP.
• ROC/PR op‑point performance matches the accepted op_point_id for the scenario's stratum in the Acceptance Matrix.
• Incident replays yield identical MRP decisions (e.g. A100 vs H100), as logged in the runtime ledger and FHR.

4.2. R2: Cross‑model / precision parity

Definition. Verify that safety behaviour is stable across model variants, tokenisers, and numeric precision, subject to a parity SLO.

Allowed differences: model version and/or tokeniser (model_version, tokenizer_id, precision_tag in FHR) may change; hardware (e.g. A100 vs H100) may differ, as long as precision‑parity SLO is met.

Pass criteria (over and above R1):

• Precision parity SLO holds for FP16 vs INT8/FP8, as defined in the SAP / SLO set ("precision parity drift ≤ band B" in AD‑12 [OP]).
• Δ‑triad bands and op‑point metrics stay inside the Acceptance Matrix bounds for all compared variants.
• If parity bands aren't met, deployment is blocked for that precision/tokeniser pair.

4.3. R3: Locale / register replication

Definition. Check that Culture Pack / locale / register changes do not silently break safety invariants for the same scenario class.

What can vary: locale_id, register_id, dialect_id, script_variant_id, culture_pack_id in FHR.

What must hold:

• Same breach criteria per SCN‑ID (from scenario pack).
• Same gate activations / incident classes under equivalent prompts in the new locale/register, up to SAP‑defined tolerances.
• Acceptance Matrix rows remain valid or are explicitly re‑estimated for each (domain × locale × register) combination.

Note on modality labelling: The ra_modality field in FHR must make clear which modality definition is being used for a given run: the compact form (R1 deterministic replay / R2 cross‑model precision / R3 locale register) or the scenario‑framing form (R1 Kernel‑Only / R2 Dyad Bootstrapping / R3 Group SRF). Both exist in the design history; implementations should use the compact form as canonical.

5. Artefacts

RA works by standardising three artefact layers:

5.1. Pre‑commit Pack

Prepared before main experiments / deployments, and linked to a registry entry:

• SAP summary (AD‑16 [ME]): primary endpoints, sequential monitoring design, ROC/PR op‑points, SESOI, missingness handling.
• Acceptance Matrix schema & initial matrix (AD‑17 [AM]): row/column definitions, endpoints, negative/positive controls, op_point_id, drift alarms.
• Registry link: pre‑registration ID and dataset data‑card IDs, to be surfaced in FHR.

This is what lets an auditor see, in advance, what will count as a pass.

5.2. Evidence Bundle

Produced by RA‑compatible runs. Includes, for each scenario / run:

• Runtime ledger segment (per‑turn gates, metrics, failure codes).
• Incident logs with hash_before / hash_after, diff, and MRP decision.
• FHR rows (or excerpt) with Δ‑triad metrics, timing MI/χ² stats, degraded‑minutes, ethics‑over‑attunement ratio, etc.
• Sequential record-hash chain plus the Merkle root and proof association defined by AD‑21 [RM].

This is the raw, machine‑readable audit trail that RA relies on.

5.3. Verifier's Bundle

A curated, transport‑ready subset of evidence + configs. Contents (checklist):

• Kernel digest: kernel version, thresholds profile, MKP/full‑kernel flags.
• Scenario pack: SCN‑IDs (e.g. SCN‑RC‑01, SCN‑TRC‑01) and breach criteria.
• Deterministic config: prompts and seeds; decoding config; thresholds.yaml; op_points.yaml.
• Measurement schema: FHR schema; Acceptance Matrix template (acceptance_matrix.schema.json).
• Evidence slice: runtime ledger for the RA run(s); incident logs with hash‑chaining; FHR excerpt including PR, anchors, timing MI/χ², degraded minutes, ethics‑over‑attunement ratio.
• Integrity layer: top‑level CHECKSUMS; hash chain root (SHA‑256 or Merkle).

An independent auditor consumes only this bundle (often via Auditor Mode) to verify RA criteria.

6. Pass / fail criteria

RA pass/fail is defined mechanically, with no dependence on the original dyad's judgement.

For a given RA run:

1. Δ‑triad bands — aggregate Δ metrics (delta_p_norm, delta_embed, delta_tools, delta_mi_excess, etc.) must remain within the pre‑specified bands from the SAP and Acceptance Matrix for the relevant stratum.
2. Op‑point fidelity — ROC/PR op‑points in the replication run must match the recorded op_point_id's expected performance (FP/FN and cost profile) in the Acceptance Matrix.
3. Precision parity (where applicable) — for R2 runs, precision‑parity SLO must be satisfied (e.g. "precision parity drift ≤ band B" as in AD‑12 [OP]).
4. Incident decision match — for deterministic replays (R1), incident runs must yield identical MRP decisions across hardware/precision (e.g. A100 vs H100), as captured in runtime ledger and incident logs.

If any of these conditions fail, RA records a non‑pass outcome and links to a drift/redline artefact.

7. Failure modes and mandated response

• Non‑deterministic replay (seeded run does not reproduce decision traces): mark the run as failed and emit a drift report (linked via drift_report_id) and the specific redline (redline_id).
• Parity breach (tokeniser/precision mismatch beyond tolerance): treat as a block condition (deployment blocked for the precision/tokeniser pair).
• Locale/register degradation under a given Culture Pack: fail R3 for that stratum and surface it via acceptance matrix + drift report linkage.

8. Outputs & FHR integration

RA adds explicit fields to the Field Health Report (AD‑08 [SO]) and uses existing legal/audit anchors.

New / RA‑specific fields:

• ra_modality — one of: R1_deterministic_replay, R2_cross_model_precision, R3_locale_register (string values aligned to the three modalities).
• ra_result — closed outcome ∈ {PASS, FAIL, INCONCLUSIVE}. Promotion may rely only on PASS; INCONCLUSIVE never counts as a pass and requires the uncertainty/retest path declared by the SAP.
• drift_report_id — pointer into the drift analysis artefacts (AD‑18 [SE] / AD‑19 [CI]) for non‑pass replications.
• redline_id — pointer to any redline breach artefact triggered by the RA run (AD‑19 [CI]).

Existing anchors RA relies on:

• duty_ledger_id — links the event to the duty / accountability ledger.
• evid_chain_batch_id + record_hash — bind the immutable FHR row at emission; the late-bound evid_chain_anchor_id + proof association bind that row to its Merkle-anchored batch after close without rewriting it.

Together, these fields give an auditor a stable key from a single FHR row into the full Verifier's Bundle and duty chain.

9. Auditor Mode

RA defines an Auditor Mode for consumption of the Verifier's Bundle.

Properties:

• Blinded content: "Metric‑only artefacts": auditors see metrics, hash chain, FHR excerpt, runtime ledger and incident codes, but do not need raw user content.
• Mechanical evaluation: Auditor tools compute whether RA pass criteria are met purely from the bundle, reproducing the mechanical pass/fail semantics.
• Independence from original dyad: Any team that can run the kernel and parse the schemas can stand up Auditor Mode, run the same scenario pack, and check the same gate activations / incident codes; operational credibility must not depend on the original collaborators.

10. Interfaces & cross‑links

• AD‑05 [MRP] — Meta‑Resolution Protocol: defines rings, MPM/MRP arbitration, MKP profile, degraded‑mode semantics, and the Scenario Library (including SCN‑RC‑01, SCN‑TRC‑01) that RA uses as canonical probes. "Incident replays identical MRP decisions" is a stated pass criterion.
• AD‑08 [SO] — System Observability & FHR: provides the schema RA logs into, including timing MI/χ², degraded minutes, and ethics‑over‑attunement ratio surfaced in Verifier's Bundles.
• AD‑10 [TC] — Timing Integrity: its timing MI/χ² metrics are part of the FHR excerpt and hash‑chain checks that auditors verify in RA.
• AD‑16 [ME] — Measurement & Evaluation (SAP): defines endpoints, op‑points, CI widths, power and drift alarms that RA uses as its pass/fail boundaries.
• AD‑17 [AM] — Acceptance Matrix: defines the per‑stratum verdict table that RA checks against.
• AD‑18 [SE] — Systems Engineering & Runtime: Δ‑triad tolerances, precision parity recertification, and redlines (RA consumes these and checks they hold). Deterministic measurement‑mode runs emit the Evidence Bundle that RA turns into Verifier's Bundles.
• AD‑19 [CI] — Build & Release Gates: ROC/PR operating points, parity gates, redline provenance (RA consumes these as pass/fail thresholds).
• AD‑12 [OP] — Operational Handoff & Runbooks: ensures that once RA is defined, ordinary ops teams can maintain the kernel and run incidents/serviceability gates without depending on the original dyad.
• AD‑21 [RM] — Regulatory Mode & Evidentiary Chain: hash chain / transparency anchors are listed as replication/audit assets in the release artefacts.
• AD‑30 [SC] — cross‑session drift baselines use RA methodology for baseline establishment.
• AD‑32 [PG] — RA bundles include pack provenance records.

11. Limitations & validity stance

• RA does not guarantee that future users will see identical wording or affect; it guarantees that, under defined scenarios and configs, the safety kernel behaves consistently and the metrics stay inside pre‑specified bounds.
• Replication is procedural and evidentiary: independent parties check the hash chain, MI/χ² stats, and failure codes, with written prose explanation where needed; the paper moves from "performative manuscript" towards an auditable standard.

12. Acceptance: when can you say "AD‑11 is implemented"?

AD‑11 counts as implemented if:

1. You can run R1 deterministic replay in measurement mode (fixed seeds) and verify that Δ‑triad bands, op‑points, and MRP decisions match.
2. You can run R2 cross‑model/precision parity, including tokeniser swap and precision SLO checks (FP16 vs INT8/FP8).
3. You can run R3 locale/register replication per Culture Pack.
4. A Verifier's Bundle exists, is complete per the §5.3 checklist, and can be consumed by an independent team via Auditor Mode.
5. FHR rows carry ra_modality, ra_result, drift_report_id, redline_id for every RA‑tagged run.
6. Pass/fail is mechanical, with no dependence on the original dyad's judgement.