Design evidence
Architecture, threat model, ownership, policy, data flow, retention, authority, and declared limits.
CONTROL · EVIDENCE · AUTHORITY · LIMIT
Connect every control claim to the evidence that can support it.
A policy declaration, local fixture, production log, signed attestation, browser replay, and accountable owner are different evidence. Keep them separate, name what is missing, and route each unresolved conclusion to the correct authority.
Quick answer
What does the Control-to-Evidence Assurance Matrix do?
It maps each of thirteen source-bound defensive control families to separate design, implementation, test, and operational evidence; the relevant maturity dimensions; existing Machine Tradecraft guides, reports, labs, actions, fixtures, and Casebook cases; exact external-verification authorities; limitations; and review cadence. Evidence states remain separate and are never converted into a score, pass/fail result, compliance finding, certification, or production authorization.
- Controls
- 13 families derived from the governed defense-operations report.
- Maturity dimensions
- 9 independent dimensions; no aggregate maturity score.
- System profiles
- 7 allowlisted architectures from document ingestion through software and AI supply chains.
Evidence architecture
Do not treat a document, test, log, or signature as interchangeable proof.
The matrix separates four evidence classes and then adds two decision boundaries: what the local site can demonstrate and what still requires an exact external authority.
Implementation evidence
Versioned source, configuration, isolation, schema, validation, allowlists, and enforcement paths.
Test evidence
Governed fixtures, negative cases, deterministic outputs, browser or parser replay, and fail-closed behavior.
Operational evidence
Minimum run records, versions, hashes, decisions, exceptions, monitoring, ownership, review, and recovery.
Local evidence
Bounded Machine Tradecraft fixtures and allowlisted engines under the current release—never production assurance.
External verification
Exact production browser, parser, model, trust service, registry, DNS authority, owner, or incident authority.
Assurance boundary: the word “assurance” names the evidence-planning discipline. This workflow does not issue an assurance opinion, certification, compliance result, or independently verified efficacy claim.
Source-bound control families
Thirteen controls, each with its own evidence burden.
The labels and source implementations trace to the governed defense-operations report. The evidence requirements, routes, limitations, and review cadence are implementation-owned derivatives stored separately from that exact source body.
Input Acquisition
Quarantine raw byte streams upon ingestion and rely on deep file-signature analysis rather than a spoofable extension or declared media type.
- Evidence plane
- source bytes
- Remediation target
- Bypasses that rely on superficial file-type validation.
- Maturity dimensions
- Inventory and ownership, Input controls, Monitoring and evidence
Review evidence requirements
Design evidence
- Data-flow diagram showing the first trust boundary and raw-source owner
- Retention and privacy policy for original bytes
- Declared size, type, and source limits
Implementation evidence
- Quarantine configuration or source boundary
- Pre-transformation SHA-256 and byte-count implementation
- No-execution and no-public-extraction controls
Test evidence
- Valid, malformed, type-mismatched, oversized, and duplicate-source fixtures
- Proof that hashing occurs before normalization or sanitization
- Fail-closed acquisition tests
Operational evidence
- Minimum run records containing source hash, bytes, declared/detected type, owner, and acquisition outcome
- Exception and quarantine records
- Evidence-retention review
Known limitation: Local fixtures can demonstrate preservation and rejection behavior, but they cannot establish a production ingestion path, retention duty, or organizational ownership.
Type Validation
Inventory container objects without triggering active execution, and fail closed when declared type, magic bytes, syntax, structure, or parser boundaries disagree.
- Evidence plane
- container structure
- Remediation target
- Execution and parser-confusion vulnerabilities in processing libraries.
- Maturity dimensions
- Input controls, Parser and representation testing
Review evidence requirements
Design evidence
- Allowlisted formats and exact parser versions
- Media-type, encoding, archive, recursion, and expansion limits
- Ambiguity and unsupported-format policy
Implementation evidence
- Strict type and syntax checks
- Local/central record, marker, offset, and boundary agreement
- Parser isolation and resource limits
Test evidence
- Malformed container, duplicate entry, overlap, encryption, path traversal, and extension/type mismatch fixtures
- Cross-parser differential tests where required
- Fail-closed error tests
Operational evidence
- Detected type and parser version in each run record
- Rejected ambiguity and exception telemetry
- Parser update and vulnerability-review records
Known limitation: One local parser cannot establish conformance or agreement with every production renderer, archive library, mail client, or document application.
Canonicalization
Apply context-specific Unicode, serialization, and representation rules while preserving the original and refusing destructive transformations where they would alter meaning.
- Evidence plane
- unicode text
- Remediation target
- Homographs, bidirectional manipulation, parser drift, and inconsistent comparison boundaries.
- Maturity dimensions
- Input controls, Parser and representation testing, AI context construction
Review evidence requirements
Design evidence
- Field-by-field normalization and identifier policy
- Original-value preservation rule
- Explicit prohibited contexts such as passwords or source-code rewriting
Implementation evidence
- Versioned normalization, confusable, bidi, and canonical serialization implementation
- Separate raw and transformed values
- Context-specific exceptions
Test evidence
- NFC/NFD, compatibility-sensitive, mixed-script, joiner, bidi, grapheme, and multilingual fixtures
- False-positive and destructive-transformation tests
- Idempotency and concatenation-boundary tests where applicable
Operational evidence
- Unicode data/version identity
- Transformation history and before/after hashes
- Exception and policy-override records
Known limitation: The local Unicode maps are educational subsets; exact Unicode, locale, font, identifier, protocol, and production tokenizer behavior require version-pinned verification.
Representation Comparison
Process the same source through separately named structural, rendered, semantic, metadata, extracted, token/model, and behavioral paths, then preserve agreement and conflict without flattening them.
- Evidence plane
- transformation survival
- Remediation target
- Hidden text, CSS and accessibility divergence, OCR differences, parser confusion, and model-view mismatch.
- Maturity dimensions
- Parser and representation testing, Monitoring and evidence
Review evidence requirements
Design evidence
- Declared evidence planes and comparison method
- Rules for Same, Different, Conditional, Not comparable, and Not evaluated
- Escalation path for unresolved differences
Implementation evidence
- Independent representation derivation
- Method and representation SHA-256 identities
- No silent flattening or automatic trust selection
Test evidence
- Known-difference fixtures spanning DOM/render/accessibility, MIME alternatives, metadata/pixels, and tokenizer views
- Transformation-survival tests
- Named browser or parser replay
Operational evidence
- Comparison records and unresolved conflicts
- Named external-verification owner
- Trend and exception review without an aggregate anomaly score
Known limitation: Local comparisons establish only the declared methods and fixtures; they do not establish production equivalence or universal visual, semantic, or model behavior.
Provenance
Bind each representation and model-bound span to a source, transformation, actor or tool claim, and exact artifact identity while separating structural claims from trust verification.
- Evidence plane
- metadata provenance
- Remediation target
- Context confusion, false attribution, indirect prompt injection, and untraceable transformations.
- Maturity dimensions
- Provenance and supply chain, AI context construction, Monitoring and evidence
Review evidence requirements
Design evidence
- Provenance data model and trust assumptions
- Source and transformation labelling rules
- Privacy and metadata minimization policy
Implementation evidence
- Hashes, manifest/claim parsing, ingredient/material relationships, and transformation records
- Source labels in model context
- Explicit structural-versus-cryptographic status
Test evidence
- Missing, malformed, stripped, conflicting, and negative provenance fixtures
- Digest mismatch and lineage tests
- Trust-verifier negative cases
Operational evidence
- Artifact and transformation chain records
- Trust-service and revocation evidence where applicable
- Provenance gaps and privacy exceptions
Known limitation: Provenance can describe origin, integrity, attribution, and transformation claims; it does not by itself prove semantic truth, safety, completeness, or benign intent.
Content Sanitization
Create separate inert derivatives that remove or neutralize active content, unsafe relationships, unsupported controls, and unnecessary metadata while preserving the original evidence.
- Evidence plane
- decoded extracted
- Remediation target
- Macros, scripts, active relationships, unsafe rendering, and machine-readable payloads reaching privileged contexts.
- Maturity dimensions
- Input controls, Parser and representation testing, AI context construction
Review evidence requirements
Design evidence
- Sanitization policy by format and field
- Preservation, derivative, and rollback rules
- Declared transformations and data-loss warnings
Implementation evidence
- Inert decoding and allowlisted extraction
- Removal or neutralization of scripts, macros, external links, active objects, and unsafe markup
- Derivative hashes and transformation log
Test evidence
- Active-content, external-relationship, hidden-content, metadata, and malformed-fixture tests
- Before/after representation comparison
- Accessibility and internationalization regression tests
Operational evidence
- Original and derivative identities
- Sanitizer/parser versions
- Transformation exceptions and reprocessing records
Known limitation: Sanitization can remove useful, accessible, evidentiary, or provenance data and cannot establish the semantic safety of retained content.
Model-Input Construction
Construct model input from typed, provenance-labelled fields rather than unbounded string concatenation, keeping policy, user data, retrieved evidence, metadata, and tool results distinct.
- Evidence plane
- token model
- Remediation target
- Delimiter confusion, role-marker spoofing, context flattening, and instruction/data ambiguity.
- Maturity dimensions
- AI context construction, Retrieval and agent isolation
Review evidence requirements
Design evidence
- Context schema and authority hierarchy
- Source-labelling and truncation policy
- Special-token, chat-template, and encoding contract
Implementation evidence
- Typed serialization, random or structurally enforced boundaries, provenance labels, and token-aware limits
- Exact model/tokenizer/template version capture
- No raw active objects in privileged context
Test evidence
- Delimiter, role-marker, Unicode, truncation, retrieval, and tool-result fixtures
- Exact production tokenizer/template replay
- Prompt-injection and output-boundary tests
Operational evidence
- Assembled context identity and source map
- Model/tokenizer/template version
- Truncation, exception, and blocked-action records
Known limitation: Structured context reduces ambiguity but does not create a mathematically perfect instruction/data boundary inside a language model.
Retrieval Controls
Enforce inherited access control, source identity, chunk hashing, freshness, ranking review, and provenance labels before retrieved evidence enters model context.
- Evidence plane
- behavior capability
- Remediation target
- Retrieval poisoning, unauthorized data access, source impersonation, and stale or split-view content.
- Maturity dimensions
- Retrieval and agent isolation, AI context construction, Monitoring and evidence
Review evidence requirements
Design evidence
- Retrieval data flow, ACL inheritance, source ownership, and freshness policy
- Index and chunk identity model
- Poisoning and rollback threat model
Implementation evidence
- Access-filtered retrieval, chunk/source hashes, ranking provenance, and source labels
- Index rebuild and invalidation controls
- No tool authority inherited from retrieved text
Test evidence
- Poisoned, stale, unauthorized, ranking-manipulated, and source-conflict fixtures
- ACL and user-context tests
- Index rebuild/replay tests
Operational evidence
- Query, retrieved source, chunk, rank, user authority, and index version records
- Poisoning alerts and quarantine records
- Owner-approved content restoration
Known limitation: Local retrieval simulations cannot establish production ACL inheritance, embedding behavior, ranking exposure, or live content freshness.
Capability Isolation
Separate model reasoning from tool execution through narrow allowlists, scoped credentials, read-only defaults, policy mediation, and no self-modifying tool definitions.
- Evidence plane
- behavior capability
- Remediation target
- Unauthorized tool use, privilege escalation, agent confusion, and excessive agency.
- Maturity dimensions
- Retrieval and agent isolation, Governance and training
Review evidence requirements
Design evidence
- Capability inventory and threat model
- Tool-by-tool authorization policy
- Credential, network, filesystem, memory, and side-effect boundaries
Implementation evidence
- Allowlisted tool adapters and parameter validation
- Just-in-time scoped credentials outside model context
- Read-only and network-denied defaults
Test evidence
- Unknown tool, invalid parameter, unauthorized destination, stale request, and confused-deputy fixtures
- Privilege and isolation tests
- No-side-effect simulation
Operational evidence
- Tool request, validated parameters, actor, authorization, result, and denial logs
- Credential scope and rotation evidence
- Capability exception review
Known limitation: A local deterministic agent simulation can demonstrate application-state transitions but not production model behavior, identity, credential isolation, or tool side effects.
Human Confirmation
Require an accountable human or policy authority to approve consequential actions through an exact transaction preview outside the untrusted evidence channel.
- Evidence plane
- operational evidence
- Remediation target
- Excessive agency, silent high-impact actions, and model-generated authorization errors.
- Maturity dimensions
- Retrieval and agent isolation, Governance and training, Incident response
Review evidence requirements
Design evidence
- Consequential-action inventory and approval thresholds
- Decision rights and separation of duties
- Preview and out-of-band confirmation design
Implementation evidence
- Non-bypassable confirmation gate
- Raw validated parameters and destination preview
- Authorization token or accountable approval record
Test evidence
- Approve, deny, modify, timeout, stale, replay, and missing-authority tests
- Prompt-injected and misleading-preview tests
- Accessibility and usability tests
Operational evidence
- Who approved what exact action, parameters, destination, policy, and time in the organizational system of record
- Denied and expired approvals
- Exception and escalation review
Known limitation: The local workflow can model the need for confirmation but cannot establish identity, legal authority, informed consent, or non-bypassability in production.
Output Validation
Validate structured output, destinations, links, active markup, sensitive data, and action parameters before rendering, transmission, storage, or tool use.
- Evidence plane
- decoded extracted
- Remediation target
- Data exfiltration, unsafe rendering, malformed structured output, and downstream application-layer injection.
- Maturity dimensions
- AI context construction, Retrieval and agent isolation, Monitoring and evidence
Review evidence requirements
Design evidence
- Output schema, destination policy, sensitive-data categories, and safe-rendering contract
- Error and fallback behavior
- Human review boundary
Implementation evidence
- Strict schema validation, inert rendering, link/destination allowlists, escaping, CSP, and sensitive-data checks
- No automatic remote resource loading
- Bounded error output
Test evidence
- Malformed JSON, unexpected fields, Markdown/HTML injection, external link, secret marker, and over-limit output fixtures
- Rendering and accessibility tests
- Blocked-destination tests
Operational evidence
- Validated output identity, destination, policy result, redaction, and blocked reason
- False-positive/negative review
- Renderer and policy version
Known limitation: Local bounded checks cannot establish every sensitive-data category, production renderer behavior, destination reputation, or downstream consumer semantics.
Egress Controls
Route network and external communication through explicit destination, method, data-classification, and authorization policy rather than allowing model-generated destinations.
- Evidence plane
- behavior capability
- Remediation target
- Exfiltration through URLs, remote resources, mail, tool calls, registries, and attacker-controlled infrastructure.
- Maturity dimensions
- Retrieval and agent isolation, Monitoring and evidence, Incident response
Review evidence requirements
Design evidence
- Destination and protocol allowlist
- Data-classification and egress policy
- Proxy, DNS, mail, and tool mediation architecture
Implementation evidence
- Default-deny outbound proxy or equivalent policy
- Validated destination and method
- No direct model or parser network access
Test evidence
- Unknown domain, redirect, encoded destination, image/Markdown fetch, mail, DNS, and alternate-protocol tests
- Bypass and fail-closed tests
- Incident containment drill
Operational evidence
- Outbound request, destination, data class, authorizer, policy result, and block reason
- Allowlist change history
- Alert and containment records
Known limitation: The local runtime performs no external request, so it can describe and simulate egress policy but cannot verify production proxy, DNS, mail, or network enforcement.
Logging and Evidence Preservation
Maintain a Minimum Run Record containing raw and transformed identities, parser/model versions, representation views, decisions, tool events, errors, and authority gaps without destroying source evidence.
- Evidence plane
- operational evidence
- Remediation target
- Loss of forensic evidence during normalization, transformation, model assembly, or incident response.
- Maturity dimensions
- Monitoring and evidence, Incident response, Governance and training, Inventory and ownership
Review evidence requirements
Design evidence
- Minimum Run Record schema
- Retention, privacy, access, immutability, and deletion policy
- Required parser/model/tool/version fields
Implementation evidence
- Append-only or otherwise protected records
- Raw and transformed SHA-256 values
- Request, decision, tool, exception, and authority-gap linkage
Test evidence
- Deterministic record, tamper, omission, malformed, rollover, access, deletion, and restoration tests
- Incident replay
- Cross-artifact identity verification
Operational evidence
- Complete run records and retention evidence
- Review and access logs
- Incident handoff, recovery replay, and lessons-learned records
Known limitation: Deterministic local records prove only their own bytes and declared inputs; production completeness, immutability, access control, retention, and legal sufficiency require organizational evidence.
Observable dimensions
Keep maturity dimensions separate from control evidence states.
The source report defines nine maturity dimensions. The matrix uses them as organization and ownership lenses, not as a single grade or certification.
inventory-ownershipInventory and ownership
Assets, boundaries, versions, source identities, and accountable owners are known and reviewable.
input-controlsInput controls
Acquisition, type, encoding, size, canonicalization, and active-content rules are explicit and bounded.
parser-representation-testingParser and representation testing
Independent parser, structural, rendered, semantic, metadata, and extracted views are exercised with deterministic fixtures.
ai-context-constructionAI context construction
Policy, user input, retrieved evidence, metadata, model templates, and tool results remain provenance-labelled and structurally separated.
retrieval-agent-isolationRetrieval and agent isolation
Retrieval access, tool capability, credentials, egress, memory, and consequential actions are separately constrained.
provenance-supply-chainProvenance and supply chain
Artifacts, dependencies, models, datasets, manifests, signatures, and transformation histories have explicit identity and verification boundaries.
monitoring-evidenceMonitoring and evidence
Minimum run records, hashes, versions, outcomes, exceptions, and evidence gaps are preserved for review.
incident-responseIncident response
Triage, preservation, containment, authority, recovery, replay, and lessons-learned procedures are established.
governance-trainingGovernance and training
Decision rights, review cadence, training, exceptions, and accountable acceptance of residual uncertainty are documented.
Non-numeric state vocabulary
Six states describe different evidence—not better and worse scores.
A production reviewer may need several states at once across different controls. The workflow preserves that structure rather than averaging it away.
A policy, design, owner, or intended control is documented, but the local workflow does not establish implementation or efficacy.
A bounded Machine Tradecraft fixture and allowlisted local method demonstrate a narrow behavior under the declared release.
A named production, organizational, cryptographic, registry, DNS, browser, parser, or model authority has established the evidence outside this local runtime.
The evidence expected for this review has not been supplied or identified.
The control does not apply to the selected system boundary, with the reason retained in the plan.
The current evidence state is unresolved; absence has not been inferred and verification is still required.
System boundary profiles
Applicability changes when the system boundary changes.
A document parser, RAG assistant, browser agent, multimodal assistant, calendar assistant, and supply-chain pipeline do not require identical controls or evidence.
document-ingestionDocument-ingestion pipeline
A service that receives, inspects, extracts, transforms, indexes, or renders PDF, OOXML, image, archive, or related document containers.
Default accountable owner: System owner
Build this profile’s planweb-retrievalWeb-retrieval system
A crawler, browser-assisted extractor, search collector, or application that retrieves and converts web content into structured evidence.
Default accountable owner: System owner
Build this profile’s planrag-assistantRAG assistant
A retrieval-augmented assistant that indexes governed sources, retrieves chunks, constructs model context, and may propose or perform actions.
Default accountable owner: System owner
Build this profile’s planbrowser-agentBrowser agent
An agent that perceives DOM, accessibility, layout, pixels, network, and browser state and may interact with web applications.
Default accountable owner: System owner
Build this profile’s planmultimodal-assistantMultimodal assistant
A system that receives images or visual documents and combines pixels, OCR, alternative text, metadata, provenance, and model-facing representations.
Default accountable owner: System owner
Build this profile’s planemail-calendar-assistantEmail and calendar assistant
A system that parses MIME and iCalendar content, summarizes messages, drafts responses, proposes scheduling, or interacts with mail and calendar tools.
Default accountable owner: System owner
Build this profile’s plansoftware-ai-supply-chainSoftware and AI supply-chain pipeline
A build, packaging, registry, deployment, model, tokenizer, dataset, SBOM, attestation, or artifact-consumption pipeline.
Default accountable owner: System owner
Build this profile’s plan
Evidence requirement map
Use local fixtures to collect narrow evidence, then preserve the external gap.
The generated plan links each control to existing guides, reports, laboratories, actions, fixtures, and Casebook cases. Local evidence is useful only when its input, method, release, result, and limitations remain attached.
Declare the claim
Name the system boundary, control owner, exact statement, and expected evidence class.
Demonstrate locally
Use governed fixtures and allowlisted engines to establish a bounded behavior under this release.
Verify externally
Reproduce the relevant claim in the exact production browser, parser, model, registry, trust, or organizational system.
Preserve operationally
Retain run identity, versions, hashes, decisions, exceptions, ownership, review date, and recovery evidence.
External authority routing
Name the verifier instead of writing “needs verification.”
The plan keeps the authority class and description attached to every unresolved claim. It performs none of these external checks itself.
Deterministic record
Export the evidence plan without creating an assurance verdict.
Identical inputs under the same release produce the same JSON and Markdown bytes, Control Plan ID, and full SHA-256. The record contains no timestamp, request address, random value, account, session, or hidden history.
Source and profile
Exact report identity, source sections, release, system boundary, review objective, and bounded context.
Control and evidence
Applicability, selected evidence state, four evidence classes, maturity dimensions, limitations, and review cadence.
Local and external routes
Existing guides, reports, labs, actions, fixtures, Casebook cases, organizational evidence, and exact authorities.
Non-conclusions
Null score, readiness, maturity, severity, probability, pass/fail, compliance, certification, assurance, intent, and authorization fields.
Governed source and implementation boundary
Preserve the source report; store derived evidence rules separately.
Source identity
- Report: Machine Tradecraft Defense Operations.
- SHA-256:
77ede5cba60a8121d3054a50969c1ab6950b046deb484f759b24d20790136d5e. - Source sections: Control Families, Maturity Model, and Assessment Evidence Requirements.
- The submitted report body remains byte-identical and is not silently rewritten.
Method boundary
- The matrix organizes evidence; it does not independently observe a production control.
- A local fixture does not establish control efficacy in a different parser, model, browser, network, or organization.
- Visitor-selected Externally verified remains an unauthenticated declaration until exact evidence is retained.
- The record cannot authorize deployment, recovery, communication, payment, publication, or containment.