Global site search

Search guides, labs, glossary, and research

Type two or more characters to search.

Start with a channel, artifact, or defense term

Examples include zero-width, metadata, tokenizer, or prompt injection.

    RAG · TOOLS · MEMORY · EGRESS

    Retrieval and Agent Trust Boundaries

    Model indirect prompt injection as an application trust-boundary failure across retrieval, context assembly, tools, credentials, memory, output, and egress.

    Quick answer

    What does this Machine Tradecraft expansion explain?

    Indirect prompt injection occurs when an AI system retrieves attacker-controlled content and treats embedded instructions as authority; durable defense requires provenance-labelled context, least-privilege tools, transaction previews, human approval, output validation, memory isolation, and egress control outside the model.

    Assume compromise
    A model may follow an injected instruction even when the surrounding application is functioning as designed.
    Contain impact
    The capability boundary determines whether a successful injection becomes a real side effect.
    Preserve evidence
    Record the retrieved sources, assembled context, proposed tool call, policy decision, and result.
    Reviewed implementation boundary

    Know what is measured, approximated, and still external.

    The submitted report remains byte-identical. This separate review, checked 2026-08-26, narrows implementation claims and gives visitors a decision path before they generalize from a local result.

    IMPLEMENTATION MODE Deterministic application-control state machine
    VISIBLE SOURCE PROFILE 0 standards/specifications · 3 research · 1 government · 3 implementation
    SOURCE BODY Preserved; corrections live in this review layer
    Directly computed

    Output produced deterministically by the local runtime.

    • The selected scenario and five explicit application-control states
    • A deterministic proposed action, policy trace, control-gap count, and allow/block result
    • Proof that the local simulation used no model, credential, write tool, network request, or external side effect
    Bounded approximation

    Useful model or subset that must not be mistaken for full conformance.

    • The state machine does not model language-model probabilities, instruction hierarchy, jailbreak behavior, or adaptive attackers
    • The control-gap score is a local scenario rule, not a risk score or attack-success estimate
    • Mock tools and evidence bundles do not reproduce a production agent framework
    Requires external verification

    Conclusion that needs an exact implementation, trust system, model, parser, or human review.

    • Model susceptibility requires controlled evaluation of the exact model, prompt stack, retriever, tools, permissions, and versions
    • Tool authorization and confirmation must be verified at the actual execution boundary
    • Incident and benchmark claims require primary evidence and current vendor or vulnerability records
    Decision support

    Choose the next evidence step instead of treating one result as a verdict.

    QuestionWhat the local page can answerWhat it does not establishNext evidence step
    Would this fixed application configuration permit the simulated transition?Yes — the deterministic trace answers that exact scenario.Whether a real model would attempt the action.Evaluate the exact production model and tool mediator in a side-effect-free environment.
    Which application control blocks the path?The trace identifies the configured blocking stage.Control efficacy against adaptive inputs.Red-team each boundary and verify fail-closed behavior.
    Is the agent secure?No global verdict is issued.Residual prompt-injection, retrieval, memory, permission, or egress risk.Use a complete threat model and evidence-backed control assessment.
    Focused deterministic fixture

    Fixed harmless marker across web, email, and retrieval scenarios

    Compares application-layer controls without executing arbitrary instructions or tools.

    Expected boundary: Strong configurations block the fixed transition; weak configurations report a simulated open path with zero real side effect.

    Open prepared laboratory
    Version and authority checkpoints
    • OWASP LLM01Current security guidance; not a conformance certification
    • NIST AI 100-2e2025Government adversarial-machine-learning taxonomy used for threat terminology
    • InjecAgentPrimary benchmark research; results are not reproduced by this deterministic lab
    Compare independent views

    One artifact, several machine-readable representations

    No single view is automatically authoritative. Preserve the source, identify each parser or receiver, and compare their outputs before authorizing a consequential decision.

    01Trusted task

    The user-authorized objective and constraints.

    02Retrieved evidence

    Source-labelled untrusted documents and metadata.

    03Assembled context

    The exact representation delivered to the model.

    04Proposed action

    A tool, parameters, destination, and expected effect.

    05Policy decision

    Allowlist, confirmation, egress, and memory controls.

    06Run record

    Sources, decision trace, blocked or allowed result, and fixed marker.

    Bounded method

    Analysis workflow

    The workflow preserves evidence before transformation and keeps structural inspection separate from execution, remote verification, or model behavior.

    1. Classify every context span by source and authority.
    2. Keep the reader uncredentialed and tools outside the model boundary.
    3. Validate tool names and parameters against a narrow allowlist.
    4. Show a raw transaction preview for consequential actions.
    5. Require explicit human approval and enforce destination/egress policy.
    6. Log retrieval, context, tool proposal, decision, and result.
    Defense in depth

    Controls carried into implementation

    These controls are contextual. They reduce a defined risk; they do not guarantee safety, truth, attribution, or resistance to every adaptive attack.

    01

    Use read-only defaults and just-in-time scoped credentials.

    02

    Treat retrieved text, metadata, email, web content, and tool output as untrusted data.

    03

    Prevent arbitrary tool-definition or memory changes by model output.

    04

    Disable automatic remote media and Markdown exfiltration paths.

    05

    Assume a model-level defense can fail and contain the resulting authority.

    Shared vocabulary

    Key terms

    Definitions are linked into the site-wide glossary and back to the full report.

    Capability boundary

    The enforced set of tools, parameters, data, and side effects available to an agent.

    Egress control

    Policy restricting outbound destinations and data carried by tool or rendering actions.

    Memory poisoning

    Insertion of attacker-controlled content into a persistent or repeatedly retrieved agent memory.

    Retrieval poisoning

    Manipulation of a knowledge source or ranking path so malicious content enters model context.

    Tool mediation

    A deterministic policy layer between model output and external action.

    Confused deputy

    A component misusing legitimate authority on behalf of an untrusted requester.

    Continue with primary material

    External standards and research

    These links are provided for visitors who want the governing specification, paper, framework, or implementation documentation. Links open in a new tab; the site does not fetch them during runtime analysis.

    Primary research Research paper

    Indirect Prompt Injection research

    Foundational application-integrated threat model.

    Original research or formal conference publication.
    arxiv.org
    Primary research Research paper

    InjecAgent

    Benchmark for indirect injection in tool-using agents.

    Original research or formal conference publication.
    arxiv.org
    Authoritative guidance Security guidance

    OWASP Prompt Injection Prevention Cheat Sheet

    Application-layer controls and limitations.

    First-party guidance, framework, registry, or standards-program material.
    cheatsheetseries.owasp.org
    Authoritative guidance Security guidance

    OWASP LLM01 Prompt Injection

    Risk definition and mitigation context.

    First-party guidance, framework, registry, or standards-program material.
    genai.owasp.org
    Authoritative guidance Security guidance

    OWASP AI Agent Security Cheat Sheet

    Tool, memory, identity, and orchestration controls.

    First-party guidance, framework, registry, or standards-program material.
    cheatsheetseries.owasp.org
    Authoritative guidance Government report

    NIST Adversarial Machine Learning Taxonomy

    Taxonomy for evasion, poisoning, and generative-AI attacks.

    First-party guidance, framework, registry, or standards-program material.
    csrc.nist.gov
    Authoritative guidance Knowledge base

    MITRE ATLAS

    Knowledge base of adversarial AI techniques and mitigations.

    First-party guidance, framework, registry, or standards-program material.
    atlas.mitre.org
    Implementation reference Vendor research

    Anthropic Prompt Injection Defenses

    Vendor account of browser-agent mitigations and residual risk.

    Tool, vendor, or implementation documentation; behavior is version-specific.
    www.anthropic.com
    Continue the investigation

    Read the evidence, then test the bounded model

    The full submitted report is preserved byte-for-byte in the governed research library and in durable repository documentation. The laboratory turns selected concepts into deterministic local output without external calls or hidden persistence.

    Detailed report

    Indirect Prompt Injection, Retrieval Poisoning, and Agent Trust Boundaries

    A defense-in-depth analysis of direct and indirect prompt injection, retrieval poisoning, tool scopes, credentials, memory, transaction confirmation, output validation, egress, detection, incident response, and deterministic simulation.

    Read governed report
    Focused laboratory

    Retrieval and Agent Trust-Boundary Simulator

    Compare deterministic application-layer outcomes for source labelling, instruction/data separation, tool allowlisting, human confirmation, egress filtering, and bounded memory using a fixed harmless marker.

    Open bounded laboratory