Defensive Preprocessing for Hidden Instructions

Executive summary

Hidden-instruction defense should be treated as a document-forensics and provenance problem, not merely a prompt-classification problem. Indirect prompt injection arises because an AI system may interpret attacker-controlled document or webpage content as instructions rather than as untrusted data. The attack surface includes ordinary visible prose, but hidden or machine-only content is especially dangerous because it creates a discrepancy between what a human reviewer sees and what the AI pipeline receives. This trust-boundary failure has been demonstrated since the early indirect-prompt-injection literature and remains a central weakness of document- and agent-based systems. source

The strongest preprocessing design is therefore multi-view and differential:

1. Preserve the original bytes and provenance.
2. Parse the container without executing active content.
3. Construct multiple representations: original text, Unicode-normalized security views, structural extraction, human-rendered pixels, and OCR of those pixels.
4. Compare machine-visible content with human-visible content.
5. Detect structural hiding, active content, unusual Unicode, confusables, and metadata separately from semantic instruction detection.
6. Feed only a newly constructed, least-privileged canonical representation to the AI.
7. Quarantine rather than pass through when parsing, rendering, or cross-view comparison is unreliable.

This design is important because sanitization can otherwise erase the very evidence needed for detection. A July 2026 controlled PDF study, CrackedPDFs, explicitly found that flattening documents before guardrails inspect their structure can remove evidence that text was hidden; its best document-aware hybrid detector substantially outperformed text-only PromptGuard on its controlled benchmark, while the authors cautioned that these results do not establish broad real-world robustness. source

Unicode normalization should be used as an analysis transform, not as evidence destruction. NFC preserves canonical equivalence and is a sensible normal representation; NFKC additionally collapses compatibility distinctions such as full-width versus ordinary characters and is therefore valuable for security matching. NFD and NFKD are useful diagnostic decompositions. The raw string must nevertheless be retained because the fact that normalization changes a string is itself evidence, and NFKC can intentionally eliminate distinctions present in the source. Unicode defines NFD as canonical decomposition, NFC as canonical decomposition plus composition, NFKD as compatibility decomposition, and NFKC as compatibility decomposition plus canonical composition. source

Invisible characters must be classified contextually rather than blindly stripped. U+200B ZERO WIDTH SPACE and U+00AD SOFT HYPHEN can legitimately affect word or line breaking; ZWJ and ZWNJ have real orthographic and shaping functions; and bidirectional isolates are needed in mixed-direction text. Conversely, directional overrides such as U+202D LRO and U+202E RLO deserve a much stronger security signal: the Unicode Bidirectional Algorithm explicitly recommends avoiding overrides where possible because of security concerns. source

Confusable detection should use Unicode TR39-style skeletons but never act alone. Unicode specifies confusable skeletons and mixed-script detection, but also warns that its mappings can flag many legitimate strings. The best use in prose is therefore as an anomaly feature: normalize a token, compute its UTS #39 skeleton, inspect script mixing, and compare the skeleton against security-sensitive terms. source

HTML needs three distinct views: structural DOM content such as textContent, browser-rendered text such as innerText, and screenshot OCR. The HTML Standard defines innerText as text “as rendered” and omits non-rendered descendants in a rendered element, whereas DOM textContent represents descendant text independent of CSS visibility. The difference is therefore directly useful for discovering display:none, visibility:hidden, content-visibility:hidden, zero opacity, clipping, off-screen placement, or related hiding. Screenshot OCR adds coverage for CSS-generated content, canvas/SVG text and effects that DOM text APIs do not reliably model. source

PDFs require object-level inspection before flattening. Relevant structures include actions and JavaScript, optional-content groups/layers, annotations and their appearance streams, attachments, XObjects, fonts and ToUnicode mappings, XML/XMP metadata, and extracted text with zero opacity or other nonpainting behavior. PyMuPDF's text-tracing API exposes opacity, painting order and optional-content-group information and explicitly notes that it can detect text hidden by zero opacity, non-visible text types, or later overlapping objects. PDF's name dictionary can also contain document-level JavaScript and embedded-file trees. source

OCR is best used as an independent witness, not as a replacement for parsing. For every page in a high-assurance configuration, render pixels in a network-isolated process, OCR the pixels, and compare that result with extracted PDF text. OCRmyPDF's force-OCR mode illustrates the underlying principle: it rasterizes pages, discards hidden OCR text, rasterizes printable text and flattens interactive objects to their visual representation. source

Classifiers are a second line of defense. Meta Prompt Guard 2 provides purpose-built prompt-injection classifiers; LLM-based PromptArmor has reported very low error rates on AgentDojo; alignment-aware research such as AlignSentinel argues that the correct distinction is often aligned instruction vs. misaligned instruction vs. non-instruction, rather than simply “contains an instruction.” Yet recent evaluation work shows that ordinary train/test splitting can grossly overstate classifier generalization, and adaptive-attack studies continue to break defenses that depend on the language model protecting itself. source

The recommended architecture is consequently:

flowchart TD
    A[Untrusted document or webpage] --> B[Quarantine original bytes]
    B --> C[Hash, provenance, MIME and structure sniffing]
    C --> D{Container type}

    D -->|Plain text| E[Unicode and control-character audit]
    D -->|HTML| F[Static DOM parse + isolated rendering]
    D -->|PDF| G[PDF object-graph inspection + isolated rendering]
    D -->|Image| H[Decode + metadata inventory + OCR]

    F --> I[DOM text / rendered text / screenshot OCR]
    G --> J[Extracted text / structural evidence / page OCR]
    H --> K[OCR text]
    E --> L[Multi-view Unicode analysis]
    I --> L
    J --> L
    K --> L

    L --> M[Hiddenness and discrepancy scoring]
    M --> N[Instruction-classifier ensemble]
    N --> O{Security policy}

    O -->|Low risk| P[Construct canonical safe payload]
    O -->|Ambiguous| Q[Quarantine or human review]
    O -->|Active / hidden malicious / parser failure| R[Reject or quarantine]

    P --> S[Attach source and provenance boundaries]
    S --> T[AI system]

The architectural principle is more important than any individual detector: do not let raw untrusted document internals become model instructions merely because an extraction library returned them as text. Microsoft Spotlighting similarly treats provenance as a continuous signal intended to help a model distinguish sources, while architectural work on prompt-injection resistance argues for separating trusted control flow from attacker-controlled data. source

Threat model and security objectives

The relevant adversary should be assumed to know that the document will be processed by an AI and, for a high-assurance system, to know the broad preprocessing strategy. Static defenses should therefore be evaluated against adaptive attackers rather than only against a fixed set of obvious phrases. Recent evaluation work found defenses that appeared strong against static attacks could fail under iterative adaptive attacks, and another 2026 study showed that standard cross-validation can exploit dataset provenance rather than actual malicious semantics. source

The defender should pursue six properties simultaneously.

Evidence preservation. Raw bytes, hashes and important offsets/object references should be retained before any normalization or transformation. OWASP's RAG guidance explicitly recommends hashing documents at ingestion and recording provenance, while warning against ingestion without scanning. source

Human/machine view consistency. Content that an extractor claims exists but that cannot be accounted for in the rendered view should become suspicious evidence, not silently accepted content. CrackedPDFs is particularly relevant here because it frames hidden PDF injection as a gap between human-visible rendering and machine-visible extraction. source

No active execution. Document preprocessing should not execute attacker-supplied JavaScript, macros, PDF actions or arbitrary network requests merely to discover textual content. CSP can reduce browser execution privileges and resource loading, but W3C explicitly describes CSP as defense-in-depth rather than a replacement for input validation. source

Canonical output rather than trusted input mutation. The output sent to the AI should be a newly constructed representation—ideally structured plain text plus separately typed image/OCR regions—not a notionally “cleaned” copy of an attacker-controlled object graph.

Provenance preservation. Every span supplied to the AI should remain attributable to the user, a particular external document, OCR, an annotation, or another source. Spotlighting's core idea is similarly to transform inputs so that provenance remains continuously apparent to the model. source

Fail-safe ambiguity. Encryption without authorized decryption, unsupported document features, malformed parsers, rendering failures, major OCR/extraction disagreements, or detector outages should produce quarantine rather than raw-input passthrough.

There is an important limit to set explicitly: no document preprocessor can establish that all visible prose is benign. An attacker can put a perfectly visible sentence telling the AI to ignore the user's objective. Structural detection solves hiddenness; semantic and architectural defenses address instruction/data confusion. Adaptive-defense results are strong evidence against treating any single downstream guard model as a proof of safety. source

Unicode, invisible characters, controls, and homoglyphs

Unicode security should operate on parallel representations rather than selecting one normalization and discarding the rest.

These definitions and the distinction between canonical and compatibility equivalence come directly from Unicode UAX #15. Unicode also notes that NFC is commonly recommended for interoperable web content, while NFKC folds additional compatibility-equivalent forms. source

A useful detection feature is therefore:

\[
\Delta_\text{NFKC}(s) = [\,s \neq NFKC(s)\,]
\]

plus the number and location of changed spans. A change is not malicious by itself; it merely tells the classifier that what an ordinary security comparison sees differs from the exact source.

Characters of particular interest include the following.

The last two rows are recommended engineering policy, rather than Unicode requirements.

A defensive implementation can preserve raw positions while generating security views:

from __future__ import annotations

import unicodedata as ud
from dataclasses import dataclass


HIGH_RISK_FORMATS = {
    "\u202D": "LEFT-TO-RIGHT OVERRIDE",
    "\u202E": "RIGHT-TO-LEFT OVERRIDE",
    "\u202A": "LEFT-TO-RIGHT EMBEDDING",
    "\u202B": "RIGHT-TO-LEFT EMBEDDING",
    "\u202C": "POP DIRECTIONAL FORMATTING",
}

WATCH_FORMATS = {
    "\u00AD": "SOFT HYPHEN",
    "\u200B": "ZERO WIDTH SPACE",
    "\u200C": "ZERO WIDTH NON-JOINER",
    "\u200D": "ZERO WIDTH JOINER",
    "\u2066": "LEFT-TO-RIGHT ISOLATE",
    "\u2067": "RIGHT-TO-LEFT ISOLATE",
    "\u2068": "FIRST STRONG ISOLATE",
    "\u2069": "POP DIRECTIONAL ISOLATE",
    "\uFEFF": "ZERO WIDTH NO-BREAK SPACE / BOM",
}


@dataclass(frozen=True)
class Finding:
    offset: int
    codepoint: str
    name: str
    category: str
    severity: str


def inspect_unicode(text: str) -> tuple[dict[str, str], list[Finding]]:
    views = {
        form: ud.normalize(form, text)
        for form in ("NFC", "NFD", "NFKC", "NFKD")
    }

    findings: list[Finding] = []

    for i, ch in enumerate(text):
        cp = f"U+{ord(ch):04X}"
        category = ud.category(ch)
        name = ud.name(ch, "<unnamed>")

        if ch in HIGH_RISK_FORMATS:
            findings.append(
                Finding(i, cp, HIGH_RISK_FORMATS[ch], category, "high")
            )
        elif ch in WATCH_FORMATS:
            findings.append(
                Finding(i, cp, WATCH_FORMATS[ch], category, "medium")
            )
        elif category == "Cc":
            # Do not flag ordinary newline/tab here; policy handles them separately.
            if ch not in ("\n", "\r", "\t"):
                findings.append(Finding(i, cp, name, category, "high"))
        elif category == "Cf":
            # Broad catch-all. Cf has legitimate members, so this is only a signal.
            findings.append(Finding(i, cp, name, category, "low"))

    return views, findings

The crucial property of this code is not the particular severity levels; it is that detection happens before removal and positions remain attributable to the raw source.

Homoglyph and confusable handling. Unicode TR39 supplies a standardized notion of a confusable skeleton: two strings can be considered confusable when their security skeletons match. It also defines mixed-script and whole-script confusables. source A practical pipeline should compute skeletons token-by-token and test them against a small high-risk vocabulary—for example terms associated with role changes, tool use, secrets, system messages or overriding instructions—rather than attempting to classify all prose as spoofed.

Conceptually:

for each token t:
    raw        = t
    compat     = NFKC(t)
    skeleton   = UTS39_CONFUSABLE_SKELETON(compat)
    scripts    = RESOLVED_SCRIPT_SET(t)

    features += {
        raw != compat,
        raw != skeleton,
        mixed_script(scripts),
        skeleton in sensitive_instruction_lexicon,
        edit_distance(raw, skeleton)
    }

ICU's SpoofChecker is the preferred concrete implementation family for Unicode security checks rather than maintaining an ad hoc confusables table. UTS #39 itself warns that confusable mappings can be overly inclusive and that mixed-script checks may flag legitimate strings, particularly across Latin and Cyrillic, so a hit should raise risk rather than automatically reject a document. source

A robust control-removal strategy consequently has three outputs:

This avoids two common failures: blindly stripping ZWJ/ZWNJ and breaking legitimate languages, or normalizing immediately and losing proof that an attacker used an obfuscated representation. Unicode itself stresses both legitimate joining-control contexts and the security need for special handling. source

Web and HTML defenses

HTML is dangerous precisely because DOM existence is not equivalent to visual existence. The DOM Standard's textContent walks descendant textual content; the HTML Standard's innerText, by contrast, represents text as rendered and excludes non-rendered descendants when queried on a rendered element. This gives a useful first-order differential detector. source

CSS offers many ways to create disagreement. display:none suppresses box generation, visibility:hidden prevents a generated box from being rendered, content-visibility:hidden skips content, and zero opacity can also make an element visually absent; browser visibility APIs explicitly account for content visibility, opacity and CSS visibility. source Clipping, transforms, zero/tiny dimensions, overflow and matching foreground/background colors add further hiding possibilities. CSS should therefore be considered evidence during visibility analysis, even though it should normally be removed from the final AI-bound HTML representation.

A good web preprocessor obtains at least these views:

The final decision should use the intersection and discrepancies, not declare one view authoritative.

A defensible Playwright-style audit looks like this:

import { chromium } from "playwright";

export async function inspectHtml(html) {
  const browser = await chromium.launch({ headless: true });

  const context = await browser.newContext({
    javaScriptEnabled: false,
    // Do not attach cookies, credentials, storage state, etc.
  });

  const page = await context.newPage();

  // Network isolation is independent of CSP and is intentionally redundant.
  await page.route("**/*", route => route.abort());

  await page.setContent(html, { waitUntil: "domcontentloaded" });

  const analysis = await page.evaluate(() => {
    const body = document.body;
    const rawDomText = body?.textContent ?? "";
    const renderedText = body?.innerText ?? "";

    const suspiciousVisibility = [];

    for (const el of body?.querySelectorAll("*") ?? []) {
      const cs = getComputedStyle(el);
      const r = el.getBoundingClientRect();

      const hidden =
        cs.display === "none" ||
        cs.visibility === "hidden" ||
        cs.visibility === "collapse" ||
        Number.parseFloat(cs.opacity || "1") <= 0.01 ||
        r.width === 0 ||
        r.height === 0;

      if (hidden && (el.textContent ?? "").trim()) {
        suspiciousVisibility.push({
          tag: el.tagName,
          id: el.id,
          className: el.className,
          text: el.textContent.slice(0, 500),
          display: cs.display,
          visibility: cs.visibility,
          opacity: cs.opacity,
          rect: {
            x: r.x, y: r.y, width: r.width, height: r.height
          }
        });
      }
    }

    return { rawDomText, renderedText, suspiciousVisibility };
  });

  const screenshot = await page.screenshot({
    fullPage: true,
    type: "png"
  });

  await browser.close();

  return { ...analysis, screenshot };
}

Playwright provides both browser evaluation and screenshot functionality suitable for constructing this independent rendered view. source In production, a second pass should additionally test clipping, viewport intersection, transforms, font-size, text/background contrast, text-indent, overflow, pseudo-elements, SVG, canvas, shadow DOM and overlapping/occluding elements.

Script and style handling requires sequencing. Do not first delete all styles and then ask whether text was visible: that can turn an attacker-hidden node into apparently ordinary visible text and erase the hiding evidence. Instead:

Raw HTML
  → non-executing structural parse
  → render with CSS under network isolation and JavaScript disabled
  → record visible/non-visible differences
  → screenshot + OCR
  → classify discrepancies
  → only then construct a CSS-free sanitized payload

A DOM sanitizer such as DOMPurify is appropriate for producing a restricted HTML representation: DOMPurify is explicitly designed as a DOM-based sanitizer with a secure default. source It should nevertheless be viewed as an XSS/content sanitizer, not a prompt-injection detector. The safest AI payload is usually structured plain text; where HTML must be retained, use a narrow allowlist and remove scripts, styles, event-handler attributes, forms and active embedded content.

CSS hiding heuristics should include at minimum display:none, visibility:hidden/collapse, content-visibility:hidden, opacity near zero, zero-size boxes, clipping to effectively empty regions, off-viewport absolute/fixed positioning, transforms that move content outside the viewport, tiny font sizes, text/background colors with effectively no contrast, overflow clipping, and content obscured by later opaque elements. W3C specifications establish the underlying rendering behavior for display, visibility, content visibility and opacity. source

Treat CSS ::before/::after generated text separately. DOM descendant-text extraction may not reproduce generated painted content, while screenshot OCR can. Conversely, alt, ARIA labeling and other accessibility-only information may be machine-visible yet absent from screenshot OCR. The pipeline should therefore attach an explicit provenance class such as VISIBLE_TEXT, ACCESSIBILITY_TEXT, HIDDEN_DOM, or OCR_ONLY rather than indiscriminately concatenating all of them.

Data URIs are real embedded objects, not harmless strings. RFC 2397 permits inline data of the form data:[<mediatype>][;base64],<data>. source The defensive policy should parse the media type, apply decoded-size limits, decode into an inert buffer and recursively inspect the resulting content. Raster images can be separately decoded and OCRed; HTML, SVG/XML, script-like or unsupported data URIs should be rejected or quarantined rather than executed by the browser.

Iframes and embedded objects should become nested provenance boundaries. srcdoc is recursively scanned as another HTML input. A remote src should not be fetched automatically with the user's credentials; if retrieval is necessary, a separate stateless fetcher can obtain it and begin a new preprocessing transaction. <object> and <embed> should likewise not cause automatic execution.

CSP belongs in the sandbox but is not the sandbox. W3C describes CSP as a mechanism for controlling resource requests, inline script, dynamic code and related execution privileges, while explicitly warning that it is defense-in-depth rather than a first-line input validator. source A preprocessing renderer can enforce an aggressive policy such as conceptually:

Content-Security-Policy:
  default-src 'none';
  script-src 'none';
  object-src 'none';
  frame-src 'none';
  connect-src 'none';

CSS required for visibility analysis can be applied in a separate analysis environment while network I/O is independently blocked. The final sanitized payload should not depend on CSP to be safe.

The web-processing trade-offs are:

The recommended default is real-browser differential analysis plus OCR, with JavaScript and networking disabled. Because the user specified no compute constraint, there is little reason to skip the independent screenshot/OCR witness for high-value ingestion.

PDF, metadata, OCR, and visible-versus-extracted comparison

PDF deserves stronger treatment than “extract all strings and scan them.” Its logical object graph and rendered appearance can differ substantially. The PDF specification includes name-tree entries for document-level JavaScript and embedded files; files can also appear through file-attachment annotations, and XObjects can contain reusable graphical or image content. source

A PDF inspection pass should inventory these structures before any rasterization:

PDF fonts merit special emphasis. An embedded font is not itself suspicious; embedding is normal. The relevant anomaly is a disagreement among encoded character values, a font's character mapping, extracted Unicode and the pixels actually shown to the user. Rendering-to-image and OCR are a powerful independent check precisely because OCR sees the painted glyph shapes rather than trusting the PDF's claimed textual mapping.

PyMuPDF is especially useful for structural visibility analysis. Its text tracing exposes an opacity value where zero denotes invisible text, painting sequence information that can indicate that later objects cover a text span, and the Optional Content Group associated with text; its documentation explicitly calls out these mechanisms as ways to detect additional forms of text invisibility. source

Representative inspection code:

from __future__ import annotations

import pymupdf


def inspect_pdf_text_visibility(path: str) -> list[dict]:
    doc = pymupdf.open(path)
    findings: list[dict] = []

    for page_no, page in enumerate(doc):
        page_rect = page.rect

        for span in page.get_texttrace():
            opacity = float(span.get("opacity", 1.0))
            text_type = int(span.get("type", 0))
            ocg = span.get("layer") or span.get("ocg")

            chars = span.get("chars", ())
            text = "".join(
                chr(ch[0]) if isinstance(ch[0], int) else str(ch[0])
                for ch in chars
            )

            suspicious = (
                opacity <= 0.01 or
                text_type > 1
            )

            if suspicious and text.strip():
                findings.append({
                    "page": page_no + 1,
                    "text": text,
                    "opacity": opacity,
                    "text_type": text_type,
                    "optional_content_group": ocg,
                })

    return findings

The exact fields should be pinned to the deployed PyMuPDF version and tested against a corpus of generated PDFs; the important design is to retain page/object/span identity rather than only the final extracted string.

qpdf and pikepdf are complementary. qpdf can expose the underlying PDF in structured JSON, including otherwise difficult-to-see objects, while pikepdf provides a Python interface built around qpdf. qpdf's documentation notes that its JSON representation includes unreferenced objects and recommends removing object streams for easier inspection. source Do not rely on a raw grep for /JavaScript: object streams, indirection and encoded names make recursive parsed-object inspection the correct approach.

Conceptually:

parse PDF object graph
for every reachable and suspicious unreferenced object:
    record object id and type

    recursively examine dictionaries for:
        /JavaScript, /JS
        /OpenAction, /AA
        /Launch, /GoToR, /SubmitForm
        /EmbeddedFiles, /EF
        /RichMedia
        /OCProperties
        /Annots, /AP
        /XObject
        /Metadata

    do not execute actions
    do not recursively ingest attachments into the same trust context

Layers and annotations. A high-assurance system should consider rendering at least a default user-visible configuration and, for detection purposes only, an “expanded” inspection configuration that exposes otherwise optional content. Text appearing only in hidden/non-default OCGs becomes a strong hiddenness feature. Annotation textual contents and annotation appearances should be compared separately because one may exist without being visually prominent. PDF Association material documents the presence of optional content, attachments and rich-media/annotation structures in the format. source

OCR differential analysis should make the rendered page an independent information channel:

\[
E_p = \text{normalized extractor text on page }p
\]

\[
O_p = \text{normalized OCR of pixels rendered for page }p
\]

Useful metrics include:

\[
D_\text{char}(E,O)=
\frac{\operatorname{editDistance}(E,O)}
{\max(|E|,|O|,1)}
\]

and token overlap:

\[
J(E,O)=\frac{|tokens(E)\cap tokens(O)|}
{|tokens(E)\cup tokens(O)|}.
\]

Global metrics are only a triage aid. More important is span-level asymmetry:

extraction-only suspicious span:
    instruction classifier says span is suspicious
    AND extracted span has no acceptable OCR match
    AND PDF geometry/opacity/layer evidence indicates invisibility
    → very high risk

OCR-only suspicious span:
    OCR sees an instruction
    AND ordinary text extraction does not
    → inspect raster image/XObject or broken font mapping

large general disagreement:
    extractor and OCR disagree widely
    → parsing/font/OCR ambiguity; quarantine rather than choose one silently

The spatial version is stronger. Obtain word or character bounding boxes from the PDF extractor and word boxes from OCR, align them by page coordinates, and ask not merely “does this sentence occur?” but “is the same linguistic content present in approximately the same painted location?” This catches cases where an invisible extraction layer contains extra text elsewhere on the page.

For a compute-unconstrained high-assurance deployment, I recommend OCRing every rendered PDF page at roughly 300 DPI rather than OCRing only files that appear scanned. The 300 DPI figure is an engineering starting point, not a universal standard; very small typography may justify higher resolution.

A practical starting policy—not a published universal threshold—is:

These thresholds should be tuned against benign scans, multilingual documents, equations and low-quality historical material rather than treated as standardized security values.

OCRmyPDF provides a useful canonicalization primitive. Its redo mode distinguishes visible and invisible text and removes invisible OCR layers before re-OCR, while its force mode rasterizes pages, discards hidden OCR text, rasterizes printable text and flattens interactive objects to their visual representation. source For the highest-assurance AI payload, an attractive design is therefore:

original PDF (retained only in quarantine/audit storage)
          |
    structure scan
          |
    render pages
          |
       page images
          |
          +----> OCR text + spatial boxes
          |
          +----> optional new image-only PDF / fresh OCR layer

This does not mean rasterization is perfect. It loses hyperlinks, semantic tagging, digital-signature meaning, some accessibility information and editability; renderer vulnerabilities remain a consideration. It does, however, collapse much of PDF's hidden logical state into the visual surface that a human would ordinarily review. The correct operating model is to retain the original for forensic purposes while never letting its unchecked object graph become AI context.

Metadata should be excluded by default, not merely cleaned opportunistically. Adobe describes XMP as an extensible mechanism for embedding metadata such as titles, descriptions, keywords and author information into files. PDFs can have both XMP and the older DocumentInformation dictionary. source Open XML documents likewise store core, custom and other document properties in package parts, and Microsoft's Open XML SDK exposes the ZIP/XML package structure programmatically. source

A metadata policy should distinguish inventory from exposure:

before sanitization:
    record metadata namespaces, field names, lengths and hashes
    optionally store sensitive values in restricted forensic storage

AI-bound artifact:
    omit XMP
    omit EXIF textual/comment/GPS fields
    omit PDF document properties
    omit OOXML core/custom/extended properties
    omit unsupported/custom XML unless explicitly part of the user's task

For images, ExifTool is valuable for metadata inspection, but its own documentation warns that deleting “all metadata” is not guaranteed to remove every kind of metadata from every file. source Consequently, decode → apply rendering-critical orientation/profile behavior → re-encode into a new minimal image provides a stronger canonicalization boundary than repeatedly deleting tags in place.

For PDFs, pikepdf directly exposes XMP and the older document-information model. source For DOCX/XLSX/PPTX, the Microsoft Open XML SDK is preferable to treating OOXML as an opaque file; Office Open XML is itself a ZIP/XML package, which makes explicit part-by-part policy possible. source

Suspicious-instruction classifiers and evaluation

A good classifier should answer a narrower and more context-sensitive question than “does this text contain an imperative sentence?” Manuals, emails, contracts, code, security reports and troubleshooting instructions contain perfectly legitimate imperatives. AlignSentinel's 2026 formulation is useful precisely because it distinguishes misaligned instructions, aligned instructions and non-instruction text, addressing false positives caused by treating every instruction as malicious. source

The classifier should receive both text and document-forensic features.

Textual/semantic features include instruction-like verbs, role references (system, assistant, user, developer, tool), attempts to override earlier directives, requests to disclose secrets or hidden context, claims of authority, requests for tool execution, and semantic divergence from the user's actual task. OWASP similarly recommends scanning ingested material for injection markers, while recent classifier work emphasizes that semantic generalization rather than dataset-specific surface cues is the core challenge. source

Obfuscation features should include the number and density of Unicode controls, NFKC change rate, confusable skeleton changes, mixed-script anomalies, zero-width/soft-hyphen density, base64/hex-like sections, unusual markup and suspicious instruction matches that appear only after canonicalization.

Visibility features should be first-class classifier inputs: raw DOM vs innerText difference, screenshot-OCR difference, count/length of CSS-hidden nodes, PDF zero-opacity spans, non-default OCG text, out-of-bounds text, extraction/OCR disagreement, annotation-only text, and metadata-only text. The 2026 CrackedPDFs study supports exactly this direction: its document-aware hybrid detector did substantially better than a text-only guardrail within its controlled PDF benchmark, while pure structural and shortcut-prone text models had important limitations. source

Provenance features include whether a span came directly from the user's trusted instruction, an uploaded attachment, a third-party webpage, an OCR image, a hidden DOM node, a PDF annotation, an attachment inside another attachment, or metadata. Provenance should not simply be a classifier feature—it should also directly constrain downstream privileges. Spotlighting's approach similarly introduces transformations that continuously identify the source of input. source

The major model families have materially different trade-offs:

PromptArmor is a useful example of the guard-LLM approach: the authors report false-positive and false-negative rates below 1% on AgentDojo for several guard models and an attack-success rate below 1% after removal. Those figures should be read as benchmark-specific, not as production guarantees. source CAPTURE, in contrast, reports that guardrails can simultaneously exhibit false negatives on adversarial prompts and excessive false positives on benign context-aware cases. source

Recent work gives an especially strong warning about benchmark methodology. A July 2026 study assembled 18 datasets totaling roughly 105,000 samples and found standard cross-validation yielded 0.996 AUC while leave-one-dataset-out evaluation fell to 0.912—a difference of 8.4 percentage points. The authors attribute this to classifiers exploiting dataset-identity cues rather than attack semantics. source Thus, a detector intended to protect real documents should never be certified from a random train/test split of pooled attack and benign datasets alone.

A robust training corpus should instead contain:

The paired-counterfactual recommendation is strongly supported by CrackedPDFs, which deliberately used benign matched confounders and base-document-grouped splits to prevent trivial leakage. source

Evaluation should report precision, recall, PR-AUC, false-positive rate at realistic prevalence, false-negative rate, attack success after filtering, benign-task utility and retained-content quality. The base rate matters greatly. For example, suppose only 0.1% of incoming documents are truly malicious. Even a detector with 99% recall and a 1% benign false-positive rate would, per 10,000 documents, produce roughly 10 true-positive detections but about 100 false positives. Only about 9% of alerts would actually be attacks. Therefore production tuning should target very low FPRs and use contextual escalation rather than assuming a seemingly impressive “99% accuracy” is operationally sufficient.

The cleanest false-positive mitigation is to maintain two conceptually separate scores:

\[
H = P(\text{content is hidden/obfuscated or view-inconsistent})
\]

\[
I = P(\text{content is a misaligned instruction}\mid
\text{task, source, provenance})
\]

Then policy can reason about combinations:

This is substantially safer than a single binary “malicious” probability.

Finally, preprocessing should not be the only security boundary. A 2026 adaptive evaluation concluded that every defense in its experiment that relied on the model itself to protect a secret eventually failed. source The downstream application should therefore also minimize the model's authority: untrusted content should not be allowed to decide tool calls, recipients, payment destinations, secret access or irreversible actions merely because it survived document scanning.

Secure preprocessing architecture, defaults, and implementation recommendations

A production implementation is best divided into components so that the parser, renderer, classifier and AI runtime do not share unnecessary privileges:

flowchart LR
    SRC[Untrusted Sources<br/>uploads / web / email / RAG] --> GW[Ingress Gateway]

    GW --> RAW[(Immutable Raw Store)]
    GW --> TYPE[Type / Container Inspector]

    TYPE --> UP[Unicode Processor]
    TYPE --> HP[HTML Static Parser]
    TYPE --> PP[PDF Object Inspector]
    TYPE --> IP[Image Decoder]

    HP --> RENDER[Isolated Renderer]
    PP --> RENDER
    IP --> OCR[OCR Engine]
    RENDER --> OCR

    UP --> FEAT[Security Feature Store]
    HP --> FEAT
    PP --> FEAT
    OCR --> FEAT

    FEAT --> CLASS[Instruction / Alignment Classifiers]
    CLASS --> POLICY[Policy Engine]

    POLICY -->|allow| CANON[Canonical Payload Builder]
    POLICY -->|review| REVIEW[Human Review Queue]
    POLICY -->|reject| QUAR[Quarantine]

    CANON --> LLM[AI Runtime]
    LLM --> TOOLPOL[Independent Tool Authorization]

    GW --> AUDIT[(Tamper-evident Audit Log)]
    TYPE --> AUDIT
    RENDER --> AUDIT
    CLASS --> AUDIT
    POLICY --> AUDIT

The relationship that matters is that the raw object never travels directly from ingress to the LLM. The AI receives a separate canonical entity whose lineage points back to the original.

A recommended processing sequence is:

UNTRUSTED BYTES
    │
    ├─ compute cryptographic hash
    ├─ record uploader/source/time/provenance
    ├─ sniff actual format; do not trust extension
    └─ store immutable original
             │
             ▼
    CONTAINER-SPECIFIC STRUCTURAL AUDIT
             │
             ├─ HTML: DOM, links, CSS, frames, active content
             ├─ PDF: object graph, actions, layers, annotations, fonts, XObjects
             └─ image/OOXML/etc.: metadata and embedded-object inventory
             │
             ▼
    MULTIPLE INDEPENDENT CONTENT VIEWS
             │
             ├─ raw extracted text
             ├─ NFC/NFKC/confusable security forms
             ├─ human-rendered text
             └─ render-to-image OCR
             │
             ▼
    DIFFERENTIAL / HIDDENNESS ANALYSIS
             │
             ▼
    METADATA AND ACTIVE-CONTENT EXCLUSION
             │
             ▼
    MISALIGNED-INSTRUCTION CLASSIFIER ENSEMBLE
             │
             ▼
          POLICY
      ┌──────┼─────────┐
      │      │         │
    allow  review   quarantine
      │
      ▼
    NEW CANONICAL PAYLOAD
      │
      ├─ visible content only
      ├─ source/span provenance
      ├─ OCR explicitly labeled
      └─ no executable behavior
      │
      ▼
    AI + independent authorization boundary

The following defaults are appropriate for a high-assurance deployment with no material compute constraint:

An implementation can formalize the policy without tying it to one classifier score:

from dataclasses import dataclass


@dataclass
class Risk:
    parser_ok: bool
    active_content: bool
    encrypted_uninspected: bool

    unicode_high_risk: bool
    hidden_instruction_span: bool
    extraction_ocr_instruction_mismatch: bool
    severe_view_disagreement: bool

    hiddenness_score: float      # calibrated 0..1
    misalignment_score: float    # calibrated 0..1


def decide(r: Risk) -> str:
    # Fail-safe conditions.
    if not r.parser_ok or r.encrypted_uninspected:
        return "QUARANTINE"

    # Executable document behavior is unnecessary for AI text ingestion.
    if r.active_content:
        return "QUARANTINE"

    # Cross-view evidence of an actual instruction is stronger than a
    # document-wide similarity metric.
    if r.hidden_instruction_span:
        return "QUARANTINE"

    if r.extraction_ocr_instruction_mismatch:
        return "QUARANTINE"

    # Ambiguity must not degrade into raw passthrough.
    if r.severe_view_disagreement:
        return "REVIEW"

    # Example starting bands only: calibrate on deployment data.
    combined = max(
        r.misalignment_score,
        0.6 * r.misalignment_score + 0.4 * r.hiddenness_score,
    )

    if combined >= 0.85:
        return "QUARANTINE"
    if combined >= 0.40 or r.unicode_high_risk:
        return "REVIEW"

    return "ALLOW"

The numerical bands are intentionally illustrative. Production thresholds should be selected from the deployment's observed benign prevalence and acceptable false-positive rate, then continuously tested against held-out sources and adaptive attacks. Recent distribution-shift research makes clear why adopting a benchmark's threshold unchanged is unsafe. source

Logging should be forensic but privacy-minimizing. Each transaction should record at least the original and canonical hashes, declared and detected media type, parser/renderer/OCR/library versions, structural anomaly types, Unicode code point and offset findings, DOM selectors or PDF page/object/bounding-box identifiers, OCR/extraction discrepancy metrics, classifier versions and scores, final policy decision, and the canonical-output hash. Sensitive full text need not be duplicated in ordinary logs; retain content in a separately access-controlled evidence store.

Versioned reproducibility is important. When a later incident is investigated, the security team should be able to answer: which parser produced this extraction, which Unicode data version supplied the confusable mapping, which renderer generated the screenshot, which classifier model and threshold were applied, and exactly which canonical bytes reached the AI. This is particularly important because Unicode security data evolve and learned detectors are vulnerable to distribution shift. source

The principal tools and primary references are:

The comparative security value of the complete defense stack is:

The resulting security doctrine is straightforward: preserve first, inspect structure second, render independently third, compare views fourth, sanitize fifth, classify sixth, and only then construct an AI-specific payload. Sanitization without detection can erase forensic evidence; classification without rendering cannot tell visible text from hidden text; OCR without structural parsing misses active and nonvisual state; Unicode stripping without linguistic context damages legitimate text; and a language-model guard without deterministic boundaries remains susceptible to adaptive prompt injection. The layered approach is consistent with Unicode's security mechanisms, W3C's defense-in-depth position on browser content, modern PDF visibility tooling, document-aware 2026 benchmark results, and the continuing evidence that prompt-injection classifiers and model-side defenses should be treated as fallible security components rather than proofs of safety. source