# Defensive Preprocessing for Hidden Instructions in Documents and Webpages Sent to AI Systems

## Executive summary

Hidden-instruction defense should be treated as a **document-forensics and provenance problem**, not merely a prompt-classification problem. Indirect prompt injection arises because an AI system may interpret attacker-controlled document or webpage content as instructions rather than as untrusted data. The attack surface includes ordinary visible prose, but hidden or machine-only content is especially dangerous because it creates a discrepancy between what a human reviewer sees and what the AI pipeline receives. This trust-boundary failure has been demonstrated since the early indirect-prompt-injection literature and remains a central weakness of document- and agent-based systems. citeturn22view5turn15view10turn22view1

The strongest preprocessing design is therefore **multi-view and differential**:

1. Preserve the original bytes and provenance.
2. Parse the container without executing active content.
3. Construct multiple representations: original text, Unicode-normalized security views, structural extraction, human-rendered pixels, and OCR of those pixels.
4. Compare machine-visible content with human-visible content.
5. Detect structural hiding, active content, unusual Unicode, confusables, and metadata separately from semantic instruction detection.
6. Feed only a newly constructed, least-privileged canonical representation to the AI.
7. Quarantine rather than pass through when parsing, rendering, or cross-view comparison is unreliable.

This design is important because sanitization can otherwise erase the very evidence needed for detection. A July 2026 controlled PDF study, *CrackedPDFs*, explicitly found that flattening documents before guardrails inspect their structure can remove evidence that text was hidden; its best document-aware hybrid detector substantially outperformed text-only PromptGuard on its controlled benchmark, while the authors cautioned that these results do not establish broad real-world robustness. citeturn22view0

**Unicode normalization should be used as an analysis transform, not as evidence destruction.** NFC preserves canonical equivalence and is a sensible normal representation; NFKC additionally collapses compatibility distinctions such as full-width versus ordinary characters and is therefore valuable for security matching. NFD and NFKD are useful diagnostic decompositions. The raw string must nevertheless be retained because the fact that normalization changes a string is itself evidence, and NFKC can intentionally eliminate distinctions present in the source. Unicode defines NFD as canonical decomposition, NFC as canonical decomposition plus composition, NFKD as compatibility decomposition, and NFKC as compatibility decomposition plus canonical composition. citeturn17view3turn17view4

**Invisible characters must be classified contextually rather than blindly stripped.** U+200B ZERO WIDTH SPACE and U+00AD SOFT HYPHEN can legitimately affect word or line breaking; ZWJ and ZWNJ have real orthographic and shaping functions; and bidirectional isolates are needed in mixed-direction text. Conversely, directional overrides such as U+202D LRO and U+202E RLO deserve a much stronger security signal: the Unicode Bidirectional Algorithm explicitly recommends avoiding overrides where possible because of security concerns. citeturn16search2turn16search14turn15view2

**Confusable detection should use Unicode TR39-style skeletons but never act alone.** Unicode specifies confusable skeletons and mixed-script detection, but also warns that its mappings can flag many legitimate strings. The best use in prose is therefore as an anomaly feature: normalize a token, compute its UTS #39 skeleton, inspect script mixing, and compare the skeleton against security-sensitive terms. citeturn17view0turn17view1turn17view2

**HTML needs three distinct views:** structural DOM content such as `textContent`, browser-rendered text such as `innerText`, and screenshot OCR. The HTML Standard defines `innerText` as text “as rendered” and omits non-rendered descendants in a rendered element, whereas DOM `textContent` represents descendant text independent of CSS visibility. The difference is therefore directly useful for discovering `display:none`, `visibility:hidden`, `content-visibility:hidden`, zero opacity, clipping, off-screen placement, or related hiding. Screenshot OCR adds coverage for CSS-generated content, canvas/SVG text and effects that DOM text APIs do not reliably model. citeturn15view4turn15view5turn18search0turn18search1turn18search29

**PDFs require object-level inspection before flattening.** Relevant structures include actions and JavaScript, optional-content groups/layers, annotations and their appearance streams, attachments, XObjects, fonts and `ToUnicode` mappings, XML/XMP metadata, and extracted text with zero opacity or other nonpainting behavior. PyMuPDF's text-tracing API exposes opacity, painting order and optional-content-group information and explicitly notes that it can detect text hidden by zero opacity, non-visible text types, or later overlapping objects. PDF's name dictionary can also contain document-level JavaScript and embedded-file trees. citeturn15view8turn19search9turn19search0

**OCR is best used as an independent witness, not as a replacement for parsing.** For every page in a high-assurance configuration, render pixels in a network-isolated process, OCR the pixels, and compare that result with extracted PDF text. OCRmyPDF's force-OCR mode illustrates the underlying principle: it rasterizes pages, discards hidden OCR text, rasterizes printable text and flattens interactive objects to their visual representation. citeturn19search2

**Classifiers are a second line of defense.** Meta Prompt Guard 2 provides purpose-built prompt-injection classifiers; LLM-based PromptArmor has reported very low error rates on AgentDojo; alignment-aware research such as AlignSentinel argues that the correct distinction is often *aligned instruction vs. misaligned instruction vs. non-instruction*, rather than simply “contains an instruction.” Yet recent evaluation work shows that ordinary train/test splitting can grossly overstate classifier generalization, and adaptive-attack studies continue to break defenses that depend on the language model protecting itself. citeturn15view11turn20search1turn20academia29turn22view1turn15view12

The recommended architecture is consequently:

```mermaid
flowchart TD
    A[Untrusted document or webpage] --> B[Quarantine original bytes]
    B --> C[Hash, provenance, MIME and structure sniffing]
    C --> D{Container type}

    D -->|Plain text| E[Unicode and control-character audit]
    D -->|HTML| F[Static DOM parse + isolated rendering]
    D -->|PDF| G[PDF object-graph inspection + isolated rendering]
    D -->|Image| H[Decode + metadata inventory + OCR]

    F --> I[DOM text / rendered text / screenshot OCR]
    G --> J[Extracted text / structural evidence / page OCR]
    H --> K[OCR text]
    E --> L[Multi-view Unicode analysis]
    I --> L
    J --> L
    K --> L

    L --> M[Hiddenness and discrepancy scoring]
    M --> N[Instruction-classifier ensemble]
    N --> O{Security policy}

    O -->|Low risk| P[Construct canonical safe payload]
    O -->|Ambiguous| Q[Quarantine or human review]
    O -->|Active / hidden malicious / parser failure| R[Reject or quarantine]

    P --> S[Attach source and provenance boundaries]
    S --> T[AI system]
```

The architectural principle is more important than any individual detector: **do not let raw untrusted document internals become model instructions merely because an extraction library returned them as text.** Microsoft Spotlighting similarly treats provenance as a continuous signal intended to help a model distinguish sources, while architectural work on prompt-injection resistance argues for separating trusted control flow from attacker-controlled data. citeturn15view10turn22view3

## Threat model and security objectives

The relevant adversary should be assumed to know that the document will be processed by an AI and, for a high-assurance system, to know the broad preprocessing strategy. Static defenses should therefore be evaluated against adaptive attackers rather than only against a fixed set of obvious phrases. Recent evaluation work found defenses that appeared strong against static attacks could fail under iterative adaptive attacks, and another 2026 study showed that standard cross-validation can exploit dataset provenance rather than actual malicious semantics. citeturn15view12turn22view1

| Attacker capability | Representative technique | Desired attacker outcome | Defensive requirement |
|---|---|---|---|
| Control document prose | Insert an apparently ordinary instruction into a report, email, RAG document, or webpage | Cause the AI to treat external data as higher-priority instructions | Provenance-aware instruction classification; model-side privilege separation. citeturn22view5turn15view10 |
| Control Unicode representation | Insert zero-width characters, soft hyphens, bidi controls, compatibility characters, or homoglyphs | Evade lexical detectors or human review | Raw-code-point audit plus NFC/NFKC security views and UTS #39 confusable analysis. citeturn15view1turn15view2turn17view0 |
| Control HTML and CSS | Put text in non-rendered DOM nodes, transparent/off-screen elements, clipped regions, nested frames, generated content or data URIs | Create machine-visible/human-invisible instructions | DOM/render/OCR comparison and static sandbox rendering. citeturn15view4turn18search29turn22view8 |
| Control PDF internals | Use invisible PDF text, unusual fonts, layers, annotations, XObjects, attachments, JavaScript or actions | Make extraction disagree with visible rendering or introduce active content | Object-graph inspection before rendering; page OCR differential; active-content rejection. citeturn15view8turn19search9turn19search0 |
| Control metadata | Put text in XMP, EXIF, document properties or custom package data | Reach pipelines that concatenate metadata into model context | Inventory metadata separately and exclude it from model input unless explicitly needed. citeturn15view14turn15view7turn21search1 |
| Control images | Put text in scans, screenshots, raster XObjects or images referenced from HTML | Reach OCR-capable AI while bypassing text scanners | OCR images independently and classify OCR-derived text with image provenance. citeturn19search2turn19search0 |
| Probe the detector repeatedly | Rephrase, translate, encode or structurally mutate injections after observing blocking behavior | Find a classifier or heuristic blind spot | Ensemble detection, held-out transformation tests, adaptive red teaming and fail-safe policy. citeturn15view12turn22view1 |
| Abuse parser complexity | Supply malformed, recursive, encrypted, huge or unusually structured inputs | Create parser disagreement, inspection bypass or resource exhaustion | Multiple parser checks where useful, isolation, explicit budgets and fail-closed parser errors. OWASP separately recommends not trusting extension or MIME claims alone. citeturn22view6 |

The defender should pursue six properties simultaneously.

**Evidence preservation.** Raw bytes, hashes and important offsets/object references should be retained before any normalization or transformation. OWASP's RAG guidance explicitly recommends hashing documents at ingestion and recording provenance, while warning against ingestion without scanning. citeturn22view6

**Human/machine view consistency.** Content that an extractor claims exists but that cannot be accounted for in the rendered view should become suspicious evidence, not silently accepted content. *CrackedPDFs* is particularly relevant here because it frames hidden PDF injection as a gap between human-visible rendering and machine-visible extraction. citeturn22view0

**No active execution.** Document preprocessing should not execute attacker-supplied JavaScript, macros, PDF actions or arbitrary network requests merely to discover textual content. CSP can reduce browser execution privileges and resource loading, but W3C explicitly describes CSP as defense-in-depth rather than a replacement for input validation. citeturn22view7

**Canonical output rather than trusted input mutation.** The output sent to the AI should be a newly constructed representation—ideally structured plain text plus separately typed image/OCR regions—not a notionally “cleaned” copy of an attacker-controlled object graph.

**Provenance preservation.** Every span supplied to the AI should remain attributable to the user, a particular external document, OCR, an annotation, or another source. Spotlighting's core idea is similarly to transform inputs so that provenance remains continuously apparent to the model. citeturn15view10

**Fail-safe ambiguity.** Encryption without authorized decryption, unsupported document features, malformed parsers, rendering failures, major OCR/extraction disagreements, or detector outages should produce quarantine rather than raw-input passthrough.

There is an important limit to set explicitly: **no document preprocessor can establish that all visible prose is benign.** An attacker can put a perfectly visible sentence telling the AI to ignore the user's objective. Structural detection solves hiddenness; semantic and architectural defenses address instruction/data confusion. Adaptive-defense results are strong evidence against treating any single downstream guard model as a proof of safety. citeturn15view12turn22view5

## Unicode, invisible characters, controls, and homoglyphs

Unicode security should operate on **parallel representations** rather than selecting one normalization and discarding the rest.

| Representation | Unicode definition | Defensive role | Recommended handling |
|---|---|---|---|
| Original | Exact decoded source code points | Forensic truth; preserves obfuscation evidence | Store and hash; never overwrite |
| NFD | Canonical decomposition | Exposes combining sequences and equivalent representations | Diagnostic view |
| NFC | Canonical decomposition followed by canonical composition | Normal interoperability representation while retaining compatibility distinctions | Default canonical textual representation |
| NFKD | Compatibility decomposition | Aggressively exposes compatibility-equivalent sequences | Diagnostic/security matching only |
| NFKC | Compatibility decomposition followed by canonical composition | Collapses distinctions such as width and many compatibility forms | Primary security-search/classifier view, but not forensic replacement |

These definitions and the distinction between canonical and compatibility equivalence come directly from Unicode UAX #15. Unicode also notes that NFC is commonly recommended for interoperable web content, while NFKC folds additional compatibility-equivalent forms. citeturn17view3turn17view4

A useful detection feature is therefore:

\[
\Delta_\text{NFKC}(s) = [\,s \neq NFKC(s)\,]
\]

plus the number and location of changed spans. A change is **not malicious by itself**; it merely tells the classifier that what an ordinary security comparison sees differs from the exact source.

**Characters of particular interest** include the following.

| Character/family | Behavior | Recommended policy |
|---|---|---|
| U+200B ZERO WIDTH SPACE | No intrinsic width; legitimately indicates word/line-break opportunities, including in languages without visible word spacing. citeturn16search2 | Record all occurrences. Remove or map to a separator only in a security-matching view; do not blindly alter multilingual source text. |
| U+200C ZWNJ / U+200D ZWJ | Affect joining and shaping and have legitimate orthographic uses. citeturn16search22turn15view1 | Flag contextually. Never use a blanket “delete all zero-width characters” policy. |
| U+00AD SOFT HYPHEN | Normally has a null appearance inside a line but can indicate an intraword line-break opportunity. citeturn16search14 | Strip in a security-token view; retain in raw evidence. High density is an obfuscation feature. |
| U+202A/U+202B embeddings and U+202C PDF | Alter bidirectional embedding. citeturn15view2 | Record and validate nesting; elevated risk in predominantly LTR material. |
| U+202D LRO / U+202E RLO | Force directional interpretation. Unicode specifically warns that overrides should be avoided where possible for security reasons. citeturn15view2 | High-severity finding outside narrowly justified contexts. |
| U+2066–U+2069 bidi isolates | Legitimately isolate directionality in mixed RTL/LTR text. citeturn15view2 | Inspect, but apply lower risk than overrides; locale-aware policy. |
| Default-ignorable characters | Some are expected to be invisible/non-advancing when unsupported. citeturn16search6 | Inventory all; treat density and unusual context as signals rather than automatically malicious. |
| C0/C1 controls | Generally inappropriate in ordinary prose except specific whitespace/transport uses | For AI-bound plain text, normally permit LF/TAB and normalized line endings; visibly escape or drop other controls in the sanitized copy while retaining the original evidence. |
| NUL and ESC | Especially anomalous in ordinary document text | Strong finding; reject NUL-containing “plain text” where downstream components may terminate strings or disagree about parsing. |

The last two rows are recommended engineering policy, rather than Unicode requirements.

A defensive implementation can preserve raw positions while generating security views:

```python
from __future__ import annotations

import unicodedata as ud
from dataclasses import dataclass


HIGH_RISK_FORMATS = {
    "\u202D": "LEFT-TO-RIGHT OVERRIDE",
    "\u202E": "RIGHT-TO-LEFT OVERRIDE",
    "\u202A": "LEFT-TO-RIGHT EMBEDDING",
    "\u202B": "RIGHT-TO-LEFT EMBEDDING",
    "\u202C": "POP DIRECTIONAL FORMATTING",
}

WATCH_FORMATS = {
    "\u00AD": "SOFT HYPHEN",
    "\u200B": "ZERO WIDTH SPACE",
    "\u200C": "ZERO WIDTH NON-JOINER",
    "\u200D": "ZERO WIDTH JOINER",
    "\u2066": "LEFT-TO-RIGHT ISOLATE",
    "\u2067": "RIGHT-TO-LEFT ISOLATE",
    "\u2068": "FIRST STRONG ISOLATE",
    "\u2069": "POP DIRECTIONAL ISOLATE",
    "\uFEFF": "ZERO WIDTH NO-BREAK SPACE / BOM",
}


@dataclass(frozen=True)
class Finding:
    offset: int
    codepoint: str
    name: str
    category: str
    severity: str


def inspect_unicode(text: str) -> tuple[dict[str, str], list[Finding]]:
    views = {
        form: ud.normalize(form, text)
        for form in ("NFC", "NFD", "NFKC", "NFKD")
    }

    findings: list[Finding] = []

    for i, ch in enumerate(text):
        cp = f"U+{ord(ch):04X}"
        category = ud.category(ch)
        name = ud.name(ch, "<unnamed>")

        if ch in HIGH_RISK_FORMATS:
            findings.append(
                Finding(i, cp, HIGH_RISK_FORMATS[ch], category, "high")
            )
        elif ch in WATCH_FORMATS:
            findings.append(
                Finding(i, cp, WATCH_FORMATS[ch], category, "medium")
            )
        elif category == "Cc":
            # Do not flag ordinary newline/tab here; policy handles them separately.
            if ch not in ("\n", "\r", "\t"):
                findings.append(Finding(i, cp, name, category, "high"))
        elif category == "Cf":
            # Broad catch-all. Cf has legitimate members, so this is only a signal.
            findings.append(Finding(i, cp, name, category, "low"))

    return views, findings
```

The crucial property of this code is not the particular severity levels; it is that **detection happens before removal and positions remain attributable to the raw source**.

**Homoglyph and confusable handling.** Unicode TR39 supplies a standardized notion of a confusable skeleton: two strings can be considered confusable when their security skeletons match. It also defines mixed-script and whole-script confusables. citeturn17view0turn17view1 A practical pipeline should compute skeletons token-by-token and test them against a small high-risk vocabulary—for example terms associated with role changes, tool use, secrets, system messages or overriding instructions—rather than attempting to classify all prose as spoofed.

Conceptually:

```text
for each token t:
    raw        = t
    compat     = NFKC(t)
    skeleton   = UTS39_CONFUSABLE_SKELETON(compat)
    scripts    = RESOLVED_SCRIPT_SET(t)

    features += {
        raw != compat,
        raw != skeleton,
        mixed_script(scripts),
        skeleton in sensitive_instruction_lexicon,
        edit_distance(raw, skeleton)
    }
```

[ICU's SpoofChecker](https://unicode-org.github.io/icu/userguide/security/) is the preferred concrete implementation family for Unicode security checks rather than maintaining an ad hoc confusables table. UTS #39 itself warns that confusable mappings can be overly inclusive and that mixed-script checks may flag legitimate strings, particularly across Latin and Cyrillic, so a hit should raise risk rather than automatically reject a document. citeturn17view2turn11search0

A robust control-removal strategy consequently has **three outputs**:

| Output | Transformation | Purpose |
|---|---|---|
| Evidence text | None | Audit, replay and incident investigation |
| Security-analysis text | NFKC; soft-hyphen/ZWSP-aware tokenization; confusable skeletons; escaped controls | Pattern matching and classifiers |
| Canonical AI text | Usually NFC; normalized line endings; unsafe C0/C1 controls removed or visibly escaped; suspicious formatting removed only after detection | Downstream model input |

This avoids two common failures: blindly stripping ZWJ/ZWNJ and breaking legitimate languages, or normalizing immediately and losing proof that an attacker used an obfuscated representation. Unicode itself stresses both legitimate joining-control contexts and the security need for special handling. citeturn15view1turn16search22

## Web and HTML defenses

HTML is dangerous precisely because **DOM existence is not equivalent to visual existence**. The DOM Standard's `textContent` walks descendant textual content; the HTML Standard's `innerText`, by contrast, represents text as rendered and excludes non-rendered descendants when queried on a rendered element. This gives a useful first-order differential detector. citeturn15view4turn15view5

CSS offers many ways to create disagreement. `display:none` suppresses box generation, `visibility:hidden` prevents a generated box from being rendered, `content-visibility:hidden` skips content, and zero opacity can also make an element visually absent; browser visibility APIs explicitly account for content visibility, opacity and CSS visibility. citeturn18search0turn18search6turn18search1turn18search29 Clipping, transforms, zero/tiny dimensions, overflow and matching foreground/background colors add further hiding possibilities. CSS should therefore be considered **evidence during visibility analysis**, even though it should normally be removed from the final AI-bound HTML representation.

A good web preprocessor obtains at least these views:

| View | Captures | Main blind spot |
|---|---|---|
| Raw DOM/source | Hidden nodes, attributes, script/style source, `srcdoc`, data URLs | Does not identify what was visible |
| `textContent` | Descendant DOM text irrespective of normal rendering | CSS-generated pixels, canvas; includes hidden DOM text |
| `innerText` | Browser's rendered textual view | Can miss text painted through nonstandard mechanisms and may depend on rendering state |
| Computed-style visibility inventory | `display`, `visibility`, opacity, rectangles, transforms, clipping etc. | Occlusion and pixel-level effects can be complex |
| Screenshot OCR | What was actually painted as text-like pixels | OCR errors; accessibility-only/alt content; low-resolution or stylized text |

The final decision should use the **intersection and discrepancies**, not declare one view authoritative.

A defensible Playwright-style audit looks like this:

```javascript
import { chromium } from "playwright";

export async function inspectHtml(html) {
  const browser = await chromium.launch({ headless: true });

  const context = await browser.newContext({
    javaScriptEnabled: false,
    // Do not attach cookies, credentials, storage state, etc.
  });

  const page = await context.newPage();

  // Network isolation is independent of CSP and is intentionally redundant.
  await page.route("**/*", route => route.abort());

  await page.setContent(html, { waitUntil: "domcontentloaded" });

  const analysis = await page.evaluate(() => {
    const body = document.body;
    const rawDomText = body?.textContent ?? "";
    const renderedText = body?.innerText ?? "";

    const suspiciousVisibility = [];

    for (const el of body?.querySelectorAll("*") ?? []) {
      const cs = getComputedStyle(el);
      const r = el.getBoundingClientRect();

      const hidden =
        cs.display === "none" ||
        cs.visibility === "hidden" ||
        cs.visibility === "collapse" ||
        Number.parseFloat(cs.opacity || "1") <= 0.01 ||
        r.width === 0 ||
        r.height === 0;

      if (hidden && (el.textContent ?? "").trim()) {
        suspiciousVisibility.push({
          tag: el.tagName,
          id: el.id,
          className: el.className,
          text: el.textContent.slice(0, 500),
          display: cs.display,
          visibility: cs.visibility,
          opacity: cs.opacity,
          rect: {
            x: r.x, y: r.y, width: r.width, height: r.height
          }
        });
      }
    }

    return { rawDomText, renderedText, suspiciousVisibility };
  });

  const screenshot = await page.screenshot({
    fullPage: true,
    type: "png"
  });

  await browser.close();

  return { ...analysis, screenshot };
}
```

Playwright provides both browser evaluation and screenshot functionality suitable for constructing this independent rendered view. citeturn22view9 In production, a second pass should additionally test clipping, viewport intersection, transforms, `font-size`, text/background contrast, `text-indent`, overflow, pseudo-elements, SVG, canvas, shadow DOM and overlapping/occluding elements.

**Script and style handling requires sequencing.** Do not first delete all styles and then ask whether text was visible: that can turn an attacker-hidden node into apparently ordinary visible text and erase the hiding evidence. Instead:

```text
Raw HTML
  → non-executing structural parse
  → render with CSS under network isolation and JavaScript disabled
  → record visible/non-visible differences
  → screenshot + OCR
  → classify discrepancies
  → only then construct a CSS-free sanitized payload
```

A DOM sanitizer such as [DOMPurify](https://github.com/cure53/DOMPurify) is appropriate for producing a restricted HTML representation: DOMPurify is explicitly designed as a DOM-based sanitizer with a secure default. citeturn15view3 It should nevertheless be viewed as an **XSS/content sanitizer, not a prompt-injection detector**. The safest AI payload is usually structured plain text; where HTML must be retained, use a narrow allowlist and remove scripts, styles, event-handler attributes, forms and active embedded content.

**CSS hiding heuristics** should include at minimum `display:none`, `visibility:hidden/collapse`, `content-visibility:hidden`, opacity near zero, zero-size boxes, clipping to effectively empty regions, off-viewport absolute/fixed positioning, transforms that move content outside the viewport, tiny font sizes, text/background colors with effectively no contrast, overflow clipping, and content obscured by later opaque elements. W3C specifications establish the underlying rendering behavior for display, visibility, content visibility and opacity. citeturn18search0turn18search1turn18search4turn18search6

Treat CSS `::before`/`::after` generated text separately. DOM descendant-text extraction may not reproduce generated painted content, while screenshot OCR can. Conversely, `alt`, ARIA labeling and other accessibility-only information may be machine-visible yet absent from screenshot OCR. The pipeline should therefore attach an explicit provenance class such as `VISIBLE_TEXT`, `ACCESSIBILITY_TEXT`, `HIDDEN_DOM`, or `OCR_ONLY` rather than indiscriminately concatenating all of them.

**Data URIs** are real embedded objects, not harmless strings. RFC 2397 permits inline data of the form `data:[<mediatype>][;base64],<data>`. citeturn22view8 The defensive policy should parse the media type, apply decoded-size limits, decode into an inert buffer and recursively inspect the resulting content. Raster images can be separately decoded and OCRed; HTML, SVG/XML, script-like or unsupported data URIs should be rejected or quarantined rather than executed by the browser.

**Iframes and embedded objects** should become nested provenance boundaries. `srcdoc` is recursively scanned as another HTML input. A remote `src` should not be fetched automatically with the user's credentials; if retrieval is necessary, a separate stateless fetcher can obtain it and begin a new preprocessing transaction. `<object>` and `<embed>` should likewise not cause automatic execution.

**CSP belongs in the sandbox but is not the sandbox.** W3C describes CSP as a mechanism for controlling resource requests, inline script, dynamic code and related execution privileges, while explicitly warning that it is defense-in-depth rather than a first-line input validator. citeturn22view7 A preprocessing renderer can enforce an aggressive policy such as conceptually:

```http
Content-Security-Policy:
  default-src 'none';
  script-src 'none';
  object-src 'none';
  frame-src 'none';
  connect-src 'none';
```

CSS required for visibility analysis can be applied in a separate analysis environment while network I/O is independently blocked. The final sanitized payload should not depend on CSP to be safe.

The web-processing trade-offs are:

| Technique | Hidden-content accuracy | Cost | False-positive risk | Principal limitation |
|---|---:|---:|---:|---|
| Static tag stripping | Low | Very low | Low | Destroys evidence and misses CSS/visual tricks |
| DOM `textContent` vs `innerText` | Good for ordinary CSS hiding | Low | Low–medium | Not pixel-complete |
| Computed-style scanner | Good | Medium | Medium | Legitimate responsive/accessibility hiding |
| Screenshot + OCR | High for painted text | High | Medium | OCR errors; not accessibility metadata |
| Network-isolated real-browser render | Highest fidelity | High | Low for rendering semantics | Browser itself becomes hostile-input attack surface |
| Semantic injection classifier only | Poor for structural hiddenness | Medium | Model-dependent | Cannot know why text was human-invisible |

The recommended default is **real-browser differential analysis plus OCR**, with JavaScript and networking disabled. Because the user specified no compute constraint, there is little reason to skip the independent screenshot/OCR witness for high-value ingestion.

## PDF, metadata, OCR, and visible-versus-extracted comparison

PDF deserves stronger treatment than “extract all strings and scan them.” Its logical object graph and rendered appearance can differ substantially. The PDF specification includes name-tree entries for document-level JavaScript and embedded files; files can also appear through file-attachment annotations, and XObjects can contain reusable graphical or image content. citeturn19search9turn19search0

A PDF inspection pass should inventory these structures before any rasterization:

| PDF structure | What to inspect | Default defensive treatment |
|---|---|---|
| Catalog/actions | `/OpenAction`, `/AA`, JavaScript/ECMAScript actions, launch/remote/form actions | Do not execute; high-severity or reject from AI-ingestion path |
| `/Names` | `/JavaScript`, `/EmbeddedFiles`, alternate/rendition entries | Inventory recursively; embedded material never automatically joins model context. citeturn19search9 |
| Annotations | `/Annots`, `/Contents`, appearance `/AP`, file attachments, widgets, rich media | Parse text separately; compare annotation-on and annotation-off renderings where relevant |
| Optional content | `/OCProperties`, OCG/OCMD layer membership | Identify text that exists only in non-default/hidden layers |
| XObjects | Image and Form XObjects, recursively referenced resources | OCR images; recursively inspect Form XObjects |
| Fonts | Embedded fonts, encodings, `ToUnicode` maps, Type 3 or unusual mappings | Do not reject merely for embedding; compare extracted Unicode with rendered glyph OCR |
| Content streams | Text painting, transformations, clipping, opacity, overlap and geometry | Determine whether extracted spans are actually painted and visible |
| Attachments | File specifications and embedded-file streams | Treat as separate child documents with their own provenance |
| Metadata | XMP `/Metadata` and older document information | Inventory then omit from model context |
| Encryption | Any content that cannot be completely inspected | Quarantine unless authorized decryption is performed in the isolated preprocessing environment |

PDF fonts merit special emphasis. **An embedded font is not itself suspicious**; embedding is normal. The relevant anomaly is a disagreement among encoded character values, a font's character mapping, extracted Unicode and the pixels actually shown to the user. Rendering-to-image and OCR are a powerful independent check precisely because OCR sees the painted glyph shapes rather than trusting the PDF's claimed textual mapping.

[PyMuPDF](https://pymupdf.readthedocs.io/) is especially useful for structural visibility analysis. Its text tracing exposes an opacity value where zero denotes invisible text, painting sequence information that can indicate that later objects cover a text span, and the Optional Content Group associated with text; its documentation explicitly calls out these mechanisms as ways to detect additional forms of text invisibility. citeturn15view8

Representative inspection code:

```python
from __future__ import annotations

import pymupdf


def inspect_pdf_text_visibility(path: str) -> list[dict]:
    doc = pymupdf.open(path)
    findings: list[dict] = []

    for page_no, page in enumerate(doc):
        page_rect = page.rect

        for span in page.get_texttrace():
            opacity = float(span.get("opacity", 1.0))
            text_type = int(span.get("type", 0))
            ocg = span.get("layer") or span.get("ocg")

            chars = span.get("chars", ())
            text = "".join(
                chr(ch[0]) if isinstance(ch[0], int) else str(ch[0])
                for ch in chars
            )

            suspicious = (
                opacity <= 0.01 or
                text_type > 1
            )

            if suspicious and text.strip():
                findings.append({
                    "page": page_no + 1,
                    "text": text,
                    "opacity": opacity,
                    "text_type": text_type,
                    "optional_content_group": ocg,
                })

    return findings
```

The exact fields should be pinned to the deployed PyMuPDF version and tested against a corpus of generated PDFs; the important design is to retain page/object/span identity rather than only the final extracted string.

[qpdf](https://qpdf.readthedocs.io/) and [pikepdf](https://pikepdf.readthedocs.io/) are complementary. qpdf can expose the underlying PDF in structured JSON, including otherwise difficult-to-see objects, while pikepdf provides a Python interface built around qpdf. qpdf's documentation notes that its JSON representation includes unreferenced objects and recommends removing object streams for easier inspection. citeturn15view6 Do not rely on a raw `grep` for `/JavaScript`: object streams, indirection and encoded names make recursive parsed-object inspection the correct approach.

Conceptually:

```text
parse PDF object graph
for every reachable and suspicious unreferenced object:
    record object id and type

    recursively examine dictionaries for:
        /JavaScript, /JS
        /OpenAction, /AA
        /Launch, /GoToR, /SubmitForm
        /EmbeddedFiles, /EF
        /RichMedia
        /OCProperties
        /Annots, /AP
        /XObject
        /Metadata

    do not execute actions
    do not recursively ingest attachments into the same trust context
```

**Layers and annotations.** A high-assurance system should consider rendering at least a default user-visible configuration and, for detection purposes only, an “expanded” inspection configuration that exposes otherwise optional content. Text appearing only in hidden/non-default OCGs becomes a strong hiddenness feature. Annotation textual contents and annotation appearances should be compared separately because one may exist without being visually prominent. PDF Association material documents the presence of optional content, attachments and rich-media/annotation structures in the format. citeturn19search0turn19search14

**OCR differential analysis** should make the rendered page an independent information channel:

\[
E_p = \text{normalized extractor text on page }p
\]

\[
O_p = \text{normalized OCR of pixels rendered for page }p
\]

Useful metrics include:

\[
D_\text{char}(E,O)=
\frac{\operatorname{editDistance}(E,O)}
{\max(|E|,|O|,1)}
\]

and token overlap:

\[
J(E,O)=\frac{|tokens(E)\cap tokens(O)|}
{|tokens(E)\cup tokens(O)|}.
\]

Global metrics are only a triage aid. More important is **span-level asymmetry**:

```text
extraction-only suspicious span:
    instruction classifier says span is suspicious
    AND extracted span has no acceptable OCR match
    AND PDF geometry/opacity/layer evidence indicates invisibility
    → very high risk

OCR-only suspicious span:
    OCR sees an instruction
    AND ordinary text extraction does not
    → inspect raster image/XObject or broken font mapping

large general disagreement:
    extractor and OCR disagree widely
    → parsing/font/OCR ambiguity; quarantine rather than choose one silently
```

The spatial version is stronger. Obtain word or character bounding boxes from the PDF extractor and word boxes from OCR, align them by page coordinates, and ask not merely “does this sentence occur?” but “is the same linguistic content present in approximately the same painted location?” This catches cases where an invisible extraction layer contains extra text elsewhere on the page.

For a compute-unconstrained high-assurance deployment, I recommend **OCRing every rendered PDF page at roughly 300 DPI** rather than OCRing only files that appear scanned. The 300 DPI figure is an engineering starting point, not a universal standard; very small typography may justify higher resolution.

A practical starting policy—not a published universal threshold—is:

| Condition | Default action |
|---|---|
| Instruction-like text exists in extraction but is absent from rendered-page OCR | Quarantine regardless of overall document similarity |
| Zero-opacity/nonpainting instruction-like text | Quarantine |
| Instruction-like text resides only in a non-default optional-content layer | Quarantine |
| Major extracted/OCR disagreement, e.g. normalized character discrepancy above roughly 10–15% | Review/quarantine; tune threshold on local corpus |
| OCR detects meaningful text absent from extraction | Classify the OCR span and inspect image/XObject provenance |
| OCR confidence is poor over large portions of page | Do not infer “clean”; render again or quarantine |
| Extraction and OCR closely agree, no structural anomalies | Continue to semantic injection classification |

These thresholds should be tuned against benign scans, multilingual documents, equations and low-quality historical material rather than treated as standardized security values.

[OCRmyPDF](https://ocrmypdf.readthedocs.io/) provides a useful canonicalization primitive. Its `redo` mode distinguishes visible and invisible text and removes invisible OCR layers before re-OCR, while its force mode rasterizes pages, discards hidden OCR text, rasterizes printable text and flattens interactive objects to their visual representation. citeturn19search2 For the **highest-assurance AI payload**, an attractive design is therefore:

```text
original PDF (retained only in quarantine/audit storage)
          |
    structure scan
          |
    render pages
          |
       page images
          |
          +----> OCR text + spatial boxes
          |
          +----> optional new image-only PDF / fresh OCR layer
```

This does not mean rasterization is perfect. It loses hyperlinks, semantic tagging, digital-signature meaning, some accessibility information and editability; renderer vulnerabilities remain a consideration. It does, however, collapse much of PDF's hidden logical state into the visual surface that a human would ordinarily review. The correct operating model is to **retain the original for forensic purposes while never letting its unchecked object graph become AI context**.

**Metadata should be excluded by default, not merely cleaned opportunistically.** Adobe describes XMP as an extensible mechanism for embedding metadata such as titles, descriptions, keywords and author information into files. PDFs can have both XMP and the older DocumentInformation dictionary. citeturn15view14turn15view7 Open XML documents likewise store core, custom and other document properties in package parts, and Microsoft's Open XML SDK exposes the ZIP/XML package structure programmatically. citeturn21search1turn21search4

A metadata policy should distinguish **inventory** from **exposure**:

```text
before sanitization:
    record metadata namespaces, field names, lengths and hashes
    optionally store sensitive values in restricted forensic storage

AI-bound artifact:
    omit XMP
    omit EXIF textual/comment/GPS fields
    omit PDF document properties
    omit OOXML core/custom/extended properties
    omit unsupported/custom XML unless explicitly part of the user's task
```

For images, [ExifTool](https://exiftool.org/) is valuable for metadata inspection, but its own documentation warns that deleting “all metadata” is not guaranteed to remove every kind of metadata from every file. citeturn21search0 Consequently, **decode → apply rendering-critical orientation/profile behavior → re-encode into a new minimal image** provides a stronger canonicalization boundary than repeatedly deleting tags in place.

For PDFs, pikepdf directly exposes XMP and the older document-information model. citeturn15view7 For DOCX/XLSX/PPTX, the [Microsoft Open XML SDK](https://learn.microsoft.com/en-us/office/open-xml/open-xml-sdk) is preferable to treating OOXML as an opaque file; Office Open XML is itself a ZIP/XML package, which makes explicit part-by-part policy possible. citeturn21search4

## Suspicious-instruction classifiers and evaluation

A good classifier should answer a narrower and more context-sensitive question than **“does this text contain an imperative sentence?”** Manuals, emails, contracts, code, security reports and troubleshooting instructions contain perfectly legitimate imperatives. AlignSentinel's 2026 formulation is useful precisely because it distinguishes **misaligned instructions, aligned instructions and non-instruction text**, addressing false positives caused by treating every instruction as malicious. citeturn20academia29turn22view2

The classifier should receive both text and document-forensic features.

**Textual/semantic features** include instruction-like verbs, role references (`system`, `assistant`, `user`, `developer`, `tool`), attempts to override earlier directives, requests to disclose secrets or hidden context, claims of authority, requests for tool execution, and semantic divergence from the user's actual task. OWASP similarly recommends scanning ingested material for injection markers, while recent classifier work emphasizes that semantic generalization rather than dataset-specific surface cues is the core challenge. citeturn22view6turn22view1

**Obfuscation features** should include the number and density of Unicode controls, NFKC change rate, confusable skeleton changes, mixed-script anomalies, zero-width/soft-hyphen density, base64/hex-like sections, unusual markup and suspicious instruction matches that appear only after canonicalization.

**Visibility features** should be first-class classifier inputs: raw DOM vs `innerText` difference, screenshot-OCR difference, count/length of CSS-hidden nodes, PDF zero-opacity spans, non-default OCG text, out-of-bounds text, extraction/OCR disagreement, annotation-only text, and metadata-only text. The 2026 *CrackedPDFs* study supports exactly this direction: its document-aware hybrid detector did substantially better than a text-only guardrail within its controlled PDF benchmark, while pure structural and shortcut-prone text models had important limitations. citeturn22view0

**Provenance features** include whether a span came directly from the user's trusted instruction, an uploaded attachment, a third-party webpage, an OCR image, a hidden DOM node, a PDF annotation, an attachment inside another attachment, or metadata. Provenance should not simply be a classifier feature—it should also directly constrain downstream privileges. Spotlighting's approach similarly introduces transformations that continuously identify the source of input. citeturn15view10

The major model families have materially different trade-offs:

| Model | Strength | Weakness | Recommended role |
|---|---|---|---|
| Regex/rule engine | Deterministic, extremely fast, interpretable, excellent for Unicode controls and explicit attack phrases | Easily paraphrased around; high FP if broad | First-stage high-precision indicators |
| UTS #39/structural feature model | Detects obfuscation independent of semantics | Legitimate multilingual/formatting behavior causes anomalies | Hiddenness/obfuscation score |
| Small fine-tuned encoder | Low latency; good lexical/semantic abstraction | Distribution shift and adaptive attacks | Main scalable text classifier |
| Meta Prompt Guard 2 | Purpose-built prompt-injection/jailbreak classifier; available in 86M and 22M variants. citeturn15view11 | Still a text classifier and therefore cannot infer document visibility by itself | Strong ensemble member |
| Embedding + classical classifier | Simple and inspectable; can capture semantic clusters | Embedding/domain shift; often learns dataset identity | Secondary ensemble member |
| Guard LLM / LLM-as-judge | Understands task context and nuanced alignment | Higher cost; guard itself may be prompt-injectable | Ambiguous-case adjudication only |
| Activation/attention probe | Can expose internal model signals | Model-specific and often requires white-box inference | Self-hosted research/high-assurance deployment |
| Alignment-aware multiclass detector | Directly models whether an instruction matches the intended task | Requires reliable task representation and contextual training | Preferred semantic formulation |

PromptArmor is a useful example of the guard-LLM approach: the authors report false-positive and false-negative rates below 1% on AgentDojo for several guard models and an attack-success rate below 1% after removal. Those figures should be read as **benchmark-specific**, not as production guarantees. citeturn20search1turn20search13 CAPTURE, in contrast, reports that guardrails can simultaneously exhibit false negatives on adversarial prompts and excessive false positives on benign context-aware cases. citeturn20academia31turn22view4

Recent work gives an especially strong warning about benchmark methodology. A July 2026 study assembled 18 datasets totaling roughly 105,000 samples and found standard cross-validation yielded 0.996 AUC while leave-one-dataset-out evaluation fell to 0.912—a difference of 8.4 percentage points. The authors attribute this to classifiers exploiting dataset-identity cues rather than attack semantics. citeturn22view1 Thus, a detector intended to protect real documents should **never be certified from a random train/test split of pooled attack and benign datasets alone**.

A robust training corpus should instead contain:

| Training category | What to include | Why |
|---|---|---|
| True injections | Direct and indirect injections from web, email, RAG, document and agent benchmarks | Base attack semantics |
| Structural transformations | Same injection hidden with Unicode, CSS, PDF invisibility, layers, metadata, images and OCR | Prevent surface-form dependence |
| Aligned instructions | Manuals, task-relevant forms, recipes, procedures, quoted user instructions | Teach that “instruction” ≠ “attack” |
| Hard benign negatives | Security articles describing attacks, source code containing phrases like “ignore previous instructions,” legal text, email quotations | Control false positives |
| Multilingual/RTL text | Arabic, Hebrew, Persian, Indic shaping, CJK and mixed-language material | Prevent Unicode security checks from becoming language discrimination |
| Paired counterfactuals | Same base document with benign hidden formatting, visible instruction, and hidden malicious instruction | Force learning of attack-specific evidence rather than document-generator artifacts |
| Adaptive attacks | Attacks generated after observing current detector decisions | Test attacker-moving-second behavior |

The paired-counterfactual recommendation is strongly supported by *CrackedPDFs*, which deliberately used benign matched confounders and base-document-grouped splits to prevent trivial leakage. citeturn22view0

Evaluation should report **precision, recall, PR-AUC, false-positive rate at realistic prevalence, false-negative rate, attack success after filtering, benign-task utility and retained-content quality**. The base rate matters greatly. For example, suppose only 0.1% of incoming documents are truly malicious. Even a detector with 99% recall and a 1% benign false-positive rate would, per 10,000 documents, produce roughly 10 true-positive detections but about 100 false positives. Only about 9% of alerts would actually be attacks. Therefore production tuning should target very low FPRs and use contextual escalation rather than assuming a seemingly impressive “99% accuracy” is operationally sufficient.

The cleanest false-positive mitigation is to maintain **two conceptually separate scores**:

\[
H = P(\text{content is hidden/obfuscated or view-inconsistent})
\]

\[
I = P(\text{content is a misaligned instruction}\mid
\text{task, source, provenance})
\]

Then policy can reason about combinations:

| Hiddenness \(H\) | Instruction risk \(I\) | Interpretation | Action |
|---|---|---|---|
| Low | Low | Ordinary content | Allow canonical representation |
| High | Low | Possibly legitimate hidden/accessibility/layout information | Sanitize from AI payload; log/review if unusual |
| Low | High | Visible prompt injection | Quarantine or retain only as explicitly labeled untrusted data |
| High | High | Strong hidden-instruction signal | Block/quarantine |
| Unknown | Any | Parser/render/OCR failure | Fail closed |

This is substantially safer than a single binary “malicious” probability.

Finally, preprocessing should not be the only security boundary. A 2026 adaptive evaluation concluded that every defense in its experiment that relied on the model itself to protect a secret eventually failed. citeturn15view12 The downstream application should therefore also minimize the model's authority: untrusted content should not be allowed to decide tool calls, recipients, payment destinations, secret access or irreversible actions merely because it survived document scanning.

## Secure preprocessing architecture, defaults, and implementation recommendations

A production implementation is best divided into components so that the parser, renderer, classifier and AI runtime do not share unnecessary privileges:

```mermaid
flowchart LR
    SRC[Untrusted Sources<br/>uploads / web / email / RAG] --> GW[Ingress Gateway]

    GW --> RAW[(Immutable Raw Store)]
    GW --> TYPE[Type / Container Inspector]

    TYPE --> UP[Unicode Processor]
    TYPE --> HP[HTML Static Parser]
    TYPE --> PP[PDF Object Inspector]
    TYPE --> IP[Image Decoder]

    HP --> RENDER[Isolated Renderer]
    PP --> RENDER
    IP --> OCR[OCR Engine]
    RENDER --> OCR

    UP --> FEAT[Security Feature Store]
    HP --> FEAT
    PP --> FEAT
    OCR --> FEAT

    FEAT --> CLASS[Instruction / Alignment Classifiers]
    CLASS --> POLICY[Policy Engine]

    POLICY -->|allow| CANON[Canonical Payload Builder]
    POLICY -->|review| REVIEW[Human Review Queue]
    POLICY -->|reject| QUAR[Quarantine]

    CANON --> LLM[AI Runtime]
    LLM --> TOOLPOL[Independent Tool Authorization]

    GW --> AUDIT[(Tamper-evident Audit Log)]
    TYPE --> AUDIT
    RENDER --> AUDIT
    CLASS --> AUDIT
    POLICY --> AUDIT
```

The relationship that matters is that the **raw object never travels directly from ingress to the LLM**. The AI receives a separate canonical entity whose lineage points back to the original.

A recommended processing sequence is:

```text
UNTRUSTED BYTES
    │
    ├─ compute cryptographic hash
    ├─ record uploader/source/time/provenance
    ├─ sniff actual format; do not trust extension
    └─ store immutable original
             │
             ▼
    CONTAINER-SPECIFIC STRUCTURAL AUDIT
             │
             ├─ HTML: DOM, links, CSS, frames, active content
             ├─ PDF: object graph, actions, layers, annotations, fonts, XObjects
             └─ image/OOXML/etc.: metadata and embedded-object inventory
             │
             ▼
    MULTIPLE INDEPENDENT CONTENT VIEWS
             │
             ├─ raw extracted text
             ├─ NFC/NFKC/confusable security forms
             ├─ human-rendered text
             └─ render-to-image OCR
             │
             ▼
    DIFFERENTIAL / HIDDENNESS ANALYSIS
             │
             ▼
    METADATA AND ACTIVE-CONTENT EXCLUSION
             │
             ▼
    MISALIGNED-INSTRUCTION CLASSIFIER ENSEMBLE
             │
             ▼
          POLICY
      ┌──────┼─────────┐
      │      │         │
    allow  review   quarantine
      │
      ▼
    NEW CANONICAL PAYLOAD
      │
      ├─ visible content only
      ├─ source/span provenance
      ├─ OCR explicitly labeled
      └─ no executable behavior
      │
      ▼
    AI + independent authorization boundary
```

The following defaults are appropriate for a high-assurance deployment with no material compute constraint:

| Layer | Recommended default |
|---|---|
| Ingress | SHA-256 or stronger content hash; immutable original; source/uploader provenance; inspect magic/container structure rather than trusting filename or MIME claim. OWASP recommends document hashing, provenance and independent scanning. citeturn22view6 |
| Unsupported/encrypted input | Quarantine unless the content can be fully inspected under an explicitly authorized decryption workflow |
| Unicode | Retain raw; NFC canonical copy; NFKC security copy; generate NFD/NFKD only as needed for diagnostics |
| C0/C1 | In canonical plain-text output retain only explicitly approved whitespace controls such as LF/TAB; escape/drop the rest after logging; NUL is high risk |
| ZWSP/soft hyphen | Record every occurrence; strip/map in the classifier view; do not destroy raw evidence |
| ZWJ/ZWNJ | Record; retain where linguistically valid; no blanket stripping |
| Bidi overrides | High severity by default; Unicode itself warns about override security risk. citeturn15view2 |
| Confusables | UTS #39/ICU skeleton at token level; mixed-script signal; require context before blocking because UTS #39 warns of legitimate confusable hits. citeturn17view2 |
| HTML JavaScript | Never execute in the default ingestion renderer |
| HTML network | No outbound networking from the rendering process; no user cookies, credentials or browser storage |
| HTML CSS | Apply in isolated visibility-analysis renderer, then remove from final AI-bound representation |
| HTML comparisons | Always collect `textContent`, `innerText`, hidden-style inventory and screenshot OCR for high-assurance ingestion |
| HTML sanitizer | Strict allowlist such as DOMPurify after visibility analysis; preferably convert to structured plain text. citeturn15view3 |
| `data:` resources | Decode under bounded recursion/size policy; raster images independently OCRed; HTML/SVG/active/unsupported payloads quarantined. RFC 2397 defines the inline-data mechanism. citeturn22view8 |
| Iframes | `srcdoc` recursively scanned; external frame URLs are new fetch/provenance transactions; never merge blindly |
| CSP | `default-src 'none'`-style policy plus independent network blocking; CSP is only defense-in-depth. citeturn22view7 |
| PDF actions/JavaScript | Never execute; reject or strip from newly created artifact after recording evidence |
| PDF embedded files | Never implicitly include; recursively process as independent child documents |
| PDF layers | Inventory OCG membership; text present only in hidden/non-default layers is a strong anomaly |
| PDF invisible text | Zero opacity, nonpainting text and covered spans are high-risk, especially when instruction-like. PyMuPDF exposes these signals. citeturn15view8 |
| PDF pages | Render every page and OCR every page in high-assurance mode |
| PDF canonicalization | For maximal isolation, reconstruct from rendered pages and optionally add a fresh OCR layer; keep original separately. OCRmyPDF force mode embodies this rasterization principle. citeturn19search2 |
| Metadata | Inventory first; do not send EXIF/XMP/document properties/custom properties to the AI unless required by the task |
| Images | Apply required orientation/render interpretation, decode pixels, OCR, then optionally re-encode into a new minimal container |
| Classifier | Rules + structural hiddenness + small semantic classifier + contextual alignment classifier; guard LLM only as escalation |
| Classifier training | Domain- and attack-family-held-out splits, benign aligned instructions, paired counterfactuals and adaptive attacks; random pooled split alone is insufficient. citeturn22view1turn22view0 |
| Decision policy | Three-way allow / quarantine-review / reject rather than binary filter |
| Error behavior | Parser, renderer, OCR, sanitizer or classifier failure must not cause raw-input passthrough |
| AI payload | Canonical content plus explicit provenance/source labels; no active document object graph |
| Tool use | Separate authorization logic decides external side effects; document text must never grant itself authority |

An implementation can formalize the policy without tying it to one classifier score:

```python
from dataclasses import dataclass


@dataclass
class Risk:
    parser_ok: bool
    active_content: bool
    encrypted_uninspected: bool

    unicode_high_risk: bool
    hidden_instruction_span: bool
    extraction_ocr_instruction_mismatch: bool
    severe_view_disagreement: bool

    hiddenness_score: float      # calibrated 0..1
    misalignment_score: float    # calibrated 0..1


def decide(r: Risk) -> str:
    # Fail-safe conditions.
    if not r.parser_ok or r.encrypted_uninspected:
        return "QUARANTINE"

    # Executable document behavior is unnecessary for AI text ingestion.
    if r.active_content:
        return "QUARANTINE"

    # Cross-view evidence of an actual instruction is stronger than a
    # document-wide similarity metric.
    if r.hidden_instruction_span:
        return "QUARANTINE"

    if r.extraction_ocr_instruction_mismatch:
        return "QUARANTINE"

    # Ambiguity must not degrade into raw passthrough.
    if r.severe_view_disagreement:
        return "REVIEW"

    # Example starting bands only: calibrate on deployment data.
    combined = max(
        r.misalignment_score,
        0.6 * r.misalignment_score + 0.4 * r.hiddenness_score,
    )

    if combined >= 0.85:
        return "QUARANTINE"
    if combined >= 0.40 or r.unicode_high_risk:
        return "REVIEW"

    return "ALLOW"
```

The numerical bands are intentionally illustrative. Production thresholds should be selected from the deployment's observed benign prevalence and acceptable false-positive rate, then continuously tested against held-out sources and adaptive attacks. Recent distribution-shift research makes clear why adopting a benchmark's threshold unchanged is unsafe. citeturn22view1

**Logging should be forensic but privacy-minimizing.** Each transaction should record at least the original and canonical hashes, declared and detected media type, parser/renderer/OCR/library versions, structural anomaly types, Unicode code point and offset findings, DOM selectors or PDF page/object/bounding-box identifiers, OCR/extraction discrepancy metrics, classifier versions and scores, final policy decision, and the canonical-output hash. Sensitive full text need not be duplicated in ordinary logs; retain content in a separately access-controlled evidence store.

**Versioned reproducibility is important.** When a later incident is investigated, the security team should be able to answer: which parser produced this extraction, which Unicode data version supplied the confusable mapping, which renderer generated the screenshot, which classifier model and threshold were applied, and exactly which canonical bytes reached the AI. This is particularly important because Unicode security data evolve and learned detectors are vulnerable to distribution shift. citeturn17view2turn22view1

The principal tools and primary references are:

| Function | Recommended implementation/reference | Rationale |
|---|---|---|
| Unicode normalization | Python `unicodedata`; [Unicode UAX #15](https://unicode.org/reports/tr15/) | Normative definitions of NFC/NFD/NFKC/NFKD. citeturn15view0turn17view3 |
| Confusables/security | [Unicode UTS #39](https://www.unicode.org/reports/tr39/); [ICU security/SpoofChecker](https://unicode-org.github.io/icu/userguide/security/) | Standard confusable skeleton and mixed-script mechanisms. citeturn15view1turn17view0turn11search0 |
| Bidirectional controls | [Unicode UAX #9](https://www.unicode.org/reports/tr9/) | Defines embeddings, overrides and isolates and documents override security concerns. citeturn15view2 |
| HTML sanitization | [DOMPurify](https://github.com/cure53/DOMPurify) | Mature DOM-oriented allowlist sanitizer. citeturn15view3 |
| Browser rendering | [Playwright](https://playwright.dev/) + isolated Chromium | DOM inspection plus reproducible screenshot rendering. citeturn22view9 |
| Browser security policy | [W3C CSP Level 3](https://www.w3.org/TR/CSP3/) | Execution/resource restriction as defense-in-depth. citeturn22view7 |
| PDF structure | [qpdf](https://qpdf.readthedocs.io/) | Structured inspection, validation and JSON representation. citeturn15view6 |
| Python PDF object access | [pikepdf](https://pikepdf.readthedocs.io/) | qpdf-backed object and XMP manipulation. citeturn15view7 |
| PDF rendering/visibility | [PyMuPDF](https://pymupdf.readthedocs.io/) | Extraction, rendering, opacity, painting order and OCG information. citeturn15view8 |
| PDF raster/OCR canonicalization | [OCRmyPDF](https://ocrmypdf.readthedocs.io/) | Explicit visible/invisible OCR treatment and force-rasterization behavior. citeturn19search2 |
| Image metadata | [ExifTool](https://exiftool.org/) | Broad metadata inspection/removal, with appropriately documented limitations. citeturn21search0 |
| XMP | [Adobe XMP](https://developer.adobe.com/xmp/docs/) | Primary description of embedded XMP metadata. citeturn15view14 |
| Office documents | [Microsoft Open XML SDK](https://learn.microsoft.com/en-us/office/open-xml/open-xml-sdk) | Structured access to standardized OOXML packages and properties. citeturn21search4turn21search1 |
| Prompt-injection classifier | Meta Prompt Guard 2 | Purpose-built small classifiers for prompt-injection/jailbreak detection. citeturn15view11 |
| Context/provenance defense | Microsoft Spotlighting | Explicitly distinguishes untrusted input sources through provenance transformations. citeturn15view10 |

The comparative security value of the complete defense stack is:

| Defense | Accuracy against its target phenomenon | Performance cost | Typical false positives | Security value |
|---|---|---|---|---|
| Exact control-character scanner | Very high | Negligible | Low unless all `Cf` characters are treated equally | Essential |
| Unicode normalization comparison | Very high for equivalent/compatibility variants | Negligible | Low | Essential |
| UTS #39 confusable checks | High for covered visual confusables | Low | Medium in multilingual prose | Important supporting signal |
| HTML structural sanitizer | High for prohibited markup/XSS constructs | Low–medium | Low | Necessary but insufficient |
| Computed visibility analysis | High for common CSS hiding | Medium | Medium | Strong |
| DOM/rendered-text differential | High for DOM/CSS discrepancies | Low–medium | Low–medium | Strong |
| Screenshot OCR differential | High for pixel-vs-text discrepancies | High | Medium on poor scans/fonts | Very strong |
| PDF object audit | High for explicit PDF structural anomalies | Medium | Low if features are interpreted contextually | Essential for PDFs |
| PDF flatten/raster rebuild | Very high for removing nonvisual logical state | High | Low security FP; potentially high functionality loss | Strongest canonicalization boundary |
| Metadata deletion in place | Medium | Low | Low | Useful, not sufficient |
| Decode/re-encode/rebuild | High for container-state elimination | Medium–high | Functionality loss | Preferred high-assurance approach |
| Regex injection scanner | High precision on known phrases | Very low | Potentially high on security/manual text | Baseline only |
| Small transformer guard | Good in-distribution | Low–medium | Domain-dependent | Strong ensemble component |
| Contextual alignment classifier | Potentially better FP behavior | Medium | Model/domain-dependent | Recommended |
| LLM guard | Strong semantic reasoning in many benchmarks | High | Variable | Escalation layer |
| Model-only self-defense | Fundamentally inadequate as sole boundary | Low–medium | N/A | Never sufficient; adaptive studies demonstrate failures. citeturn15view12 |

The resulting security doctrine is straightforward: **preserve first, inspect structure second, render independently third, compare views fourth, sanitize fifth, classify sixth, and only then construct an AI-specific payload.** Sanitization without detection can erase forensic evidence; classification without rendering cannot tell visible text from hidden text; OCR without structural parsing misses active and nonvisual state; Unicode stripping without linguistic context damages legitimate text; and a language-model guard without deterministic boundaries remains susceptible to adaptive prompt injection. The layered approach is consistent with Unicode's security mechanisms, W3C's defense-in-depth position on browser content, modern PDF visibility tooling, document-aware 2026 benchmark results, and the continuing evidence that prompt-injection classifiers and model-side defenses should be treated as fallible security components rather than proofs of safety. citeturn15view1turn22view7turn15view8turn22view0turn22view1turn15view12