LLM apps constantly handle names, emails, health records, and account numbers - and much of that work happens at an external provider you don't control. Safe systems detect personal data, redact or tokenize it before it crosses the trust boundary, and rehydrate the answer afterward, so the model does useful work while the raw PII never leaves your perimeter.
Personally identifiable information (PII) is any data that can identify a person - directly (name, email, SSN) or indirectly (an address plus a birth date). A data boundary is the perimeter around a trusted zone: your database, your VPC, your on-prem cluster. Data-boundary handling is the discipline of deciding which fields may cross which perimeter to which model or provider - and transforming them so the crossing is safe.
Find PII in the text before it moves - using regex for structured identifiers, ML classifiers for names and addresses, and context to disambiguate.
Redact, mask, tokenize, or anonymize the detected spans so the payload leaving the boundary carries no raw personal data.
After the model responds, swap the placeholders back for the real values inside your perimeter, so the user sees a complete answer.
| Piece | What it does | Common choices |
|---|---|---|
| Detector | Locates PII spans in text and structured fields | Regex, NER models, Presidio, cloud DLP |
| Classifier | Labels each span by entity type & sensitivity | PERSON, EMAIL, SSN, MRN, PAN |
| Transformer | Applies redaction, masking, or tokenization | Mask, hash, format-preserving token |
| Token vault | Stores reversible token↔value mappings securely | KMS-backed store, HSM, secrets vault |
| Policy engine | Decides what may cross which boundary | Field allow-lists, residency rules, DPAs |
Sending a prompt to a hosted model means copying data to a third party. Without controls, that copy can be retained, logged, used for training, or stored in the wrong jurisdiction. Boundary handling keeps the useful signal flowing while the identifying details stay home.
GDPR, HIPAA, CCPA, and sector rules demand a lawful basis, data minimization, and control over where personal data goes. Redaction and DPAs are how you meet them.
If the model never sees raw PII, a provider incident or a leaked prompt exposes only meaningless tokens - not real identities.
Some data legally cannot leave a region. Boundary policy routes those fields to in-region or on-prem models, or strips them entirely before egress.
Zero-retention, no-training endpoints and signed agreements ensure your users' data isn't absorbed into a future model version.
Same idea, four ways to picture it, so it clicks whoever you are.
It's like blacking out your address on a form before you photocopy it for a group project. The copy still shows your answers, but the parts that identify you never leave your hands.
Think of it as a proxy that swaps secrets for placeholders before an outbound call, then swaps them back on the way in. The external service works with tokens; the real values stay behind your boundary.
Like a records clerk who sends a redacted case file to an outside consultant. The consultant reasons over the facts they are allowed to see, while the identifying pages stay locked in the office.
A black marker over a document before you hand it across a counter: the reader gets the gist, but the private lines are covered so they can't read or keep them.
Two concerns run in tandem: a data-transformation path that scrubs PII on the way out and restores it on the way back, and a governance layer of provider agreements, retention settings, and log hygiene that keeps the boundary honest.
Detect → classify → tokenize (reversible via a vault) or anonymize (irreversible) → send the scrubbed prompt to the model → map placeholders back to real values in the response.
Zero-retention / no-training endpoints, data-processing agreements, residency routing, and trace scrubbing so PII never lands in logs, observability, or backups.
flowchart LR
subgraph Trust["🏠 Trust perimeter"]
IN["📝 Raw input"] --> DET["🔍 Detect PII"]
DET --> CLS["🏷️ Classify + policy"]
CLS --> TOK["🎭 Tokenize / redact"]
TOK --> VLT[("🔐 Token vault")]
REH["🔁 Rehydrate"] --> OUT["✅ Final answer"]
VLT --> REH
end
subgraph Provider["☁️ External provider"]
LLM["🧠 LLM (zero-retention)"]
end
TOK -->|scrubbed prompt| LLM
LLM -->|tokenized answer| REH
spans = detect(text) # regex + NER + context rules
mapping = {}
scrubbed = text
for s in spans: # e.g. EMAIL, PERSON, SSN, PAN
token = make_token(s.type) # "[EMAIL_1]", "[PERSON_2]"
vault.put(token, s.value, ttl=600) # reversible, KMS-encrypted
mapping[token] = s.value
scrubbed = scrubbed.replace(s.value, token)
answer = llm.generate( # provider sees only tokens
endpoint="zero-retention",
prompt=scrubbed
)
final = answer
for token, value in mapping.items(): # rehydrate inside perimeter
final = final.replace(token, value)
vault.purge(mapping.keys())Reversible tokenization lets you restore real values via the vault; irreversible anonymization (masking or hashing with no mapping) is used when the data should never come back. Choose per field based on whether the final answer needs the real value.
Three views: the detect→redact→process→rehydrate round-trip, a boundary crossing that strips fields before egress, and a logging pipeline that scrubs PII before it hits your traces.
sequenceDiagram
autonumber
participant U as 👤 User
participant App as 🖥️ App
participant Det as 🔍 PII detector
participant Vault as 🔐 Token vault
participant LLM as 🧠 External LLM
U->>App: Submit text with PII
App->>Det: detect and classify spans
Det-->>App: entities with types
App->>Vault: store token to value map
Vault-->>App: reversible tokens
App->>LLM: send tokenized prompt
LLM-->>App: answer containing tokens
App->>Vault: resolve tokens to values
Vault-->>App: real values
Note over App,Vault: rehydrate inside perimeter then purge
App-->>U: complete answer
sequenceDiagram
autonumber
participant Svc as 🏠 Internal service
participant Pol as 📋 Policy engine
participant GW as 🚪 Egress gateway
participant Prov as ☁️ External provider
Svc->>Pol: request to call model with record
Pol->>Pol: evaluate field allow-list and residency
alt field must stay inside
Pol-->>GW: strip SSN, MRN, address
else field safe to send
Pol-->>GW: keep tokenized order id
end
GW->>GW: verify no raw PII remains
GW->>Prov: forward scrubbed payload
Prov-->>GW: response
Note over GW,Prov: zero-retention endpoint under a DPA
GW-->>Svc: response for rehydration
sequenceDiagram
autonumber
participant App as 🖥️ App
participant Log as 🧹 Scrubbing middleware
participant Trace as 📊 Tracing backend
participant Store as 🗄️ Log store
App->>Log: emit event with prompt and response
Log->>Log: detect and mask PII in payload
alt sensitive field found
Log->>Log: replace with redacted marker
else no PII detected
Log->>Log: pass through unchanged
end
Log->>Trace: send sanitized span
Trace->>Store: persist scrubbed trace
Note over Log,Store: raw PII never reaches observability
Store-->>App: ack
From raw user input to a complete answer - the whole journey with PII controlled at every crossing.
Map which fields in the request are PII, PHI, or financial data, and record their sensitivity and residency requirements up front.
The policy engine checks a field allow-list, residency rules, and the active data-processing agreement to determine what can leave the perimeter.
Run regex on structured identifiers and an NER/ML classifier on unstructured text, using context to catch names, addresses, and account numbers.
Tag every detected span by entity type and confidence so the right transform is applied - and low-confidence matches can be reviewed.
Reversibly tokenize fields the answer will need back; irreversibly mask or drop fields that must never return.
Write token↔value pairs to a KMS-encrypted vault with a short TTL, so the mapping exists only as long as the request needs it.
At the boundary, scan the outgoing payload one last time and block it if any raw PII slipped through detection.
Forward the scrubbed prompt to a no-training, zero-retention model endpoint covered by a signed DPA and, where required, in-region.
Map placeholders in the model's response back to real values using the vault, entirely within the trust perimeter.
Scrubbing middleware masks PII before any prompt or response is written to logs, tracing, or analytics, then purges the token mapping.
The user receives a full, accurate answer while the provider only ever saw tokens - and audit records prove what crossed the boundary.
Most boundary failures come from gaps in coverage, not from the model. These are the usual culprits to defend against.
Patterns catch structured identifiers but miss names, addresses, and context-dependent PII. Pair regex with an NER/ML classifier and context rules so unstructured personal data isn't silently waved through.
Prompts and responses copied verbatim into logs, spans, or error reports re-expose the very data you scrubbed. Run scrubbing middleware before anything reaches observability, analytics, or backups.
Masking or hashing a field you later have to restore breaks the answer. Choose reversible tokenization (via a vault) for fields the response must return, and irreversible anonymization only when the value should never come back.
Sending raw data to an endpoint that retains or trains on inputs means a copy lives outside your control. Route regulated data only to no-training, zero-retention endpoints under a signed agreement.
Shipping fields to a model in the wrong jurisdiction can breach law even if the data is tokenized. Encode residency in policy and route or strip in-region before egress.
Scrubbing so aggressively that the model loses the context it needs yields useless answers. Tune detection so it protects identities while preserving the non-identifying signal the task depends on.
Track protection and usefulness together, so you know whether the boundary is both safe and still doing useful work.
| Metric | What it tells you | Good sign |
|---|---|---|
| Detection recall | What share of real PII the detector actually catches | High: little sensitive data slips through |
| False-positive rate | How often non-PII is flagged and needlessly redacted | Low: task text stays intact |
| Leakage incidents | Count of raw PII reaching a provider, log, or store | Zero: the boundary held |
| Rehydration accuracy | Whether placeholders map back to the correct values | High: answers restore cleanly |
| Added latency | Extra time the detect-transform-rehydrate path costs | Within your SLA budget |
| Coverage across data types | Which entity types and formats are handled (text, structured, tables) | High: no blind spots by data type |
Three representative patterns showing data-boundary handling in production-style use.
A clinical app summarizes patient notes with an LLM, but protected health information may never reach an unqualified provider.
A banking assistant answers questions about transactions, but account and card numbers must stay within the institution's boundary.
A company mines millions of support transcripts for insights, but each transcript is full of customer names, emails, and addresses.
As LLMs move deeper into regulated workflows, boundary controls are shifting from bolt-on filters to first-class infrastructure.
LLM-based detectors that understand meaning will catch indirect and combined identifiers that regex and classic NER miss.
Trusted execution environments and encrypted inference let providers process data they cannot read, tightening the boundary further.
Data-boundary rules expressed as versioned, testable policy, enforced automatically at every egress point.
Treating leak rate as a monitored metric - measuring detection recall and scrubbing coverage the way teams measure uptime.
Capable local models keep the most sensitive data inside the perimeter entirely, removing the crossing rather than protecting it.
Differential-privacy and formal guarantees that let organizations demonstrate, not just assert, that data was anonymized.