EAAPL-CMP004 — Privacy-Preserving AI Architecture

Status: Proven
Tags: privacy-act gdpr pii-handling data-isolation high-complexity
Version: 1.4
Last Updated: 2026-06-12
Author: Enterprise AI Architecture Pattern Library

1. Executive Summary

AI systems that ingest, process, or generate personal information must comply with privacy obligations across multiple jurisdictions simultaneously. This pattern addresses the dual requirements of the Australian Privacy Act 1988 (Commonwealth) Australian Privacy Principles and the EU General Data Protection Regulation. Both frameworks impose obligations on data minimisation, purpose limitation, cross-border transfer restrictions, consent management, and individual rights—each of which creates specific technical requirements when applied to AI inference pipelines, retrieval-augmented generation systems, and AI training workflows.

The Privacy-Preserving AI Architecture provides a layered control framework: a PII detection and redaction pipeline that sanitises inputs before they reach the LLM; purpose limitation enforcement through technical controls on data flows; a consent management integration that gates AI processing on valid consent; data sovereignty controls that prevent regulated personal data from crossing prohibited geographic boundaries; and a machine unlearning capability to respond to erasure requests. Organisations implementing this pattern demonstrate technical compliance with Privacy Act APP 11 (security of personal information), GDPR Articles 5, 6, 17, 20, and 25, and build AI systems that earn customer trust through verifiable privacy practices.

2. Problem Statement

Business Problem

Organisations building AI products on top of personal data face mounting privacy enforcement risk. The Australian Information Commissioner has signalled enforcement focus on AI systems in the 2025–2026 period. European Data Protection Authorities issued EUR 4.5 billion in GDPR fines through 2025, with AI-related processing accounting for a growing share. Privacy-by-design is a legal obligation under GDPR Article 25, not an optional design choice.

Technical Problem

Default LLM and AI service configurations are not privacy-preserving. Cloud LLM providers may use submitted data for model improvement. PII is routinely present in business documents, customer support tickets, and employee records that are ingested as RAG context. Training datasets frequently contain PII that was not adequately screened. The legal basis for AI inference on personal data is frequently assumed (legitimate interest) without a genuine balancing test.

Symptoms

• Customer PII (names, email addresses, account numbers) is sent directly to third-party LLM APIs without redaction
• There is no documented lawful basis under GDPR Article 6 or APP 3 for the specific AI processing activity
• Privacy Impact Assessments are not conducted before deploying AI features that process personal data
• RAG vector stores contain embeddings derived from personal data with no retention policy or deletion capability
• The organisation cannot respond to a data subject erasure request because it cannot identify all locations where AI systems have stored or derived representations of that individual's data

Cost of Inaction

3. Context

When to Apply

• Any AI system that receives, processes, or outputs personal information as defined by Privacy Act (information about an identified or reasonably identifiable individual) or GDPR (any information relating to an identified or identifiable natural person)
• AI systems processing sensitive information (health, financial, biometric, religious, political, sexual orientation) — highest risk tier requiring strongest controls
• RAG pipelines that index internal documents containing personal information
• AI training workflows using datasets that may contain personal information
• Cross-border AI services where data flows from EU or Australia to third-country AI providers

When NOT to Apply

• AI systems processing exclusively synthetic or fully anonymised data (k-anonymity + l-diversity + t-closeness with formal guarantee)
• Internal AI tools used by authorised staff on data they are already authorised to access, where no third-party cloud AI API is involved
• AI research on fully de-identified public datasets with no personal data re-identification risk

Prerequisites

Industry Applicability

4. Architecture Overview

The Privacy-Preserving AI Architecture implements privacy controls at each stage of the AI data lifecycle: input, inference, retrieval, output, and storage. The architecture is founded on Privacy by Design (GDPR Article 25) — controls are embedded in the system, not applied as an afterthought.

Stage 1 — Data Minimisation at Input Before any data reaches the AI inference engine, a data minimisation filter applies the principle of processing only what is strictly necessary for the declared purpose. For structured inputs, this means stripping fields that are not required for the AI task (e.g., if an AI system summarises customer support tickets, it does not need the customer's date of birth — this field is removed). For unstructured text inputs, a PII detection and redaction pipeline identifies and pseudonymises or redacts personal identifiers. The minimisation configuration is driven by the purpose specification: each AI use case has a documented purpose, and the data minimisation filter enforces the minimum data set for that purpose.

Stage 2 — PII Detection and Redaction Pipeline The core privacy technical control is a PII detection and redaction pipeline that processes all text inputs destined for LLM inference. The pipeline operates in two modes: redaction mode (PII replaced with a token such as [PERSON_NAME] or [EMAIL_ADDRESS]) for use cases where the AI does not need the actual values; and pseudonymisation mode (PII replaced with a reversible, keyed token) for use cases where the actual value is needed downstream but should not be sent to the LLM. Pseudonymisation uses a per-context encryption key managed in a secrets manager; the mapping between pseudonym and real value is stored in a secure lookup store that is never exposed to the AI model. The pipeline supports Australian and EU PII categories: names, addresses, dates of birth, email addresses, phone numbers, government identifiers (TFN, Medicare, ABN, passport), financial account numbers, health information, and biometric descriptors.

Stage 3 — Purpose Limitation Technical Controls Purpose limitation (APP 6; GDPR Article 5(1)(b)) requires that data collected for one purpose is not used for a different, incompatible purpose. In AI systems, the risk is that data collected for customer service is used to train a marketing personalisation model. Technical purpose limitation controls include: purpose tags on all data assets, enforced by a policy engine that blocks AI workflows from accessing data tagged for incompatible purposes; data access audit logs that record which AI workflow consumed which data; and change management gates that require re-assessment of lawful basis and consent when an AI workflow's purpose changes.

Stage 4 — Consent Management Integration Where consent is the lawful basis for AI processing (GDPR Article 6(1)(a); APP 3.3), the AI processing pipeline must check consent status before processing. The consent management integration queries the consent management platform (CMP) at the start of each processing job or inference request. If valid consent does not exist for the specific processing activity, the request is blocked and the user is prompted to provide consent. Consent must be granular — separate consent for different AI processing activities (personalisation vs. model training vs. analytics). Consent withdrawal must immediately prevent further processing; it does not require the same technical immediacy as an erasure request but must be honoured within a commercially reasonable timeframe (GDPR guidance: without undue delay).

Stage 5 — Cross-Border Transfer Controls GDPR Article 46 restricts transfers of personal data to third countries without adequate protection. Australia's APP 8 imposes a similar obligation on cross-border disclosure. Cloud LLM API calls that send EU or Australian personal data to a provider operating outside those jurisdictions require a valid transfer mechanism: GDPR Standard Contractual Clauses (SCCs), adequacy decision, or Binding Corporate Rules; Australian APP 8.1 accountability mechanism. The cross-border transfer control layer checks each outbound API call against the destination's jurisdiction status and the data's classification. If the data is subject to transfer restrictions and no valid mechanism exists, the call is blocked and a local model fallback is invoked.

Stage 6 — Machine Unlearning for Erasure Requests GDPR Article 17 (right to erasure) and APP 13 (correction and destruction) create obligations to delete personal information on request. For AI systems, this is technically complex because personal data may be encoded in model weights through training, in embedding vectors in vector stores, and in inference logs. The machine unlearning layer provides: immediate deletion from inference logs (database delete + audit record); deletion from RAG vector stores (identify all vectors derived from the data subject's documents; delete by document ID); for model weights — re-training or fine-tuning on a dataset with the individual's data removed (computationally expensive; may be deferred for large foundation models with documented justification); and an erasure request tracking system that records the status of each sub-task.

5. Architecture Diagram

6. Components

7. Data Flow

Primary Flow

Error Flow

8. Security Considerations

Privacy and Security Controls

OWASP LLM Top 10 — Privacy Mapping

9. Governance Considerations

Privacy Governance

Governance Artefacts

10. Operational Considerations

Monitoring and SLOs

Disaster Recovery

11. Cost Considerations

Cost Drivers

Indicative Cost Range

12. Trade-Off Analysis

Architecture Options

Architectural Tensions

13. Failure Modes

Cascading Failure Scenario

A new AI feature is deployed with RAG indexing of customer emails. The PIA was skipped because the feature was deemed "low risk" by the product team. Customer emails contain health information (sensitive personal data under GDPR Article 9). The RAG vector store creates embeddings of this health information. These embeddings are sent as context to a US-based LLM API with no GDPR Standard Contractual Clauses in place—a cross-border transfer without a valid transfer mechanism. A data subject files an erasure request; the organisation cannot identify the vectors derived from the subject's emails because the vector store has no per-document deletion capability and no document-to-vector mapping. The DPA investigation finds three simultaneous breaches: special category processing without explicit consent (Article 9), cross-border transfer without mechanism (Article 46), and inability to fulfil erasure right (Article 17). Fine issued; AI service suspended.

14. Regulatory Considerations

15. Reference Implementations

AWS

Azure

GCP

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity Label: Proven

18. Revision History