[EAAPL-GOV007] AI Audit Trail

Category: Governance / Compliance & Traceability Sub-category: Immutable Decision Logging Version: 2.0 Maturity: Mature Tags: audit-trail, immutable-log, decision-traceability, WORM, tamper-evident, regulatory-retention Regulatory Relevance: APRA CPS230 §32, APRA CPS234 §19, EU AI Act Article 12, Privacy Act APP 11, ISO/IEC 42001 §9.1, NIST AI RMF MANAGE 4.1

1. Executive Summary

The AI Audit Trail pattern establishes an immutable, tamper-evident log of every AI decision made by enterprise AI systems. It captures the full decision context: sanitised input, model version, prompt version (for generative AI), output, confidence score, policy decisions applied, human overrides, and data sources consumed. Retention is aligned to regulatory requirements—7 years for APRA-regulated entities, which exceeds standard application logging.

This pattern is foundational to enterprise AI governance. Without an audit trail, every governance claim is unverifiable: you cannot prove a policy was applied, you cannot reconstruct what a model said to a customer, you cannot demonstrate human oversight occurred, and you cannot respond to regulatory requests for decision evidence. The EU AI Act Article 12 makes logging mandatory for high-risk AI systems; APRA CPS234 requires evidence of information security controls.

Beyond compliance, the audit trail enables capability that pure governance cannot: post-incident forensics to understand exactly what an AI system did; statistical analysis of decision patterns to detect emerging bias; accountability assignment when AI decisions are disputed; and model performance benchmarking against actual outcomes.

The pattern's defining architectural commitment is immutability. Write-once, read-many (WORM) storage with cryptographic integrity verification ensures that audit records cannot be modified, deleted, or backdated. This is not a logging best practice—it is a regulatory requirement for any organisation claiming its AI audit trail as compliance evidence.

2. Problem Statement

Business Problem

When AI decisions are disputed—by customers, regulators, or in legal proceedings—organisations cannot reconstruct what the AI system actually said, what data it used, and what policies governed it. Standard application logs are insufficient: they are not retained long enough, they are not structured for decision reconstruction, and they are not tamper-proof.

Technical Problem

AI decisions have richer context than standard API calls: model version, prompt version, retrieved context (for RAG systems), token-level confidence, policy decisions from GOV004, human override events from GOV005, and ground-truth outcome feedback. Standard logging infrastructure cannot capture this structured, multi-source context and retain it in WORM format for 7 years.

Symptoms

• Inability to reconstruct what an AI system told a specific customer on a specific date
• No evidence that policy guardrails were applied to a flagged AI decision
• Human override events not captured, preventing audit of human oversight effectiveness
• AI decision logs retained for 90 days (standard app log retention) vs. 7-year regulatory requirement
• Logs mutable by administrators, failing tamper-evidence requirements
• No correlation between AI decision and downstream outcome (outcome feedback not captured)

Cost of Inaction

• Regulatory: APRA CPS234 §19 non-compliance (information security evidence); EU AI Act Article 12 (logging obligation)
• Legal: Inability to defend AI decisions in legal proceedings due to absence of evidence
• Operational: Post-incident forensics impossible; root cause analysis speculative
• Governance: Human oversight claimed but not evidenced; responsible AI controls unverifiable

3. Context

When to Apply

• All AI systems making decisions affecting individuals in regulated industries
• Any AI system subject to EU AI Act high-risk classification (Annex III)
• AI systems in APRA-regulated entities (all, per CPS234 information security obligations)
• Any AI system where post-incident forensics capability is required
• Generative AI systems producing customer-facing outputs (financial advice, healthcare recommendations)

When NOT to Apply

• Internal AI systems with no customer or regulatory exposure and sub-30-day retention acceptable
• Ultra-high-volume, low-consequence AI (e.g., real-time recommendation clickthrough) where full decision logging is cost-prohibitive — use sampled logging with defined sampling strategy

Prerequisites

• Information classification scheme for AI inputs and outputs
• WORM-capable storage infrastructure
• Identity context available at inference time (user ID, session ID)
• Model register (GOV001) operational — MRID required in each log entry

Industry Applicability

4. Architecture Overview

The AI Audit Trail is architected around two non-negotiable properties: completeness and immutability. Completeness means every decision is captured with sufficient context to reconstruct the decision scenario. Immutability means no record can be modified or deleted once written, including by system administrators.

Decision Record Schema. The audit record schema is the most critical design decision in this pattern. It must be rich enough to satisfy regulatory reconstruction requirements while not so voluminous that storage costs become prohibitive at scale. The schema stratifies into four payload tiers:

Mandatory Tier (always captured): Decision ID (UUID), MRID (model version, architecture hash), timestamp (UTC, nanosecond precision), actor identity (user ID, session ID, application context), input fingerprint (cryptographic hash of sanitised input — not the raw input, which may contain PII), output summary (sanitised, PII-free summary of the model output), decision type (classification/recommendation/generation), confidence score, latency, policy decision reference (GOV004), and regulatory context flags.

Decision Tier (for consequential decisions): Full decision rationale (for explainable models), counterfactual summary, data sources referenced (document IDs for RAG systems, feature values for tabular models), fairness context (demographic group if known), and human oversight indicator.

Override Tier (conditional — only when human override occurs): Override actor identity, override timestamp, override rationale, original AI decision preserved, override decision, and escalation reference.

Outcome Tier (populated retrospectively): Ground truth outcome, outcome timestamp, outcome source, accuracy flag. This tier enables model performance analysis against real-world outcomes and is critical for GOV006 bias detection (equalised odds requires ground truth).

Immutability Architecture. The immutability guarantee is implemented at multiple layers to prevent single points of bypass. First, the audit log writer is the only component with write access to the log store—application code cannot write directly. Second, the log store is configured as WORM (Write Once Read Many) with Compliance mode (not Governance mode), meaning even the storage administrator cannot delete or modify records during the retention period. Third, each record is written with a cryptographic hash of its content, enabling integrity verification without trusting the storage layer alone. Fourth, a Merkle-tree-based tamper-evidence chain links records so that any retrospective modification is detectable from the chain.

PII Sanitisation at Write Time. Raw AI inputs often contain personal information. The audit trail must preserve decision context without creating a 7-year PII retention risk in violation of Privacy Act obligations. The sanitisation pipeline, executed before write, applies: named entity recognition to identify PII fields, substitution of PII values with entity type tokens (e.g., [PERSON], [EMAIL], [ACCOUNT_NUMBER]), preservation of non-PII decision-relevant features, and a cryptographic binding between the sanitised record and the original request (so the original can be retrieved under legal process if required, from the primary application system which has appropriate retention).

Retention Tiering for Cost Management. Seven-year retention at full fidelity is cost-prohibitive for high-volume systems. The pattern implements tiered retention: hot tier (0–90 days, full queryable index, high-speed retrieval), warm tier (90 days–2 years, reduced query access, compressed storage), cold tier (2–7 years, compliance archive, retrieval SLA 24 hours, minimum cost storage). The WORM guarantee applies across all tiers.

Query Architecture. The audit trail serves two query patterns with different characteristics: operational queries (find all decisions for customer X in the past 30 days — requires fast index on user ID and date) and forensic queries (reconstruct decision state for model version Y on date Z — requires full scan with filter, less latency-sensitive). The pattern implements a search index over the hot and warm tiers for operational queries, with direct S3/blob scanning available for forensic queries.

5. Architecture Diagram

6. Components

7. Data Flow

Primary Audit Record Write Flow

Regulatory Query Flow

8. Security Considerations

Immutability Enforcement Layers

1. Application layer: Audit Log Writer is only write path; no other service has write credentials
2. Database layer: PostgreSQL append-only enforced via trigger (no UPDATE/DELETE permitted)
3. Storage layer: S3 Object Lock Compliance mode — storage administrators cannot override retention
4. Verification layer: Daily Merkle chain verification detects any tampering at any layer

Access Control

• Read access requires Audit Reader role (minimum); Compliance role for full record access; Legal role for original pre-sanitised data (requires court order workflow)
• All reads logged (who read which records, when) — audit of the audit
• No bulk export without specific Compliance Director approval

OWASP LLM Top 10 Mapping

9. Governance Considerations

Retention Policy Governance

Retention periods are set by Legal + Compliance, not by technology teams. Different model use cases may have different retention requirements. The retention policy table is version-controlled and reviewed annually.

Governance Artefacts

10. Operational Considerations

SLOs

Disaster Recovery

11. Cost Considerations

Cost Drivers

Indicative Total Annual Cost

12. Trade-Off Analysis

Option Comparison

Architectural Tensions

13. Failure Modes

14. Regulatory Considerations

APRA CPS230

• §32: Record-keeping obligations for APRA-regulated entities require retention of records related to material operations for 7 years. AI decision records for credit, insurance, superannuation decisions are material operation records.

APRA CPS234

• §19: APRA-regulated entities must retain information security-relevant logs. AI decision logs containing policy enforcement decisions satisfy this obligation.

EU AI Act

• Article 12: Logging capabilities for high-risk AI systems. Providers must ensure high-risk AI systems have automatic logging of events throughout lifetime. This pattern implements Article 12(1) and 12(2) requirements.
• Article 12(4): For AI systems in Annex III categories related to critical infrastructure, public authorities, migration — logs must be kept for period specific to use case. Pattern implements configurable retention per use case.

Privacy Act 1988 / APPs

• APP 11: Reasonable steps to protect personal information. PII sanitisation before writing to long-term audit log is the key control.
• APP 12: Access to personal information. Audit records about an individual are accessible to them on request; search index supports this.

ISO/IEC 42001

• §9.1: Monitoring and measurement of AI management system effectiveness. Audit trail provides the evidence base for effectiveness assessment.

15. Reference Implementations

AWS

Azure

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Mature (Level 4)

18. Revision History