[EAAPL-SEC007] Zero-Trust AI Pipeline

Category: Security / Architecture Sub-category: Pipeline Security Model Version: 1.1 Maturity: Proven Tags: zero-trust mtls identity-verification micro-segmentation jit-access continuous-verification pipeline-security Regulatory Relevance: APRA CPS234, NIST SP 800-207 (Zero Trust Architecture), EU AI Act Art. 9, ISO 27001 A.9, ISO 42001 §6.1

1. Executive Summary

The Zero-Trust AI Pipeline applies the "never trust, always verify" security architecture principle to the complete AI request pipeline — from the initial client request through gateway, prompt processing, model inference, tool calls, and response delivery. Traditional perimeter security assumes that traffic inside the enterprise network is trustworthy. Zero-trust rejects this assumption: every component, every service call, and every data exchange must be authenticated, authorised, and verified regardless of network location.

This matters specifically for AI pipelines because they involve multiple service-to-service calls (gateway → prompt filter → model → tool → output filter), often span cloud and on-premises boundaries, and process data that, if intercepted at any hop, can represent a significant security breach. An AI pipeline that trusts its internal service calls is only as secure as its least-secured component — and AI pipelines have more components than most.

The pattern establishes five pillars for a zero-trust AI pipeline: identity verification on every service call (mutual TLS + JWT), micro-segmentation of pipeline stages, just-in-time access for AI workloads (no standing permissions), continuous verification of pipeline integrity, and comprehensive audit of every inter-service exchange. Organisations implementing this pattern achieve a documented reduction in lateral movement risk and a clear audit trail that satisfies both security operations and regulatory requirements.

2. Problem Statement

Business Problem

Enterprise AI pipelines are complex distributed systems with many inter-service dependencies. Organisations that have invested in perimeter security (firewalls, VPN) often assume that traffic between internal services is safe — a "flat network" model. For AI pipelines, this creates an unacceptable risk: if any single component in the pipeline is compromised (through a vulnerability, supply chain attack, or insider threat), an attacker has implicit trust across the pipeline and can intercept or inject data at any point.

Technical Problem

AI pipelines typically have multiple service-to-service calls that are authenticated weakly or not at all:

• Gateway to prompt firewall: internal HTTP call with no authentication.
• Model inference to tool endpoints: service account with broad permissions.
• RAG retrieval service to vector database: network-only access control.
• Output filter to model: no authentication (same cluster).
• Pipeline services sharing a single service account credential.

None of these connections verify that the caller is who it claims to be, or that the caller's permissions are appropriate for the specific request being made.

Symptoms

• Internal service calls using API keys stored in environment variables.
• Microservices sharing a single omnibus service account.
• No mTLS between AI pipeline components.
• Service-to-service calls not logged separately from client-to-gateway calls.
• Broad IAM permissions on model serving infrastructure.
• No mechanism to detect if a pipeline component has been tampered with.

Cost of Inaction

3. Context

When to Apply

• Multi-component AI pipelines with more than one service-to-service call.
• AI pipelines that cross network boundaries (on-premises to cloud, VPC to VPC).
• Organisations in regulated industries where audit of data flow is required.
• AI pipelines processing data at CONFIDENTIAL classification or above.
• Organisations with mature identity infrastructure (existing PKI or SPIFFE implementation).

When NOT to Apply

• Single-process AI applications (model and application in the same process/container) — zero-trust is a network architecture pattern.
• Early-stage development environments where operational overhead outweighs security benefits.
• Extremely latency-sensitive inference paths where mTLS handshake overhead is unacceptable (note: with session resumption, mTLS adds <1ms per request after the initial handshake).

Prerequisites

Industry Applicability

4. Architecture Overview

The zero-trust AI pipeline replaces all implicit trust relationships between pipeline components with explicit, verified, and logged trust relationships. Every component operates as if it is on a hostile network — because in a zero-trust model, it is.

Pillar 1: Workload Identity (SPIFFE/SPIRE)

Every pipeline component — gateway, prompt firewall, model server, RAG retriever, tool adapter, output filter — is assigned a cryptographic workload identity via SPIFFE (Secure Production Identity Framework for Everyone). Each workload receives a short-lived X.509 certificate (SVID) issued by SPIRE, valid for 1 hour and automatically rotated. The SVID encodes the workload's identity (e.g., spiffe://enterprise.ai/gateway, spiffe://enterprise.ai/model-server). No component can forge another component's identity — it requires the SPIRE agent running on the specific compute instance.

Pillar 2: Mutual TLS on Every Service Call

Every service-to-service call in the pipeline uses mutual TLS (mTLS): both the caller and the callee present their SPIFFE SVIDs. This provides cryptographic authentication at both ends of every connection. A compromised component cannot impersonate another component; a man-in-the-middle attack is cryptographically impossible. Service mesh sidecars (Envoy via Istio or Linkerd) handle mTLS transparently — application code makes plain HTTP calls; the sidecar upgrades to mTLS and validates peer identity.

Pillar 3: Per-Request Authorisation

Authentication (you are who you claim to be) is necessary but not sufficient. Authorisation (you are permitted to make this specific request) must also be verified on every request. OPA policies evaluate: is the calling component permitted to call this component? Is the data classification of the request within the permitted range for this component pair? Is the request rate within configured limits?

Critically, authorisation is scoped to the request, not the connection. A gateway instance that is permitted to call the prompt firewall is not automatically permitted to call the model server directly — each hop's authorisation is evaluated independently.

Pillar 4: Just-in-Time Access

No pipeline component holds standing permissions to the resources it needs. Instead:

• Model weights are fetched from the registry at startup using a time-limited Vault lease.
• Tool call credentials are generated at the moment of tool invocation, scoped to the specific tool and operation.
• Database read credentials for RAG retrieval are generated per-session, scoped to the retrieval tenant's data.
• Cloud IAM role assumptions are time-limited (1-hour maximum).

JIT access eliminates the risk of credential theft: stolen credentials are expired within minutes to an hour, dramatically reducing the attack window.

Pillar 5: Continuous Verification and Pipeline Integrity

Zero-trust is not a static configuration — it requires continuous verification. This includes:

• Periodic rotation of SVIDs (every 1 hour) and re-authentication.
• Runtime integrity checks: each pipeline component signs its outputs; downstream components verify the signature before processing. If a prompt firewall output arrives at the model server without a valid signature from the firewall, the request is rejected.
• Anomaly detection on pipeline traffic patterns: unexpected call volumes, unusual source identities, or calls between non-adjacent components trigger alerts.
• Binary authorisation: container images in the pipeline must be signed and verified against an approved image registry before deployment.

5. Architecture Diagram

6. Components

7. Data Flow

Primary Flow

Error Flow

8. Security Considerations

Authentication & Authorisation

• Every component has a unique, cryptographic workload identity (SPIFFE SVID).
• Every inter-component call authenticated via mTLS using SPIFFE SVIDs.
• Every call authorised by OPA with policies that specify exactly which components may call which other components.

Secrets Management

• No standing credentials. All credentials JIT-issued by Vault with minimum TTL.
• SVID private keys never leave the compute instance.

Data Classification

• OPA policies can enforce classification-based routing: a CONFIDENTIAL request may only traverse components cleared for CONFIDENTIAL processing.

Encryption

• All inter-service communication: TLS 1.3 via mTLS.
• TLS session keys rotated with SVID rotation (every hour maximum).
• At-rest encryption on all pipeline state.

OWASP LLM Top 10 Coverage

9. Governance Considerations

Governance Artefacts

10. Operational Considerations

SLOs

Incident Management

• SVID rotation failure → P1: affected component isolated; SPIRE investigation.
• Pipeline integrity violation (signature failure) → P1: security incident; possible MITM; full pipeline forensics.
• Binary authorisation failure → P2: deployment blocked; investigate image provenance.

DR

11. Cost Considerations

Cost Drivers

Indicative Cost Range

12. Trade-Off Analysis

Option Comparison

Architectural Tensions

13. Failure Modes

14. Regulatory Considerations

15. Reference Implementations

AWS

Azure

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Proven

18. Revision History