[EAAPL-DAT006] Federated Learning Pattern

Category: Data Architecture
Sub-category: Distributed AI / Privacy-Preserving ML
Version: 1.1
Maturity: Emerging
Tags: federated-learning, FedAvg, differential-privacy, gradient-aggregation, consortium-AI, cross-silo, cross-device
Regulatory Relevance: GDPR Article 5/25, Privacy Act (Australia) APP 3, EU AI Act Article 10, APRA CPS 234, ISO 42001 §8.4

1. Executive Summary

Federated learning enables multiple organisations or devices to collaboratively train a shared AI model without any participant sharing their raw data. Each participant trains a local model on their own data and shares only model gradients or weights with a central coordinator, which aggregates them into an improved global model. No raw data ever leaves the participant's environment.

This pattern is transformative for industries where data cannot be centralised due to privacy regulation, competitive sensitivity, or jurisdictional restrictions: hospital consortia training clinical AI without sharing patient records; competing banks jointly training fraud detection models; mobile devices improving language models without uploading private messages.

The pattern covers cross-silo federation (organisation-to-organisation, typically <100 participants with high-quality data) and cross-device federation (device-to-server, potentially millions of participants with intermittent connectivity). It addresses the critical engineering challenges: communication efficiency, statistical heterogeneity (non-IID data), system heterogeneity, and privacy amplification through differential privacy on gradient updates.

Target audience: Chief Data Officers, AI Research leads, Healthcare/Banking consortium architects, ML Platform leads.

2. Problem Statement

Business Problem

High-value AI use cases require data that no single organisation can legally or competitively accumulate. Healthcare networks want clinical AI trained on all patient populations; banks want fraud models trained on cross-institution transaction patterns; governments want public health AI trained across jurisdictions. Traditional data sharing is blocked by privacy law, competition law, or data sovereignty requirements.

Technical Problem

• Centralising personal data from multiple organisations violates GDPR, APRA, and similar privacy regulations.
• Data sharing agreements between competing organisations are commercially infeasible.
• Cross-border data transfer restrictions prevent cloud-based centralisation.
• Each organisation has too little data alone; combined they have sufficient statistical power.
• Even with anonymisation, competitive organisations will not share detailed customer data.

Symptoms

• AI use case identified as high-value but blocked at data access stage indefinitely.
• Each participant's model has poor performance due to small or non-representative local dataset.
• Privacy counsel blocking data consortium proposals.
• Regulators signalling openness to federated approaches (e.g., EBA guidance on federated fraud detection).

Cost of Inaction

3. Context

When to Apply

• Multiple organisations or devices hold relevant training data that cannot be centralised.
• Privacy regulation or competition law prohibits raw data sharing.
• Cross-border data transfer restrictions apply.
• Data is sufficiently heterogeneous that centralisation would require complex harmonisation.
• Participants have sufficient compute to run local training (cross-silo: always; cross-device: modern smartphones).

When NOT to Apply

• Data can be legally and practically centralised (federated adds unnecessary complexity).
• Participants lack compute for local training.
• Data is severely heterogeneous (non-IID) to the point where federated training diverges (use transfer learning instead).
• Security of gradient transmission cannot be guaranteed (gradient inversion attacks possible in low-participant settings).
• Regulatory framework requires centralised data audit (some jurisdictions require data in a single auditable location).

Prerequisites

Industry Applicability

4. Architecture Overview

Design Philosophy

Federated learning inverts the traditional ML paradigm: rather than bringing data to the model, the model is brought to the data. The architecture must solve five engineering challenges simultaneously.

Challenge 1 — Communication Efficiency. In cross-silo FL, gradient transmission between participants and the coordinator is the primary bottleneck. Full gradient transmission for large neural networks can involve gigabytes per round. The pattern addresses this through gradient compression (sparsification: transmit only top-k% gradients by magnitude; quantisation: reduce gradient precision from float32 to int8 or 4-bit), achieving 10–100× communication reduction with minimal accuracy loss.

Challenge 2 — Statistical Heterogeneity (Non-IID Data). Each participant's data reflects their specific population, which may differ significantly from the global distribution. Naive FedAvg converges poorly on non-IID data. The pattern addresses this through FedProx (adds a proximal term to local loss function, preventing local models from drifting too far from the global model) and SCAFFOLD (corrects for client drift using control variates). For severely heterogeneous data, personalised federated learning (each participant maintains a local fine-tuned head on top of the shared global representation) is used.

Challenge 3 — Privacy Amplification. Gradient sharing, while safer than raw data sharing, is not perfectly private — gradient inversion attacks can recover training data from gradients in small-participant settings. The pattern applies Differential Privacy via DP-SGD (Opacus) at each participant before gradient transmission, adding calibrated Gaussian noise to clip-and-noised gradients. Secure aggregation protocols (Google's SecAgg) allow the coordinator to compute the aggregate gradient without seeing individual participants' gradients — even the coordinator cannot invert a specific participant's gradient.

Challenge 4 — System Heterogeneity. In cross-device FL, participants have wildly varying compute capabilities and connectivity. The pattern implements asynchronous federated learning (FedAsync): participants submit gradients when ready rather than in synchronised rounds; the global model is updated with each received gradient using a mixing hyperparameter. This handles stragglers without blocking the federation round.

Challenge 5 — Byzantine Robustness. Malicious or faulty participants may submit adversarial gradients to corrupt the global model (model poisoning). The coordinator applies gradient validation: per-participant gradient norms are compared; outliers (>3σ from mean) are rejected or down-weighted. FedMedian aggregation (median instead of mean) provides Byzantine-robust aggregation in adversarial settings.

Federation Coordinator. The coordinator orchestrates rounds: selects participants, distributes the global model, collects and aggregates gradients, validates gradient integrity, and updates the global model. The coordinator does not process raw data. In cross-silo settings, the coordinator may be hosted by a neutral third party (industry consortium body, regulator-approved platform) to ensure no participant gains competitive advantage.

5. Architecture Diagram

6. Components

7. Data Flow

Primary Flow

Error Flow

8. Security Considerations

Authentication & Authorisation

• Each participant authenticates to coordinator using mutual TLS certificates; participant identity linked to federation legal agreement.
• Coordinator validates participant identity before distributing global model.

Secrets Management

• DP-SGD noise seed managed locally by each participant; not shared with coordinator.
• Secure Aggregation protocol keys ephemeral per round; participants derive shared secrets using Diffie-Hellman.

Data Classification

• Raw training data classified as Confidential or higher at each participant; never transmitted.
• DP gradients classified as Internal; coordinator sees only aggregated gradient.
• Global model classified per use case sensitivity; clinical AI models typically Confidential.

Encryption

• All gradient transmission encrypted using TLS 1.3.
• Secure Aggregation provides additional cryptographic guarantee: coordinator cannot decrypt individual participant gradients.

Auditability

• Every federation round logged: round number, participants, aggregation method, global model version, evaluation metrics.
• DP budget consumption per participant logged; shared with participant for their own privacy accounting.
• Gradient rejection events logged with reason; reviewed by consortium governance.

OWASP LLM Top 10 Mapping

9. Governance Considerations

Responsible AI

• Federated models may perform differently across participant populations (fairness concern): hold-out evaluation must include per-participant subgroup performance metrics.
• Consortium governance board responsible for global model promotion decisions when performance is uneven.

Model Risk Management

• Global model is trained on distributed data with different quality levels; model risk documentation must describe participant data quality standards.
• Model risk committee approval required before production deployment of federated model.

Human Approval Checkpoints

• Federation agreement signed by legal representatives of all participants.
• Global model promotion requires consortium governance board approval.
• DP budget reset (increasing privacy expenditure) requires participant-level DPO approval.

Governance Artefacts

10. Operational Considerations

Monitoring

SLOs

Disaster Recovery

11. Cost Considerations

Cost Drivers

Indicative Cost Range

12. Trade-Off Analysis

Option Comparison

Architectural Tensions

13. Failure Modes

14. Regulatory Considerations

15. Reference Implementations

AWS

Azure

GCP

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Emerging — Federated learning frameworks (Flower, NVIDIA FLARE) are production-ready. Cross-silo deployments are proven in healthcare and finance consortia. Cross-device FL at scale is mature (Google deployed FL for Gboard). However, regulatory frameworks for federated AI governance are still developing.

18. Revision History