[EAAPL-AGT007] Long-Running Agent

Category: Agentic AI Sub-category: Async Execution Architecture Version: 1.2 Maturity: Proven Tags: long-running, async, task-queue, heartbeat, cost-budget, partial-results, deadline-management, human-checkin Regulatory Relevance: APRA CPS 230 (Operational Resilience), ISO 22301, NIST AI RMF (MANAGE 4.1), EU AI Act (Art. 9, 14)

1. Executive Summary

The Long-Running Agent Pattern defines the architecture for AI agents that execute tasks over hours or days — due diligence analysis, large codebase refactoring, enterprise-wide data reconciliation, or extended research synthesis. These tasks cannot fit within the synchronous request-response paradigm: calling systems cannot hold a connection open for hours, LLM context windows cannot hold 48 hours of tool results, and cost controls require active monitoring rather than post-hoc billing surprises.

For CIO/CTO audiences: this pattern transforms AI agents from interactive request-responders into asynchronous workforce members — entities you assign a task to on Monday morning and receive a deliverable from by Friday, with status updates throughout and the ability to pause or redirect them at any point. It defines how to decompose multi-day tasks into manageable segments, how to monitor and control running costs, how to ensure partial results are safely preserved if the task is interrupted, and how to maintain human oversight over extended autonomous operation. The resulting architecture is what separates a toy AI demo from a production AI workforce capability.

2. Problem Statement

Business Problem

High-value knowledge work tasks take hours or days. A due diligence review of 500 contracts, a codebase-wide security audit, or a multi-source research synthesis cannot complete in seconds. If AI agents are restricted to short tasks, the most valuable automation opportunities remain out of reach.

Technical Problem

Synchronous agent execution (HTTP request/response model) is unsuitable for long tasks: connection timeouts, LLM context window limits, token cost unpredictability, and inability to inject human checkpoints all fail at scale. Context window exhaustion on multi-hour tasks is a particularly severe problem: a 100K token context window fills after 60–100 tool calls with moderate result sizes.

Symptoms of Absence

• Tasks taking longer than 30 minutes are decomposed manually by humans into shorter subtasks, negating automation benefits
• Cost surprises: a long agent task consumes 10–50× the anticipated token budget with no warning
• Partial work is lost when infrastructure restarts or LLM provider timeouts occur at hour 3 of a 5-hour task
• No mechanism for human course-correction once a long task is launched

Cost of Inaction

• High-value automation opportunities (due diligence, audit, research) remain manual
• Ad hoc workarounds (manually splitting tasks) create brittle processes that fail when task sizes vary
• Infrastructure teams field escalations about unexplained high AI inference costs from long tasks without budget controls

3. Context

When to Apply

• Expected task duration is > 30 minutes
• Task involves processing a large corpus (hundreds of documents, thousands of records)
• Human review or approval at intermediate milestones is required
• Cost predictability and budget control are required
• Partial results have value (delivering results incrementally is better than delivering nothing if the task is interrupted)

When NOT to Apply

• Tasks that complete in < 5 minutes (async overhead not justified)
• Tasks that require a synchronous response in the same user session
• Tasks with no natural decomposition into independently useful segments

Prerequisites

• EAAPL-AGT005 (Checkpoint and Recovery) — mandatory for multi-hour tasks
• Durable task queue with dead-letter handling
• Async notification infrastructure (webhooks, event bus, push notifications)
• Cost monitoring and kill switch capability
• Human management API (pause, redirect, cancel)

Industry Applicability

4. Architecture Overview

The Long-Running Agent Pattern addresses four fundamental challenges of extended autonomous execution: context window management, task decomposition and progress tracking, cost budget enforcement, and human oversight at meaningful checkpoints.

Task Decomposition and Segment Orchestration A long task is decomposed by the Task Planner into an ordered sequence of segments — bounded sub-tasks each of which can complete within the single-agent pattern's standard execution model (typically < 30 minutes, < 50K tokens). The segment plan is stored durably at task creation and is the master execution schedule. Each segment produces a partial result that is stored in the Partial Result Store. If the task is interrupted, the segment plan acts as the recovery map: completed segments are skipped; the next incomplete segment is resumed.

The segment plan is not a rigid pre-specified plan. The Task Planner can be queried to revise the remaining segment plan based on discoveries made in early segments (adaptive planning). For example, if segment 3 discovers that 200 additional documents need to be reviewed, the plan is revised to add segments 3a–3n before segment 4.

Context Window Management Across Segments Each segment executes in a fresh context window. The context for segment N includes: the original task objective, a summary of results from segments 1 through N-1 (produced by the Context Summariser component), the current segment's specific sub-objective, and the relevant tools. The summary is a lossy compression of prior results — the Task Planner specifies what information must be preserved across segment boundaries in the task plan.

This approach solves context exhaustion by design: no single segment accumulates more context than the window can hold. The cost is that inter-segment reasoning is mediated through the summary, which may lose nuance. For tasks that require tight consistency across many segments (e.g., a legal review where clause 400 must reference clause 12), the Task Planner must preserve the critical cross-references in the carry-forward summary.

Heartbeat and Progress Monitoring The long-running agent emits a heartbeat event to the monitoring system at the completion of each segment and at configurable intervals within a segment. The heartbeat includes: current segment number, total segments estimated, cost consumed so far, cost projected to completion (based on average cost per segment × remaining segments), elapsed time, and an ETA for completion. The Heartbeat Monitor triggers alerts if heartbeat events are not received within the expected interval — indicating a stuck or crashed agent.

Human Check-in Points The task plan defines human check-in points — typically at task creation (human reviews and approves the decomposition plan), at significant milestones (e.g., 50% completion), and at completion. At check-in points, the long-running agent pauses execution (using the checkpoint mechanism from EAAPL-AGT005), delivers the partial results and a progress summary to the human via a notification, and waits for human acknowledgment or instruction. The human can: approve and resume, redirect (modify the remaining segment plan), or cancel. This implements EU AI Act Art. 14 human oversight for high-risk long-running tasks.

Cost Budget and Kill Switch Before execution begins, the calling system specifies a cost budget (maximum token spend for the task). The Cost Controller monitors cumulative spend at each segment boundary. If projected cost-to-completion exceeds the budget, the Cost Controller pauses the task and notifies the human with the current partial results and a cost projection. The human can approve budget extension or accept the partial results. A hard kill switch (emergency stop) is available to humans at any time, delivering immediately available partial results and a clean task termination.

Partial Result Delivery Each completed segment's output is written to the Partial Result Store immediately upon completion. A Partial Result Aggregator compiles the running partial results into a human-consumable intermediate deliverable. The calling system can request partial results at any time via the management API, regardless of whether the task is still running. This enables progressive value delivery — a due diligence review that identifies 20 critical issues in the first 30% of documents is actionable immediately, before the full 500-document review completes.

5. Architecture Diagram

6. Components

7. Data Flow

Task Initiation

Segment Execution

Error Flow

8. Security Considerations

Long-Running Identity Tokens

• Agent authentication tokens for accessing external tools must not expire during a multi-hour task
• Implement token refresh within the segment worker; use long-lived service account credentials, not short-lived user tokens
• Dynamic secrets (auto-rotating) must have rotation intervals longer than the maximum task duration

Data Retention of In-Progress Tasks

• Partial results contain sensitive intermediate data; they must be encrypted and access-controlled
• Partial results for cancelled tasks must be cleaned up according to the data retention policy
• Cross-segment context summaries may contain PII extracted from processed documents; apply the same classification and retention rules as the source data

OWASP LLM Top 10

9. Governance Considerations

Human Oversight for Long-Running Tasks

• All long-running tasks must have a named human owner who is notified of check-in points and receives partial results
• Tasks exceeding a configured duration (default: 4 hours) automatically escalate to the human owner's manager
• No task may run longer than 72 hours without a human re-approval of the segment plan

Governance Artefacts

10. Operational Considerations

SLOs

Capacity

• Segment workers are stateless containers; horizontal scaling is bounded by LLM provider quota and tool API rate limits
• Estimate: 1 worker per 5 concurrent segments for 30-minute segments; scale up to 1 worker per concurrent segment for 5-minute segments
• Partial result storage grows with task count × average output size; provision for 30-day retention of all partial results

11. Cost Considerations

Cost Drivers

Indicative Cost Range (USD)

12. Trade-Off Analysis

Task Decomposition Options

Architectural Tensions

13. Failure Modes

14. Regulatory Considerations

APRA CPS 230

• Long-running agents supporting material business services require RTO/RPO; the checkpoint + segmentation architecture enables sub-segment RTO
• Multi-hour tasks interacting with critical systems require operational risk assessment and business impact analysis

EU AI Act

• Art. 14 (Human Oversight): mandatory human check-ins at task creation and significant milestones implement the "meaningful human oversight" requirement for high-risk long-running agents
• For high-risk AI systems: the complete task execution log (all segments, costs, human decisions, partial results) is a required audit artefact

15. Reference Implementations

AWS

Azure

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Proven

18. Revision History