Event-Driven AI Agent Fleets: SQS, EventBridge, and Step Functions Patterns for Async Agent Work

Every platform team building AI agents hits the same wall at roughly the same size: the moment an agent chain takes longer than a client timeout, request/response stops being an architecture and becomes an incident generator. The fix is not a faster model. It is moving the fleet onto event-driven infrastructure — queues as the work ledger, an event bus as the nervous system, and a durable state machine as the thing that remembers where a twenty-minute conversation is stuck when a worker dies mid-step.

This guide is the full mechanical treatment: SQS visibility and redrive behavior, EventBridge retry policies and quotas, the Step Functions waitForTaskToken callback pattern, Bedrock AgentCore sessions for long-lived agent state, KEDA autoscaling on queue depth, and the multi-model routing layer that sits between every worker and its inference. Every quota, default, and API parameter cited here was verified against the current AWS documentation on November 11, 2026 — where AWS changed a long-standing behavior (EventBridge retry defaults, SQS message size, PutEvents entry limits), the current value is the one used, not the one blog posts from 2023 still repeat.

AWS's own architecture guidance has converged on this pattern from three directions in 2025–2026: the Machine Learning blog's async agents write-up (event-driven choreography and the agent broker pattern over EventBridge and SQS), the DevOps blog's Strands Arbiter pattern (SQS tool-call dispatch with EventBridge completion events), and the Well-Architected Agentic AI Lens, whose first agent-reliability best practice is blunt: direct agent-to-agent calls couple failure modes, so put a messaging layer with persistence, retry, and dead-letter handling between every agent pair. This guide turns that guidance into runnable infrastructure.

Why request/response breaks at fleet scale

Three failure modes compound when you scale agents synchronously:

• Latency ceiling. A multi-step agent chain — retrieval, tool calls, reasoning passes, validation — routinely runs for minutes. The longest-running AWS primitives cap well below that: Lambda functions hard-stop at 15 minutes, Step Functions Express workflows stop at 5 minutes, and API Gateway's integration timeout is 29 seconds. A synchronous design has already failed before the model finishes thinking.
• Retry storms that re-pay for everything. A client timeout on step 9 of 12 does not restart at step 9 — it restarts at step 0 and re-buys the tokens. Synchronous retries on LLM chains are the most expensive retry pattern in the building, because each retry re-executes paid inference with no checkpoint to resume from.
• Availability multiplication. The Agentic AI Lens does the arithmetic plainly: five agents with four synchronous dependencies each produce an availability product that drops below any single agent's SLA. Coupling failure domains is the point of the exercise in event-driven design — and the reason the Lens makes a durable messaging layer the first reliability best practice rather than a nice-to-have.

The event-driven answer inverts the control loop. Instead of a caller holding a connection open while an agent works, the caller publishes an event and forgets. A queue holds the work durably, a worker takes it at its own pace, and a completion event flows back through the bus when the agent finishes. Failures stop propagating — they queue. And every component in the chain has an explicit, configurable answer to "what happens when this fails," which is what request/response architectures never have.

The reference topology

The topology below is the shape that keeps recurring across AWS's own agent reference architectures — the ML blog's broker pattern, the Strands Arbiter, and the Agentic AI Lens's messaging-layer best practice — normalized into one diagram. Three planes: the control plane (event bus, orchestrator), the work plane (queues, workers), and the state plane (checkpoint and memory stores).

                       ┌──────────────────────────────────────────────────────┐
                       │                    CONTROL PLANE                     │
                       │                                                      │
  ┌──────────┐   ┌─────▼─────┐   ┌──────────────┐   ┌───────────────────────┐  │
  │ API GW / │   │ EventBridge│   │   Rules:     │   │  Step Functions        │  │
  │ app      ├──►│ custom bus ├──►│ content-based├──►│  Standard orchestrator │  │
  │ ingress  │   │ (default    │   │  pattern    │   │  (durable, 1-yr runs)  │  │
  └──────────┘   │  or custom) │   │  matching   │   └──────────┬────────────┘  │
                 └──────┬─────┘   └──────────────┘              │ .waitForTaskToken
                        │                                        │ TaskToken rides the
                        │  PutEvents (batched, ≤10 entries      │ message payload
                        │  classic / ≤100 on V2 buses)           ▼
                       │                              ┌────────────────────────┐
  ┌────────────────────▼─────────────────────────────┤   WORK PLANE           │
  │                                                  │                        │
  │   ┌──────────────┐    ┌──────────────┐           │  ┌──────────────────┐  │
  │   │ agent-class  │    │ agent-class  │           │  │  orchestrator    │  │
  │   │ ingest queue │    │ tool queue   │           │  │  callback queue  │  │
  │   │ (std/FIFO)   │    │ (per tool)   │           │  └──────────────────┘  │
  │   │  + DLQ       │    │  + DLQ       │           │                        │
  │   └──────┬───────┘    └──────┬───────┘           │                        │
  │          │                   │                   │                        │
  │   ┌──────▼───────────────────▼──────┐             │                        │
  │   │ WORKERS                         │             │                        │
  │   │  • Lambda (≤15 min steps)       │             │                        │
  │   │  • EKS pods w/ KEDA (long steps)│             │                        │
  │   │  • Bedrock AgentCore (8h-14d)   │             │                        │
  │   │     every worker = LLM router ──┼─────────────┘                        │
  │   │     (Bedrock Converse / prompt  │
  │   │      routing / LiteLLM fallback)│
  │   └──────┬───────────────────────────┘
  │          │
  └──────────┼──────────────────────────────────────────────────────────────────
             │ completion events + SendTaskSuccess
  ┌──────────▼──────────────────────────────────────────────────────┐
  │ STATE PLANE                                                      │
  │  DynamoDB: workflow tracking, idempotency keys (conditional puts)│
  │  S3: claim-check payloads (>1 MiB SQS / >256 KiB SFN)            │
  │  Bedrock AgentCore Memory: session + long-term agent memory      │
  │  LangGraph checkpoints (Postgres 3.2.0) where frameworks need   │
  └──────────┬──────────────────────────────────────────────────────┘
             │
             ▼  results re-published to the bus → next agent class → loop

Two loops run on this topology and they are easy to confuse in review: the message loop (bus → queue → worker → bus) carries individual steps, while the token loop (state machine → queue → worker → SendTaskSuccess) carries the workflow's position. The message loop is at-least-once and unordered by default; the token loop is exactly-once per execution step because Step Functions persists state at every transition. Getting these backwards — trying to infer workflow position from message delivery order — is the single most common design error in event-driven agent systems.

What follows walks each plane in dependency order: the event contract first (because everything downstream parses it), then queue mechanics, then the state machinery, then routing, idempotency, and scaling — and closes with where this pattern costs more than people expect and who should not use it at all.

The event contract: EventBridge as the nervous system

EventBridge is not a queue. It is a router with durable-ish semantics: matched events are delivered to targets (up to 5 per rule), retried on failure, and dead-lettered when retries exhaust. The contract between agents is the event envelope, so version it explicitly and put trace state in detail — everything downstream will thank you when you replay it later.

{
  "version": "0",
  "id": "6c4a1c1e-0b7f-4f0e-9d1a-3b2c5e6f7a8b",
  "detail-type": "agent.step.requested",
  "source": "fleet.coordinator",
  "time": "2026-11-11T09:15:22Z",
  "region": "us-east-1",
  "resources": [],
  "detail": {
    "trace_id": "conv-8842-step-7",
    "conversation_id": "8842",
    "step": 7,
    "agent_class": "researcher",
    "model_policy": "claude-haiku-then-sonnet",
    "payload_ref": {
      "type": "s3",
      "bucket": "fleet-claimcheck-use1",
      "key": "conv/8842/step-07/input.json"
    },
    "ttl_seconds": 86400
  }
}

Two mechanics of PutEvents shape the envelope on a classic (non-V2) bus. First, batching: a classic-bus request carries up to 10 entries, and the total request size must stay under 1 MB (1,048,576 bytes) — a single event may use the full 1 MB when it is the only entry. AWS's entry-size arithmetic: the Time parameter costs 14 bytes, Source, DetailType, Detail, and each Resources entry cost their UTF-8 byte length, and the rendered event is always slightly larger than the entry. Second, when an entry exceeds the limit, the documented answer is the claim-check pattern: upload the payload to S3 and put the object URL in the entry. An agent fleet should treat that as the default for anything beyond small control payloads, because the next bottleneck downstream (Step Functions' 256 KiB state payload cap) is far stricter than the bus. On the newer EventsV2 custom buses the per-request entry limit is 1–100 via PutEvents/PutRawEvents, at the cost of a very different subscriber model — covered in the production realities section.

Read the invocations line carefully: past the throttle, EventBridge delays delivery rather than rejecting it. That is benign at low rates and a latency landmine at fleet scale — a burst of agent completion events can silently add seconds to fan-out delivery, which matters when a downstream rule targets a 5-minute Express workflow.

Routing itself is content-based pattern matching over the envelope. One rule per agent class keeps fan-out explicit and gives each target its own retry policy and DLQ:

{
  "detail-type": ["agent.step.requested"],
  "source": ["fleet.coordinator"],
  "detail": {
    "agent_class": ["researcher", "coder"]
  }
}

When you need point-to-point movement with filtering and enrichment but no fan-out, EventBridge Pipes is the designated primitive: source, optional filter, optional enrichment step (a Lambda or API call), target — no polling code and no rules engine. Pipes reduce the integration glue; buses remain the tool for many-to-many routing between agents. The agent broker pattern from the AWS ML blog uses exactly this split: a single broker Lambda receives every inbound event, queries agent capability metadata (their example uses AWS AppConfig), asks the model via the Bedrock Converse API which agent tools should handle the message, and drops the work onto per-agent SQS ingress queues — throughput decoupled from broker latency, and adding a new agent means adding a queue and a capability record, not rewiring callers.

Queue mechanics: SQS as the work ledger

Everything about SQS in an agent fleet derives from one behavior: receiving a message does not remove it — it hides it. A receive makes the message invisible for the visibility timeout (default 30 seconds, configurable up to 12 hours). The worker must delete the message when done; if it dies, the message reappears and someone else takes it. That single rule is the entire durability story of the work plane, and its failure modes (duplicates, re-entrancy, poison messages) are all downstream consequences.

t=0    receive  ──►  message hidden 12 min (visibility timeout)
t=0..T worker processes agent step (LLM chain: 45–600 s typical)
t=T    delete   ──►  message gone. Clean path.

Case B — worker dies at t=120 s:
t=120  worker crashes (node loss, OOM kill)
t=720  visibility expires ──► message visible again
t=725  worker #2 receives it ──► step re-executes from its last checkpoint
       (idempotency table absorbs any partially-committed side effects)

Case C — worker alive but slow (LLM chain > visibility):
t=600  visibility expires while worker #1 still running
t=601  worker #2 receives the SAME message ──► two executions of one step
       Rule: visibility_timeout >= worst-case step time × safety margin,
       or worker extends visibility per-message (ChangeMessageVisibility)
       on heartbeat. Never rely on "steps are usually fast."

The hard numbers that shape worker design, verified against the current message-quotas page:

• Retention: 4 days by default, 1 minute to 14 days max. A DLQ backlog older than your source queue's retention is gone — size retention before you size anything else.
• Message size: 1 byte to 1 MiB (1,048,576 bytes) — up from the 256 KiB ceiling older articles still cite. Larger payloads go through the Extended Client Library to S3 (up to 2 GB), which is just a claim-check with a helper class.
• Batches: 10 messages per SendMessageBatch, 10 metadata attributes per message, message timer (per-message delay) 0–15 minutes.
• In-flight cap: roughly 120,000 messages received-but-not-deleted per standard queue. Cross it and short polling returns OverLimit; long polling silently returns nothing until the count drops. Long-running agent steps push in-flight numbers up by design — a fleet of 10-minute steps at 1,000 concurrent messages is already 1% of the cap, fine; 100,000 concurrent steps is not.
• Throughput: standard queues are effectively unbounded on API calls per second. FIFO queues without high-throughput mode do 300 TPS per API action (3,000 messages/s with 10-message batches); with high-throughput mode the ceiling becomes 70,000 TPS / 700,000 messages/s batched in us-east-1, us-west-2, and eu-west-1, and 19,000–2,400 TPS in other regions. Order costs throughput; buy only as much as sessions need.

FIFO, message groups, and the head-of-line tradeoff

FIFO queues deliver the two things agent conversations sometimes genuinely need — per-session ordering and ingress dedup — at prices that are easy to misjudge. Ordering is scoped to a message group (MessageGroupId, required, 128 chars max): one conversation ID per group gives you per-conversation ordering with cross-conversation parallelism. Dedup runs on a 5-minute window: a repeated MessageDeduplicationId (or a SHA-256 of the body with content-based dedup enabled) is acknowledged but not delivered, and SQS keeps tracking the ID even after the message was received and deleted — which makes client-side retry-after-timeout safe at the ingress, and only at the ingress. Everything after the queue is still at-least-once.

The cost is head-of-line blocking. While a message in group 8842 is in flight, the next message in that group is not available until the in-flight message is deleted or its visibility expires. A stuck 10-minute LLM step stalls its entire conversation's queue lane; a poison message that keeps failing stalls it indefinitely. Standard queues now accept MessageGroupId too (fair queues) without ordering guarantees — useful for weighted draining, not for correctness. The engineering decision rule: FIFO per-session lanes only when the consumer of the session is itself order-sensitive (state machine input streams, audit-ordered writes), never "because agents feel ordered."

Dead-letter queues and redrive

DLQ configuration is one attribute: a redrive policy naming the DLQ and maxReceiveCount. When a message has been received that many times without deletion, SQS moves it. Two documented behaviors bite in production: on standard queues with maxReceiveCount > 3, a message received three or more times without deletion is moved to the back of the queue — so ApproximateAgeOfOldestMessage then reflects the next message under the threshold, and your age-based alarms under-report exactly when the queue is healthiest-looking. And the redrive allow policy controls which source queues may use a given DLQ (up to 10 by ARN, or allowAll/denyAll) — set it, or any queue in the account can dump into your triage queue. For getting messages back, use the managed dead-letter queue redrive (StartMessageMoveTask) rather than home-rolled re-senders.

# --- Work plane: researcher queue + DLQ -------------------------------
resource "aws_sqs_queue" "researcher_dlq" {
  name                       = "agent-researcher-dlq"
  message_retention_seconds  = 1209600          # 14 days — DLQ outlives source
  visibility_timeout_seconds = 300
  redrive_allow_policy = jsonencode({
    redrivePermission = "byQueue",
    sourceQueueArns   = [aws_sqs_queue.researcher.arn]
  })
}

resource "aws_sqs_queue" "researcher" {
  name                       = "agent-researcher"
  visibility_timeout_seconds = 900              # > worst-case researcher step
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.researcher_dlq.arn
    maxReceiveCount     = 4                     # 4 strikes → triage, not retry hell
  })
  long_polling_wait_time_seconds = 20           # never short-poll a worker fleet
}

# --- Control plane: rule with per-target retry + DLQ ------------------
resource "aws_cloudwatch_event_rule" "agent_step_requested" {
  name           = "agent-step-requested-researcher"
  event_bus_name = aws_cloudwatch_event_bus.fleet.arn
  event_pattern = jsonencode({
    detail-type = ["agent.step.requested"]
    source      = ["fleet.coordinator"]
    detail = { agent_class = ["researcher"] }
  })
}

resource "aws_cloudwatch_event_target" "researcher_queue" {
  rule           = aws_cloudwatch_event_rule.agent_step_requested.name
  event_bus_name = aws_cloudwatch_event_bus.fleet.arn
  arn            = aws_sqs_queue.researcher.arn
  retry_policy {
    maximum_retry_attempts       = 24    # bounded retries; 185 is a ceiling, not advice
    maximum_event_age_in_seconds = 3600  # 1 h, not 24 h — see production realities
  }
  dead_letter_config {
    arn = aws_sqs_queue.agent_target_dlq.arn
  }
}

Note the retry policy values. The EventBridge RetryPolicy range is 0–185 attempts and 60–86,400 seconds, with classic-bus defaults of 185 attempts over 24 hours with exponential backoff and jitter. This guide deliberately sets 24 attempts over 1 hour instead: a target outage masked by 24 hours of invisible retries is a worse production posture than a fast DLQ drop you can alert on — you trade "eventually delivered, late and duplicative" for "loud and triageable." That is a judgment call, and the production realities section makes the case for it.

The Lambda edge: event source mappings for short agent steps

Lambda is the right worker for agent steps that fit inside its 15-minute wall: prompt assembly, small tool calls, classification, guardrail checks. The SQS event source mapping (ESM) is the polling machinery that turns a queue into an invocation stream, and its mechanics decide your duplicate rate. The ESM polls up to 10 messages per invocation by default and can buffer for up to 5 minutes (MaximumBatchingWindowInSeconds, 0–300) before invoking — it keeps polling until the window expires, the batch hits its size cap, or the invocation payload limit is reached. For agent work, keep the batching window at or near zero: agent steps are too slow and too expensive to coalesce, and a bigger window mostly increases the chance that a batch contains one poison message alongside nine good ones.

That last point is the one that costs teams money. By default, if a batch invocation fails, every message in the batch returns to visibility and will be redelivered — including the ones your function already processed successfully. The fix is the partial-batch response protocol: configure the ESM with FunctionResponseTypes: ["ReportBatchItemFailures"], and return only the failed sequence IDs. Lambda deletes the successful ones and only re-queues the failures. The documentation is explicit that the ESM processes each event at least once and duplicates can occur — idempotency remains your problem, not the ESM's.

resource "aws_lambda_event_source_mapping" "researcher" {
  event_source_arn        = aws_sqs_queue.researcher.arn
  function_name           = aws_lambda_function.agent_step.arn
  batch_size              = 5
  maximum_batching_window = 1          # seconds — do not coalesce slow agent steps
  enabled                 = true

  function_response_types = ["ReportBatchItemFailures"]

  # visibility must cover: batch × per-step worst case, or duplicates on overrun
  visibility_timeout = 900
}

import json

def handler(event, context):
    failures = []
    for record in event["Records"]:
        body = json.loads(record["body"])
        try:
            result = process_step(body)          # must be idempotent: at-least-once
        except Exception:
            failures.append({"itemIdentifier": record["messageId"]})
    return {"batchItemFailures": failures}

Two newer ESM facts matter at fleet scale. Throughput: the default SQS ESM scales conservatively; provisioned mode lets you reserve pollers explicitly, and AWS's numbers are aggressive — it scales up to 3× faster (about 1,000 additional concurrent invokes per minute) and supports roughly 80× higher concurrency, up to 100,000 concurrent invokes. AWS's poller-sizing heuristic: min_pollers = ceil(10 / average_duration_seconds) × batch_size, bounded by min(100, 10 × batch_size) — with a 1-second average step duration and batch size 10, 1,000 events/s needs roughly 100 minimum pollers. And visibility: the ESM honors the queue's visibility timeout, so the guidance from the previous section applies verbatim — set visibility above your worst-case batch duration, not the average step.

When a step can exceed 15 minutes — deep research agents, browser-driven tool runs, long code refactors — stop contorting Lambda. Route those agent classes to their own queue and run the worker as a pod (EKS, scaled by KEDA) or as a Bedrock AgentCore session. The pattern does not change; only the worker runtime does.

Long-running state: three mechanisms that don't lie

The defining property of a durable agent architecture is that workflow position survives worker death. There are three production mechanisms for that on AWS, and they compose rather than compete:

The architectural insight most teams miss: a Step Functions execution is already a checkpointing system. The Agentic AI Lens is explicit that Step Functions persists execution state at every step transition, so recovery starts at the last completed step instead of re-executing the workflow — which is exactly the property teams spend weeks building by hand in DynamoDB. Use it as the spine, and let SQS carry the work between states.

The callback pattern: waitForTaskToken in depth

The bridge between the state machine and the queue world is the task-token callback. A Standard-workflow Task state using the .waitForTaskToken integration pattern pauses the execution — for up to the one-year service quota — until someone calls SendTaskSuccess or SendTaskFailure with the token. The canonical wiring sends the token to SQS as part of the message payload; the agent worker that eventually receives the message does the work, then closes the loop:

 Step Functions                SQS                        Agent worker (any runtime)
 ────────────────              ─────                      ─────────────────────────
 Task: sendMessage
   .waitForTaskToken
        │
        ├─► SendMessage(body=task,
        │              TaskToken=$$.Task.Token)  ──►  message waits in queue
        │                                                    │
   [execution PAUSED]                                         ├─► receive + process step
   (no compute billed for                                     │    (LLM chain, tools)
    the pause)                                                │
        │                                                    ├─► SendTaskSuccess(token,
        │                                                        output)
        ◄────────────────────────────────────────────────────┘
   [execution resumes at next state]

 Failure modes to design for:
  1. Worker dies holding the token → execution stays parked.
     Fix: TimeoutSeconds + HeartbeatSeconds on the Task state.
  2. Task times out → SFN generates a NEW random token; the old
     worker's SendTaskSuccess on the stale token fails. Handle it.
  3. Heartbeat missed → task fails with States.Timeout even though
     the worker is alive. Heartbeat from a long LLM call only if the
     call supports cancellation — otherwise set the heartbeat window
     above the longest model call in the step.

{
  "Comment": "Agent step with durable callback",
  "StartAt": "DispatchStep",
  "States": {
    "DispatchStep": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
      "Parameters": {
        "QueueUrl": "https://sqs.us-east-1.amazonaws.com/111122223333/agent-researcher",
        "MessageBody": {
          "conversation_id.$": "$.conversation_id",
          "step.$": "$.step",
          "payload_ref.$": "$.payload_ref",
          "TaskToken.$": "$$.Task.Token"
        },
        "MessageGroupId.$": "$.conversation_id"
      },
      "TimeoutSeconds": 3600,
      "HeartbeatSeconds": 900,
      "Next": "AssembleResult"
    },
    "AssembleResult": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "agent-assemble-step-output",
        "Payload.$": "$"
      },
      "Retry": [
        {
          "ErrorEquals": ["Lambda.TooManyRequestsException", "Lambda.ServiceException"],
          "IntervalSeconds": 2,
          "MaxAttempts": 6,
          "BackoffRate": 2.0
        }
      ],
      "End": true
    }
  }
}

The mechanics worth internalizing, from the connect-to-resource documentation: the token is addressed inside Parameters as "TaskToken.$": "$$.Task.Token" — the $$. prefix reads the execution's context object, not your state input. The optimized SQS integration appends .waitForTaskToken to the resource ARN. On task timeout, Step Functions generates a new random token for the retried state — so a worker that took too long cannot accidentally close a state it no longer owns; its stale SendTaskSuccess fails, and the retry dispatches fresh work. Throttling on the callback APIs is generous but real: SendTaskSuccess/SendTaskFailure/SendTaskHeartbeat each get a 3,000-request token bucket refilling at 500/s in the big-3 regions (1,500/300 elsewhere) — a fleet calling back thousands of times per second will hit it, so batch or shard callbacks by execution if you are at that scale.

For steps that fan out — a research agent spawning 200 sub-queries — the Distributed Map state is the right shape: it drives up to 10,000 concurrent child executions from a single MapRun (non-adjustable), reads up to 100 million items, takes up to 10 GB of input data, and supports partial results and redrive. The payload cap still binds everywhere: 256 KiB per state input/output, which is why the claim-check pattern (S3 object, reference in state) is not optional at map scale — it is the only way to move large intermediate results between states.

Bedrock AgentCore: sessions for stateful agents that outlive any single step

Step Functions remembers position; AgentCore Runtime remembers context. Each session gets a dedicated microVM with isolated CPU, memory, and filesystem; context persists between invocations of the same session, and after session completion the microVM is terminated and memory sanitized — the isolation story is per-session, not per-agent. The lifecycle is explicit in the current docs: sessions run up to 8 hours per lifecycle on microVMs (up to 14 days on Instances, which can host multiple collaborating agents and GPUs), idle sessions stop after 15 minutes by default, and a stopped session resumes on the next invocation with the same runtimeSessionId — re-provisioned compute, same session. Files survive stop/resume only if you configure session storage; durable knowledge goes to the AgentCore Memory service, which scopes memory by actor and session and extracts long-term memory via configurable strategies (semantic, summarization, user preference, episodic, custom).

The fit with this topology is direct: the queue carries the step, the session carries the conversation. A completion event includes the runtimeSessionId; the next worker invocation with the same ID resumes the same agent context without re-hydrating history from a database. Quota reality: 5,000 active session workloads per account in us-east-1/us-west-2 (2,500 elsewhere), 2 vCPU/8 GB per session non-adjustable, 15-minute synchronous request timeout, 100 MB payload, and a 150,000 tokens-per-minute cap on built-in long-term memory extraction (50,000 per session) — memory extraction is a metered pipeline, not a free side effect, and it is the hidden cost center of "just let the agent remember everything."

If your agents are framework-shaped, the same session mechanics work underneath LangGraph 1.2.20 or Strands 1.64.0 (AgentCore supports both plus CrewAI, speaks MCP and A2A, and is model-agnostic) — with LangGraph's own durable state living in a checkpointer such as Postgres 3.2.0 when you want the graph thread state outside AWS entirely. What you must not do is keep the workflow position in the agent's head: sessions are context, not transactions. Position belongs to the state machine.

Multi-model routing: the other event loop

Every worker in the fleet makes two decisions per step: what to do (agent logic) and which model to ask (routing policy). Treating the second as a one-line model ID is how fleets end up hard-coupled to a single model's outage and price changes. Three routing mechanisms cover the production spectrum, and they layer:

Bedrock Converse with tool-use loops is the API foundation — the AWS async-agents pattern drives agent reasoning through the Converse API with tool definitions, which is what lets the broker Lambda ask a model which agent tools should handle this message and then dispatch to SQS per tool call. Because it is a single API shape across models, swapping providers is a routing decision, not a rewrite.

Bedrock intelligent prompt routing (generally available since April 2025) adds quality-vs-cost routing inside a model family: a configured prompt router takes exactly two models from the same family, a responseQualityDifference threshold (0–100), and a fallback model that anchors the decision — the router only promotes to the larger model when its predicted response quality clears the threshold, otherwise the fallback serves the request. The documented limits matter more than the feature: routing is optimized for English prompts, it cannot fold your application-specific performance data into its decisions, and it may not be optimal for specialized use cases. It is a within-family cost lever — cross-family strategy still belongs to your own routing layer.

Cross-region inference profiles (us., apac. model ID prefixes) spread inference across a geography's regions, and a Global profile on Claude Sonnet 4 routes across all commercial regions — the capacity-availability layer that a fleet under burst load actually needs. Verify destination regions with GetInferenceProfile before relying on a profile if data residency binds you; the docs are explicit that profile prefixes alone do not tell you where traffic lands.

For cross-provider fleets, the routing layer is a gateway — this site has covered the gateway pattern in depth (the AI gateway pattern and LLM router gateways), and the configuration discipline from that work carries verbatim: explicit cross-group fallbacks, bounded in-group retries, and cooldowns on failed deployments. A previously verified hands-on finding from the gateway testing is worth repeating here because it maps exactly onto fleet failure modes: with LiteLLM, in-group retries alone leak errors when a deployment is hard-down (28/30 requests succeeded, 2 surfaced failures), while explicit cross-group fallbacks with allowed_fails gave 30/30 — the difference between "retry harder inside the failure domain" and "leave it." Current versions verified for this guide: LiteLLM 1.109.0, Strands SDK 1.64.0, LangGraph 1.2.20.

model_list:
  - model_name: agent-primary          # the name agents request
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-4-...   # cross-region profile
      max_tokens: 4096
  - model_name: agent-primary
    litellm_params:
      model: bedrock/us.anthropic.claude-haiku-4-...
      max_tokens: 4096

router_settings:
  num_retries: 2              # bounded in-group retries — do not hide outages
  timeout: 120                # per LLM call, > worst single-call latency
  allowed_fails: 2            # strikes before a deployment goes on cooldown
  cooldown_time: 60           # seconds the bad deployment is benched

litellm_settings:
  drop_params: true

fallbacks:
  - agent-primary: ["agent-overflow"]   # cross-group: leave the failure domain

The routing layer must participate in the backpressure story, not just the failover story: when the primary provider rate-limits the whole fleet, the fallback model family inherits the entire load in one retry wave. Gate the fallback with the same concurrency semaphores the next section applies to workers — otherwise the router converts a primary outage into a fallback outage about ninety seconds later.

Idempotency: at-least-once is a physics problem

Accept it now: every hop in this architecture can deliver your work more than once. The Lambda ESM documentation states it outright — at-least-once processing, duplicates possible, make your code idempotent. FIFO deduplication covers a 5-minute ingress window and only on the producer side. Visibility expiry duplicates slow steps. Replay re-delivers by design. The system that actually guarantees single-processing is yours, and it is one DynamoDB table with conditional writes:

import boto3, json, time

ddb = boto3.resource("dynamodb").Table("agent-idempotency")

def process_step_once(msg):
    # key = stable id of the WORK: event id + consumer, not the delivery attempt
    key = {"pk": msg["detail"]["trace_id"], "sk": "researcher"}

    try:
        ddb.put_item(
            Item={
                "pk": key["pk"], "sk": key["sk"],
                "status": "running",
                "lease_until": int(time.time()) + 900,   # step lease, one visibility cycle
                "expires_at": int(time.time()) + 86400,  # idempotency window
            },
            ConditionExpression="attribute_not_exists(pk)"
        )
    except ddb.meta.client.exceptions.ConditionalCheckFailedException:
        return load_existing_result(key)   # duplicate delivery → return prior result

    result = run_agent_step(msg)            # exactly one execution reaches here
    ddb.update_item(
        Key=key,
        UpdateExpression="SET #s = :done, result = :r",
        ConditionExpression="lease_until > :now",
        ExpressionAttributeNames={"#s": "status"},
        ExpressionAttributeValues={":done": "done", ":r": json.dumps(result), ":now": int(time.time())},
    )
    return result

Design notes that separate working idempotency from a placebo: the key must identify the work, not the delivery (event ID plus consumer name, never SQS messageId, which differs per redelivery); the lease handles the crash case (a "running" row whose lease expired can be reclaimed safely); and the idempotency window must exceed the EventBridge retry window plus the queue retention — a record that expires while retries still reference it is a duplicate you approved. If you run on Lambda, the Powertools for AWS idempotency utility implements this pattern with less ceremony (the same library family whose Batch Processor utility wraps the partial-batch protocol); use it rather than hand-rolling a third copy of this table.

Backpressure and autoscaling: scale on queue depth, cap on model capacity

Scaling agent fleets has a property stateless services do not: the work unit runs for minutes, so scaling signals that are fine for HTTP services (queue length, CPU) under-react or over-react in ways that matter. The standard EKS-side answer is KEDA (v2.21.0), whose SQS scaler counts messages in the queue and drives pod autoscaling directly from that signal:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: agent-queue-auth
  namespace: agents
spec:
  podIdentity:
    provider: aws        # EKS pod identity — no static keys in the cluster
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: researcher-worker
  namespace: agents
spec:
  scaleTargetRef:
    name: researcher-worker
  minReplicaCount: 2
  maxReplicaCount: 60
  pollingInterval: 30          # queue-depth checks; not a latency knob
  cooldownPeriod: 300          # slow scale-down: steps run for minutes
  triggers:
    - type: aws-sqs-queue
      authenticationRef:
        name: agent-queue-auth
      metadata:
        queueURL: https://sqs.us-east-1.amazonaws.com/111122223333/agent-researcher
        awsRegion: us-east-1
        queueLength: "10"       # target: each pod owns ~10 queued messages
        activationQueueLength: "1"

Three scaler behaviors are documented and directly shape fleet behavior. The default count includes in-flight messages (ApproximateNumberOfMessages + ApproximateNumberOfMessagesNotVisible; disable with scaleOnInFlight: "false"), which is usually what you want for long steps — a pod working a 10-minute step holds NotVisible inventory, and counting it prevents the scaler from thinking the queue is empty while pods are saturated. queueLength is a messages-per-replica target: at 30 queued messages with queueLength: "10" the scaler wants 3 pods — set it from your measured per-pod concurrency, not from hope. And activationQueueLength gates scale-from-zero, so a trickle of test traffic does not keep a fleet of GPU-adjacent pods warm overnight.

Two caps must exist above the autoscaler or it will happily scale into a wall: provider concurrency (Bedrock account and regional quotas; a fleet that scales pods past its model quota converts queue pressure into throttling errors — put the semaphore in the router or a DynamoDB lease counter, not in a pod count) and downstream blast radius (maxReplicaCount sized to what your tool targets — internal APIs, browsers, databases — tolerate, because every replica is a multiplier on them). On the Lambda side, the equivalent cap is the function's reserved concurrency acting as a hard ceiling, and the provisioned-mode ESM pollers from the earlier section acting as the accelerator. In both runtimes, the queue-depth alarm that actually predicts trouble is ApproximateAgeOfOldestMessage trending up — depth alone says nothing about whether workers are keeping pace with steps that take minutes.

# steps completed per worker, 5m window
sum by (agent_class) (rate(agent_step_duration_seconds_count[5m]))

# p95 step latency — watch it against the visibility timeout
histogram_quantile(0.95,
  sum by (le, agent_class) (rate(agent_step_duration_seconds_bucket[5m])))

# stall canary: zero completions while queue depth > 0 for 10 minutes
sum(rate(agent_step_duration_seconds_count[10m])) == 0
  and on() sum(sqs_queue_depth{queue="agent-researcher"}) > 0

Failure taxonomy and DLQ triage

Every async hop fails differently, and conflating the failure modes is how teams end up re-driving the whole pipeline for what was a missing IAM statement. The taxonomy, in the order messages flow:

Triage is a queue-drain discipline, not an ad-hoc SSH session. The DLQ record carries the diagnosis as message attributes — ERROR_CODE, ERROR_MESSAGE, EXHAUSTED_RETRY_CONDITION, RETRY_ATTEMPTS, plus RULE_ARN and TARGET_ARN — so the first pass is grouping, not reading:

#!/usr/bin/env bash
# First question is never "what's in the message" — it is "what class of failure."
aws sqs receive-message \
  --queue-url "$QUEUE_DLQ_URL" \
  --max-number-of-messages 10 \
  --message-attribute-names All \
  --wait-time-seconds 5 \
| jq -r '.Messages[] | [ .MessageId, (.MessageAttributes.ERROR_CODE.StringValue // "?"), (.MessageAttributes.EXHAUSTED_RETRY_CONDITION.StringValue // "?") ] | @tsv' \
| sort | awk 'NF{count[$2]++} END {for (e in count) print count[e], e}'

# NO_PERMISSIONS / NO_RESOURCE  → fix IAM or the target, then redrive:
#   aws sqs start-message-move-task --source-arn "$DLQ_ARN" --destination-arn "$SOURCE_ARN"
# THROTTLING                     → your retry policy retried a rate limit; widen
#                                 backoff or raise quota, then redrive
# ERROR_FROM_TARGET / TIMEOUT    → real per-message failures: read the payload,
#                                 fix the worker, redrive in a canary batch first

Two EventBridge DLQ specifics from the documentation are easy to violate from muscle memory: rule-target DLQs must be standard queues — you cannot point a rule's DLQ at a FIFO queue — and the DLQ must live in the same region as the rule. The InvocationsSentToDLQ and InvocationsFailedToBeSentToDLQ metrics are the paired alarms: the first tells you triage has work, the second tells you your safety net itself is failing (a DLQ whose own permissions are broken drops events with nothing but that metric as the trace).

Replay is a feature you must survive

EventBridge archives change the failure story: any event can come back. Archives are pattern-filtered, retain events indefinitely by default, and replay is a first-class operation — which converts your idempotency layer from best practice to load-bearing infrastructure. The mechanics that make replay survivable are documented: replayed events carry a replay-name field, and EventBridge installs a managed rule that keeps replayed events out of the archive again (no archive recursion), but every other rule on the bus matches replayed events exactly as it matched them the first time. The docs also recommend waiting about 10 minutes after ingestion before replaying, because archiving lags ingestion slightly. The operational rule that follows: never replay to a bus whose rules now include non-idempotent targets (a webhook that charges a card, an email send, an external write) — replay is a re-execution of history, and your DynamoDB idempotency table is the only thing standing between "recovery" and "duplicate side effects at archive scale."

The same pattern on GCP and Azure

The topology is cloud-portable; the mechanics differ in ways that change failure behavior. The mapping that matters for agent fleets:

One asymmetry deserves emphasis because it inverts AWS instincts: Pub/Sub's ordered delivery re-delivers acknowledged successors when any message in a key needs redelivery, so an ordered agent-session lane on GCP duplicates more aggressively than SQS FIFO, not less — idempotency is at least as non-negotiable there. And Event Grid's fixed retry schedule with a 24-hour default TTL has the same "masked outage" property as EventBridge's 24-hour default, with the additional quirk that dead-letter writes are deliberately delayed five minutes and can be lost entirely if the dead-letter destination is unavailable for four hours. Every cloud gives you the pattern; every cloud makes you read its retry documentation before trusting the failure posture.

Production realities

Everything above works. Here is what it costs and where it breaks:

API-call amplification is the hidden bill. The event-driven pattern converts one user request into a storm of metered API operations. Script-verified arithmetic for a mid-size fleet — 500 new conversations per minute, 15 steps each: that is 7,500 steps/min, which at two SQS calls (receive + delete) per step is 15,000 SQS API calls per minute (250/s, 21.6 million per day), plus 7,500 completion events per minute (10.8 million entries per day — 1.08 million PutEvents requests with 10-entry batching), plus one DynamoDB write per step for idempotency (another 10.8 million per day). Each of those lines is a rounding error at chat scale and a line item at fleet scale. SQS and EventBridge bill per request, DynamoDB bills per write — the pattern's cost scales with step count, not user count, so a refactor that cuts average steps per conversation is a direct infrastructure discount.

The 5-minute Express ceiling is closer than it looks. A 12-step agent chain at 30 seconds per step is 360 seconds — over the 300-second Express maximum before a single tool call misbehaves. Express is for high-volume short event choreography (fan-out, formatting, dispatch); anything with an LLM call inside belongs on Standard, where the 1-year execution ceiling and per-transition persistence exist precisely for this. The math that bites: chain_steps × p95_step_latency × retry_safety against 300 seconds, not the average case.

The in-flight cap is a concurrency ceiling you chose. At the worked example's rates with 90-second average steps, Little's law puts ~11,250 steps in flight — 9.4% of the ~120,000 in-flight cap per standard queue. Double the fleet or double step latency and you are a quarter of the way to OverLimit on a queue that still looks shallow. Capacity-plan on in-flight steps (arrival rate × step duration), never on queue depth alone.

Default retry windows hide outages for a day. EventBridge's classic default — 185 attempts over 24 hours with backoff — means a target that went down at 09:00 keeps receiving retries until 09:00 tomorrow, and the "recovery" is a duplicate-laden thundering herd you did not observe in real time. The same posture exists on Azure (24-hour Event Grid TTL) and in Pub/Sub redelivery. Bounded retries (this guide's 24 attempts over 1 hour) with a DLQ and an alarm turn a silent 24-hour degradation into a 5-minute page. Choose latency-to-detection over delivery optimism; that is the entire argument.

The EventsV2 migration changes defaults under you. The newer custom event bus model (subscribers instead of rules, event groups, PutRawEvents) resets retry defaults to 5 attempts over 300 seconds, versus 24 hours/185 on classic targets. A design that relied on a day of retry headroom must re-set MaxEventAgeInSeconds explicitly or events start dead-lettering after five minutes of target outage. V2 also brings per-account caps with teeth: 5 V2 buses per region, subscriber filter changes limited to 24 in any rolling 24 hours, and only one subscriber create/delete in progress per bus across all accounts — a fleet-wide fan-out migration stalls on that last one if you are not prepared to sequence it.

FIFO lanes are a latency commitment. Head-of-line blocking means one stuck session slows its own conversation's subsequent steps, a poison message stalls its group indefinitely, and EventBridge retrying a FIFO target holds the group's order hostage for the retry window — the docs say it plainly: a long retry window trades order for latency. On GCP it is worse: redelivery of one ordered message re-delivers all successors, acknowledged or not.

Callback throughput throttles before you expect it. The worked fleet calls SendTaskSuccess at 125/s against a 500/s refill rate in big-3 regions — 4× headroom. Quadruple the fleet and the callback API becomes a per-account bottleneck before the queues do. Batch callbacks per execution, or shard by state machine, when you cross a few hundred callbacks per second.

Memory extraction is metered. AgentCore Memory caps long-term extraction at 150,000 tokens/minute per account (50,000 per session). A fleet that enables summarization strategies on every conversation's full history will discover this as silent gaps in memory — the meter is a quota, not an unlimited pipeline, and the CloudWatch TokenCount metric in the Bedrock-AgentCore namespace is the one to alarm on.

Observability breaks at every async hop. Queue, bus, and state machine each own a different id space (message ID, event ID, execution ARN); nothing stitches them unless your envelope carries a stable trace_id through every layer and your workers propagate it into model-call spans. The Agentic AI Lens makes traceability an acceptance criterion of the messaging layer — you must be able to follow every agent message across synchronous and asynchronous boundaries — and its prescribed instrument is AgentCore Observability, so you can trace a message across EventBridge, SQS, and agent boundaries instead of correlating three consoles by timestamp. Budget a day for this per agent class; it is the difference between "DLQ says ERROR_FROM_TARGET" and knowing which model call produced it.

Distributed Map caps are generous until they are walls. 10,000 concurrent child executions per MapRun and 1,000 open Map Runs per account are non-adjustable. A fleet-wide research fan-out pattern that spawns a MapRun per conversation can exhaust the account-level open-run cap with 1,000 concurrent conversations — queue the fan-outs or share MapRuns across conversations before you hit it.

Who should skip this pattern

Not every agent workload needs a fleet nervous system. Skip (or defer) event-driven agent architecture when:

• The product is an interactive chat. Users waiting on a streamed token do not want durability — they want latency. A synchronous gateway with streaming is the correct architecture; put a queue behind it for the tools, not the conversation.
• Your "fleet" is one agent with fewer than five steps. The bus, the rules, the DLQs, the idempotency table, and KEDA are overhead that a single Lambda and one queue do not justify. The pattern earns its complexity at fan-out, multi-class work, or multi-minute steps — not before.
• You have strict exactly-once business requirements (payments, inventory decrements, legal document generation). Event-driven infrastructure is at-least-once everywhere and exactly-once nowhere; a transactional outbox on a relational database with a worker reading the outbox gives you the durability story with real transactions. Do not bolt that requirement onto a bus.
• The team has no event-schema discipline. Without versioned detail-type contracts and a schema registry, a shared bus degenerates into couples counseling between producer and consumer teams within a quarter. Event-driven failure modes punish undisciplined orgs specifically.
• Your workload is GPU batch inference. Queue-per-request serialization destroys exactly the batching that makes inference cheap — throughput-oriented model serving wants continuous batching (or in-cluster routing such as the patterns covered in model serving topologies), not per-message pods. Async orchestration belongs around the batch boundary, not inside it.

For everyone else — platform teams running multi-class agent workloads with minutes-scale steps, fan-out research, human-in-the-loop pauses, and real blast-radius concerns — this topology is not optional polish. It is the difference between an agent product that is down when a provider rate-limits you and one that quietly drains its queues and catches up. The queue is the contract; the bus is the nervous system; the state machine is the memory; everything else is workers. Build the planes in that order and the fleet tends to stay up.