Building AI agents on Amazon Bedrock

You're evaluating Bedrock for agent work because it's managed, it has good model selection, and your infrastructure is already on AWS. The question isn't whether you can build agents there—you can—but whether the operational model fits your constraints, and where you'll hit friction.

The core model: Bedrock's tool use

Bedrock exposes Claude and other models through a standard API. Tool use works like this: you define tools as JSON schemas, pass them to the model, and the model returns structured requests to invoke them. You execute those tools, stream results back, and iterate until the model decides it's done.

The API looks straightforward:

response = bedrock_client.converse(
    modelId="anthropic.claude-3-5-sonnet-20241022",
    messages=[{"role": "user", "content": "..."}],
    tools=[
        {
            "toolUseBlock": {
                "name": "search_docs",
                "description": "Search internal documentation",
                "inputSchema": {
                    "type": "object",
                    "properties": {
                        "query": {"type": "string"}
                    },
                    "required": ["query"]
                }
            }
        }
    ]
)

The model returns tool_use blocks. You parse them, execute your code, and send results back in the next message as tool_result blocks. This is the agentic loop.

Where the friction starts

State management is your problem. Bedrock is stateless. Every call is independent. You maintain the entire conversation history in memory or a database, and you send it with every request. For a multi-turn agent that runs for minutes or hours, this adds up: you're serializing and deserializing kilobytes of context repeatedly, paying per-token for tokens you've already processed.

Claude's prompt caching mitigates this somewhat—repeated prefixes of your messages are cached for 5 minutes—but you need to structure your messages carefully to benefit. If your tool results or context change between calls, the cache misses.

Tool execution is synchronous in the API contract. The model emits a tool_use block, you execute it, and you must send the result back in the next message before the model can proceed. If your tools are slow (database queries, external API calls, LLM invocations), your agent blocks. You can parallelize tool execution client-side, but the API doesn't help you there.

Error handling requires explicit loops. If a tool fails, you need to decide: retry the tool, tell the model and let it choose a different approach, or fail the agent. Bedrock won't do this for you. You write a loop that catches exceptions, formats them as tool_result blocks with error details, and sends them back. If your tools are flaky, you'll write a lot of retry logic.

Token counting is approximate. Bedrock charges per token. The API gives you token counts in responses, but they're not always precise until billing, and they don't account for prompt caching or other optimizations. You'll need to instrument and monitor actual costs separately from what the API tells you.

A realistic agent architecture

Here's the shape of what you'll actually build:

flowchart TD
    A["User request"] --> B["Load conversation history"]
    B --> C["Call Bedrock with tools"]
    C --> D{"Model response type"}
    D -->|tool_use| E["Parse tool blocks"]
    D -->|text| F["Return to user"]
    E --> G["Execute tools in parallel"]
    G --> H{"All succeeded?"}
    H -->|yes| I["Format results"]
    H -->|no| J["Format errors"]
    I --> K["Append to history"]
    J --> K
    K --> L["Check stop conditions"]
    L -->|continue| C
    L -->|done| F

You'll maintain a message history. After each Bedrock call, you append the assistant response and all tool results to it. You check for stop conditions—max turns, timeout, explicit stop token from the model—before looping. If you hit a limit, you format what you have and return to the user.

Tool execution should be concurrent. Python's asyncio or ThreadPoolExecutor work fine. Wrap each tool in a timeout and exception handler. If a tool fails, include the error message in the tool_result so the model sees it and can adapt.

async def execute_tools(tool_uses):
    tasks = [
        asyncio.wait_for(
            execute_single_tool(tool),
            timeout=10.0
        )
        for tool in tool_uses
    ]
    results = await asyncio.gather(*tasks, return_exceptions=True)
    return results

Persistence and recovery

Store your conversation history in a database. DynamoDB works if you key by session_id and manage TTL. PostgreSQL is simpler if you already have it. Each message gets a timestamp and sequence number so you can reconstruct the exact state.

If your agent crashes mid-turn, you can resume from the last persisted state. You've already paid for those tokens; don't repeat them. Load the history, send it back to Bedrock, and continue.

For long-running agents, periodically summarize old context and replace it with a summary. This keeps token usage bounded. You lose some detail, but the model can work with high-level summaries of what's happened.

Cost and latency reality

Bedrock's pricing is per-million tokens. For a multi-turn agent making tool calls, expect 1-5 tokens per user word, plus the full context repeated on each turn. A 10-turn agent with moderate context will cost $0.01-0.10 per session depending on model and context size.

Latency depends on tool execution time, not Bedrock. The API itself responds in 1-3 seconds for a typical tool_use response. If your tools take 30 seconds, your agent takes 30 seconds per turn.

When Bedrock is the right choice

Use Bedrock if you're already on AWS and want to avoid managing model infrastructure. The API is stable, Claude's reasoning is solid for agent tasks, and you get model updates without redeploying. The operational overhead is lower than self-hosted.

Don't use it if you need sub-second latency, very high token volume (where self-hosted economics win), or you're building a system where model state matters between requests (you'll fight the statelessness).

The honest tradeoff

Bedrock abstracts away infrastructure, but it doesn't abstract away the agent engineering. You still own state management, tool execution orchestration, error handling, and cost optimization. The managed part is the model—everything else is yours. That's a fair tradeoff if your team is small and you'd rather not run a model server. It's a poor tradeoff if you're already running infrastructure and need tight integration with your system.

Start with a prototype that treats each agent invocation as independent. Get the tool definitions right, test error cases, and measure token usage. Then add persistence and multi-turn loops. The complexity compounds quickly once you're managing state across requests.