The Cart Must Outlive the Model: Designing Always-On AI Voice Ordering on Amazon Connect (Part-2)
How to step down from Connect AI agents to Amazon Lex to a human agent in the middle of an order, without losing the cart, the call, or the revenue.
In Part 1, we explored the first two rungs of our conversational AI ordering journey:
Rung 1 — AI Agent: A flexible, free-form conversational experience that supports multiple order modifications within a single interaction.
Rung 2 — Amazon Lex: A structured, guided conversational experience that handles one order modification per turn using natural language understanding (NLU) and slot-based interactions.
This is where Rung 3: A Human Closes the Order comes into play.
7. Rung 3: A Human Closes the Order
On `FinishOrder`, `SpeakToAgent`, or a Lex failure, a handoff Lambda function moves ownership to `AGENT` and returns a compact summary: cart ID, line count, unvalidated lines, rung path, and step-down reason. Set contact attributes stores that summary, not the cart; a contact’s attributes are capped at 32 KB, and exceeding the cap takes the Error branch. A Set event flow block with the default flow for the agent UI opens a step-by-step guide that loads the cart by ID, and the call goes to the ordering queue.
The human agent confirms unvalidated lines, takes payment and delivery details, and submits the order in the CRM. If the wait is long, the flow offers a callback, and because the cart persists until its TTL, the callback agent resumes the same cart.
8. The Health Loop: Deciding Where the Next Caller Starts
Per-contact exits decide one caller’s path. The circuit decides the next caller’s.
Each exit records an outcome (`AI_COMPLETE`, `AI_DEGRADED`, `AI_ERROR`, `LEX_COMPLETE`, `AGENT_CLOSED`, or `ABANDONED`, with a reason dimension such as `CRM_LATENCY`) as a CloudWatch metric in embedded metric format, alongside the Cart Service’s CRM latency and deadline misses. A health evaluator Lambda function, run every minute by EventBridge Scheduler, moves the circuit: CLOSED to OPEN when the error or step-down rate crosses a threshold over a minimum sample size, OPEN to HALF_OPEN after a cool-down, and HALF_OPEN to CLOSED when probe calls succeed.
Connect’s AI agent metrics, available through GetMetricDataV2 (for example `AI_AGENT_INVOCATION_SUCCESS_RATE` and `AVG_AI_TOOL_INVOCATION_LATENCY`) and the AI Agent Performance dashboard, suit trend analysis better than circuit tripping; the AI agent data lake tables refresh every 15 minutes. The circuit runs on your own near-real-time signals.
One Call, End to End
The opening incident, replayed against this architecture:
1. The entry Lambda function matches the caller’s number to one store and writes the contact’s META item, and creates the store’s draft cart at version 1.
2. The health gate reads the cached circuit (CLOSED), and the Connect assistant block starts the AI session.
3. Get customer input hands the conversation to the orchestrator AI agent.
4. The caller adds Velmora Lemon, and setLineQuantity at version 1 writes version 2. Two cases of Orbitta Cola follow, at version 2, writing version 3. Each accepted mutation advances the version by exactly one
5. “Six cases of Velmora Sparkling Water, the 20-ounce bottles.” The agent says “Let me add that,” calls `searchCatalog`, then `setLineQuantity` at version 3; the Cart Service prices the line within its deadline and writes version 4.
6. “Drop the lemon one, and make the Orbitta Cola three cases.” `removeLine` writes version 5. The Orbitta Cola update misses the CRM deadline twice, and the second miss returns `DEGRADE_REQUIRED`.
7. The agent calls `SwitchToGuidedOrdering` with `pendingRequest: “Orbitta Cola to three cases”`; the flow sees `Tool = SwitchToGuidedOrdering`.
8. PrepareResume moves ownership to `LEX` at version 5, builds the summary, and records `AI_DEGRADED` with reason `CRM_LATENCY`.
9. Rung 2 plays the summary and asks about Orbitta Cola. The caller confirms, and with the CRM circuit now open, the line is written in capture mode.
10. “That’s everything.” `FinishOrder` triggers the handoff. The human agent sees the cart with one unvalidated line, confirms the Orbitta Cola price once the CRM responds, and submits.
11. The outcome is recorded with rung path AI, LEX, AGENT and the cart diff. The health evaluator sees the step-down rate rising and opens the circuit, so the next callers start at Rung 2.
Don’t Ask the Model How Confident It Is
It’s tempting to have the agent return a confidence score and step down below 0.7. Resist it. A model-reported confidence is not a calibrated probability; a 0.9 doesn’t mean the model is right nine times in ten. And the failure that matters most here, confidently applying the wrong edit, is exactly the one self-reported confidence is worst at revealing. Every trigger in this design is therefore an event measured outside the model. The same caution applies to Lex’s NLU confidence scores on Rung 2: they rank interpretations, but they aren’t probabilities until calibrated against outcomes.
The thresholds themselves (two deadline misses, three corrections, a fleet error rate) are starting points, not constants. Tune them against what actually happened:
Stepped down? Stayed on Rung 1?
↓ ↓
Human barely changed the AI-built Caller abandoned, or the human
cart, caller didn't abandon corrected many AI-built lines
↓ ↓
Likely false step-down Missed step-down
(cost: agent minutes) (cost: lost order, poor experience)
\ /
Tune thresholds, weighted by costThe best ground truth is the cart diff: the difference between the cart the AI built and the order the human finally submitted. It measures whether the AI built the right cart, independent of anything the model says about itself.
What to Measure
For system quality: AI path completion rate (`Complete` over Rung 1 entries), step-down rate by reason, cart diff rate, CRM deadline-miss rate and p95 latency, time from failure to the first Rung 2 prompt, minutes the circuit spends open, and Connect’s tool-invocation success rate and prompt latency.
For the business, the headline “always on” metric is order capture rate: ordering-intent calls that end in a submitted order and generating revenue. Alongside it, track abandonment within 30 seconds of a step-down, handle time and order value by rung path, and repeat calls from the same store within 24 hours.
Tradeoffs Worth Arguing About
Get Ambuj’s stories in your inbox
Join Medium for free to get updates from this writer.
DynamoDB, not the CRM, holds the draft cart
The CRM is the system of record, but it is also a dependency this design protects against, and the draft cart needs predictable single-key reads and conditional writes on every turn. Wewould keep drafts in the CRM if it were fast, highly available, and agents needed drafts visible there natively, accepting that its outages become ordering outages. However, across 2 diff CRM, we have seen the latency compromises cusomer experience hence using fast layer such DynamoDB always won the argument.
Rung 2 is Lex, not a smaller LLM
A second LLM path usually shares the failure domain (the same Bedrock dependency, the same slow tools) and the same quality failure modes. I would add a different model family on cross-Region inference if most incidents were capacity problems on one model, knowing it won’t help when tools are slow or the conversation is going wrong.
A CRM slowdown moves callers off the AI agent
With a healthy model, capture mode could keep callers on Rung 1. We step down because one AI turn can chain several tool calls (search, read, write), multiplying CRM latency, while a Lex turn makes exactly one service call. I would revisit this with measured per-turn latency.
AgentCore Gateway, not flow module tools
A Lambda function behind AgentCore Gateway gives code-level control over deadlines and fencing, and the Cart Service is shared with Rung 2’s code hook anyway. We would choose flow module tools if the ordering logic already lived in flows.
Production Considerations
Latency. An AI turn’s synchronous path is model inference plus every tool call in the chain, so tool deadlines are the main lever. Keep the flow’s Lambda functions fast: beyond the 8-second per-function cap, Connect limits a sequence of Lambda functions to 20 seconds in total. Put static instructions first in the orchestration prompt so prompt caching can cut latency.
Fallbacks rot. A path that runs only during incidents will be broken when you need it. Route a small, steady share of traffic through Rung 2 and run game days that force the circuit OPEN. Constant tools, which return a fixed string to the AI agent, are handy for scripting failure responses.
Capacity. An open circuit shifts load toward human queues; without an AI-outage scenario in the staffing plan, the fallback just relocates the failure. Callbacks absorb bursts, and reserved concurrency keeps unrelated spikes from throttling the Cart Service.
Security. Bind the store ID server-side from the contact, or through the tool’s input overrides, so the model never chooses which account it writes to. The AI agent’s security profile limits its tools, carts hold no payment data, and Guardrails’ sensitive-information filters reduce PII in model inputs and outputs.
Observability. Put the contact ID on every mutation, metric, and log line. Connect’s ListSpans API exposes AI agent traces, including tool calls, in real time, so “why did this contact step down?” has a per-call answer.
Cost. Rung 1 is dominated by tokens per turn (orchestration prompt, history, tool results), so keep tool responses compact. Rung 2 adds Lex requests only when it runs. The expensive rung is the human at the bottom, which is why the false step-down rate deserves as much attention as the missed one.
Scope.This design covers failures within a Region; cross-Region resiliency for Connect and Lex is a separate design.
Final Thoughts
Replay the opening call against this design and the CRM still stalls. What changes is what the caller hears next: their own order read back to them, and a person who finishes it.
The lesson extends to any AI agent that executes transactions. Resilience is a state-ownership problem before it is an error-handling problem. Make the model a client of the cart rather than its container, and every failure becomes a handoff instead of a lost order.
Note: Product names, store identifiers, and dialog examples in this article are synthetic.
AWS References
Amazon Connect
- Use agentic self-service
- Enable AI agents to retrieve information and complete actions with MCP tools
- How to use orchestrator AI agents
- Flow block: Get customer input
- Grant Amazon Connect access to your AWS Lambda functions
- Flow block: Set contact attributes
- AI agent metrics
- Customize AI in Amazon Connect Customer: Agents, Prompts, and Guardrails (AWS Contact Center Blog)
Amazon Lex V2
- AMAZON.QinConnectIntent
- Improve intent classification and slot resolution with assisted NLU
- Improving recognition of slot values with runtime hints
Amazon Bedrock AgentCore and DynamoDB
