The Cart Must Outlive the Model: Designing Always-On AI Voice Ordering on Amazon Connect (Part-1)
How to step down from Connect AI agents to Amazon Lex to a human agent in the middle of an order, without losing the cart, the call, or the revenue.
A store owner calls in the weekly restock order. In under a minute they’ve asked for six cases of Velmora Sparkling Water, switched the 12-ounce cans to 20-ounce bottles, and dropped Velmora Lemon. The AI agent starts reading the cart back, then goes quiet. Five seconds. Eight. Then: “Sorry, I’m having trouble right now. Please call back later.”
Maybe they call back. Maybe the order goes to a competitor’s sales rep on Thursday.
Note: We’ll explore this topic in two parts to keep things concise, engaging, and easy to follow. Each part will be approximately a 10-minute read, making it easier to absorb the key insights without feeling overwhelmed.
The failure itself is ordinary: a CRM tool call ran long, a response failed validation, or the model endpoint returned errors for a few minutes. What makes it expensive is where it happened: mid-transaction, with a half-built cart that existed nowhere except inside the AI conversation.
We rebuilt our ordering line on Amazon Connect (now documented as Amazon Connect Customer), replacing “say add item or press 1” directed dialog. Directed dialog handles one-step tasks, not carts, where a caller says “make that two cases instead of three, and drop the lemon one”: two edits in one utterance, both pointing at earlier turns. So once intent is identified as ordering, we hand the call to a Connect AI agent. Agentic self-service uses orchestrator AI agents that reason across steps and call MCP tools against backend systems within one continuous conversation, governed by the same security profiles as human agents and running on a model you select per prompt.
This article is about what the caller experiences when that agent can’t do its job.
The Hard Part Isn’t Catching the Error
The obvious design wraps the AI agent in error handling: on failure, apologize and transfer to a queue. It has two flaws.
First, the human agent asks “What would you like to order?” and the caller repeats everything. The fallback worked; the experience didn’t.
Second, “the AI failed” is not one event. A hard failure (session or model endpoint errors) is loud and immediate. A slow failure (a tool stalled on the CRM) is dead air. A quality failure (edits keep being misapplied) raises no error at all. A catch-all handler is too late for the second and blind to the third.
Underneath both is a state problem. In a naive build the cart lives in the model’s context. Whatever the model remembers *is* the order, and no error handler can recover state that was never stored outside the component that failed.
Always-on ordering doesn’t come from making the AI agent never fail. It comes from keeping the order outside the AI agent, so every simpler mode of the experience resumes from the same cart.
Please see below standard design appraoch.
Caller identified, intent = place order
↓
AI agent conversation (cart lives in model context)
↓
CRM tool stalls / model endpoint errors / edits misapplied
↓
Generic apology → transfer to queue
↓
Human agent: "What would you like to order?"
↓
Caller repeats the order, or hangs upin this experience, the Context is lost at both handoffs, and failover begins only after the caller has heard the failure.
Reimagined Design: A Degradation Ladder Over Durable Cart State
Treat ordering as a ladder of progressively simpler modes that share one durable cart:
Rung 1 AI agent free-form, multi-edit conversation
Rung 2 Lex guided dialog one edit per turn, NLU slots
Rung 3 Human agent reviews the cart, closes the order
Rung 4 Queued callback when no agent is available (optional)Three rules make it work. A deterministic Cart Service owns the cart and persists every mutation; the model asks it to change the cart but never holds the cart. Every rung goes through that same service, so stepping down is a resume, not a restart. And stepping down is triggered by observable signals (errors, deadlines, validation failures, fleet health), never by the model’s opinion of itself. Uncertainty moves a caller one rung down, never up.
Architecture Overview
The following architecture is one way to implement this pattern with AWS services: the synchronous call path from entry to human agent, the cart every rung shares, and the asynchronous loop that decides where the next caller starts.
The sections below follow Figure 1 from left to right.
- Identity: Whose Cart Is This?
Resuming a cart needs a stable key, and caller ID is a hint, not an identity: stores share numbers, owners call from personal phones, numbers can be withheld. An entry Lambda function looks the number up in the store directory. A single match lets the caller start a draft cart provisionally, while submission requires verification (an account number and PIN by keypad). Zero or multiple matches verify up front. The verified `storeId` keys the cart. Verification design itself is out of scope; the architecture assumes a verified store account ID before any order is submitted. Or we can sync customer profiles and with CRM in frequency to to keep customer profiles update to date to avoid Lambda call.
2. Health Gate: Don’t Send Callers Into a Known Outage
Before Rung 1, an Invoke AWS Lambda function block reads the AI ordering circuit, a single DynamoDB item. CLOSED sends callers to the AI agent, OPEN starts them at Rung 2, and HALF_OPEN sends a small share to Rung 1 to probe recovery. The same item doubles as an operator kill switch for model incidents or risky prompt releases.
Every call reads that one key, so the function caches it in memory for a few seconds, keeping the gate fast and the hot item unthrottled. Connect caps synchronous flow Lambdas at 8 seconds; set the gate’s timeout far lower, and route its Error branch, like the Connect assistant block’s, to Rung 2.
3. Rung 1: The AI Agent Owns the Conversation, Not the Cart
Get Ambuj’s stories in your inbox
Join Medium for free to get updates from this writer.
- Cart tools (`searchCatalog`, `getCart`, `setLineQuantity`, `removeLine`, `submitOrder`): MCP tools backed by the Cart Service Lambda function, registered on an Amazon Bedrock AgentCore Gateway linked to Connect.
- Return to Control tools that end the AI conversation: the default `Complete` and `Escalate`, plus our `SwitchToGuidedOrdering`.
- Guardrails with sensitive-information filters and denied topics, such as price negotiation.
The model’s job is narrow: turn utterances into cart edits, resolve references like “the lemon one,” pick tools, and phrase confirmations. Everything with business consequences stays deterministic in the Cart Service: SKU validity, price, availability, limits, totals, submission eligibility, and whether to step down.
Two decisions make the tools safe for an LLM caller. Tools take absolute values (`setLineQuantity(sku, 5)`, never `addQuantity(sku, 2)`), so a repeated call converges instead of doubling a line. And every write carries the cart version the model last saw, enforced by a conditional write, so a stale or late write is rejected rather than overwriting newer state.
Latency needs its own budget. MCP tool invocations time out at 30 seconds, a platform ceiling rather than a voice budget, so the Cart Service puts a much shorter deadline on every CRM call and returns a fast, structured failure when it’s missed. The prompt also has the agent speak first: orchestrator agents can emit several `<message>` blocks per response, so the caller hears “Let me update that” while the tool runs.
4. Cart Service and DynamoDB: The Order’s Single Source of Truth
The access patterns drive the model. Every turn on every rung reads a store’s active cart and applies one mutation atomically against a known version. The agent’s screen pop finds a cart from a contact. Per-contact health counters change on every edit. Abandoned drafts expire, and the gate reads one flag.
PK SK Holds
STORE#S-10442 CART#ACTIVE Draft cart: lines, version, status, ownerRung, expiresAt
STORE#S-10442 CART#C-7f3a#HIST#000007 Append-only mutation record (op, rung, contactId), TTL
CONTACT#c-3c1e9a52 META storeId, cartId, rung path, health counters, TTL
SYSTEM HEALTH#AI_ORDERING Circuit state, reason, changedAtThe active cart is one item because each mutation re-validates the whole cart and must be atomic. DynamoDB’s 400 KB item limit comfortably fits a few hundred lines; beyond that, move lines into separate items under a versioned header. TTL deletion is asynchronous, so reads also check `expiresAt`.
`ownerRung` is a fence. Stepping down conditionally moves ownership to `LEX`, so a Rung 1 tool call that completes late against a slow CRM fails its condition instead of editing a cart another rung now owns.
import time
import boto3
from botocore.config import Config
from botocore.exceptions import ClientError
# Tight client timeouts: this code runs inside a live voice turn.
ddb = boto3.client("dynamodb", config=Config(
connect_timeout=1, read_timeout=1, retries={"max_attempts": 2}))
TABLE = "ordering"
class CartConflict(Exception):
"""Version moved or another rung owns the cart. Re-read; don't retry blindly."""
def set_line_quantity(store_id, cart_id, sku, qty, unit_price, validated,
expected_version, rung, contact_id):
new_version = expected_version + 1
now = int(time.time())
line = {"qty": {"N": str(qty)}, "validated": {"BOOL": validated}}
if unit_price is not None: # absent in capture mode
line["unitPrice"] = {"N": str(unit_price)}
try:
ddb.transact_write_items(TransactItems=[
{"Update": {
"TableName": TABLE,
"Key": {"PK": {"S": f"STORE#{store_id}"}, "SK": {"S": "CART#ACTIVE"}},
"UpdateExpression": "SET #lines.#sku = :line, #ver = :nv, "
"updatedAt = :now, lastContactId = :cid",
"ConditionExpression": "#ver = :ev AND ownerRung = :rung AND cartId = :cart",
"ExpressionAttributeNames": {"#lines": "lines", "#sku": sku, "#ver": "version"},
"ExpressionAttributeValues": {
":line": {"M": line}, ":nv": {"N": str(new_version)},
":ev": {"N": str(expected_version)}, ":rung": {"S": rung},
":cart": {"S": cart_id}, ":now": {"N": str(now)},
":cid": {"S": contact_id}},
}},
{"Put": {
"TableName": TABLE,
"Item": {
"PK": {"S": f"STORE#{store_id}"},
"SK": {"S": f"CART#{cart_id}#HIST#{new_version:06d}"},
"op": {"S": "SET_QTY"}, "sku": {"S": sku}, "qty": {"N": str(qty)},
"rung": {"S": rung}, "contactId": {"S": contact_id},
"ttl": {"N": str(now + 30 * 86400)}},
"ConditionExpression": "attribute_not_exists(SK)",
}},
])
except ClientError as err:
if err.response["Error"]["Code"] == "TransactionCanceledException":
raise CartConflict(cart_id) from err
raise
return new_versionThe transaction pairs the cart update with a history record keyed by the new version. `TransactWriteItems` is all-or-nothing, and a `ClientRequestToken` makes retries idempotent within a 10-minute window. The snippet assumes the `lines` map was created with the cart and omits validation, the CRM call, line removal, and tracing. In production, inspect the cancellation reasons to separate a failed condition (re-read the cart) from a transaction conflict (retry).
The `validated` flag enables capture mode. When the CRM is the struggling dependency, the Cart Service accepts a line without live price or availability and leaves confirmation to a human, so turns stay fast. A separate CRM circuit inside the Cart Service stops synchronous CRM calls once failures pile up.
5. Leaving Rung 1: Where the Step-Down Decision Lives
The flow regains control from the AI agent in only two ways: a Return to Control tool, which ends the AI conversation and stores the tool’s name (under the key `Tool`) and inputs as Lex session attributes, or the Get customer input Error branch. The flow cannot interrupt the agent mid-conversation.
So in-conversation quality checks belong in the Cart Service, the deterministic component the agent touches on every edit. It counts CRM deadline misses, validation rejects, and corrections (a line changed or removed soon after it was added) per contact. Past a threshold, the tool returns a directive instead of a result:
{
"status": "DEGRADE_REQUIRED",
"reason": "CRM_LATENCY",
"cartVersion": 5,
"instruction": "Stop editing the cart. Call SwitchToGuidedOrdering."
}The prompt maps that status to `SwitchToGuidedOrdering`, whose input schema captures what the next rung needs:
{
"type": "object",
"properties": {
"reason": {
"type": "string",
"enum": ["DEGRADE_REQUIRED", "REPEATED_MISUNDERSTANDING", "CUSTOMER_REQUEST", "OUT_OF_SCOPE"]
},
"cartVersion": {
"type": "integer",
"description": "Latest cart version returned by the cart tools"
},
"pendingRequest": {
"type": "string",
"maxLength": 200,
"description": "The customer's last request that was not applied to the cart"
}
},
"required": ["reason", "cartVersion"]
}The handoff is still model-mediated. If the model ignores the directive, the Cart Service refuses every further mutation from that contact, so the worst case is a stalled conversation that ends in `Escalate` or an error, and both route down the ladder. Because `pendingRequest` is model-generated, Rung 2 confirms it with the caller rather than applying it.
A Check contact attributes block on the `Tool` attribute routes the exits. The complete policy:
6. Rung 2: Guided Ordering That Doesn’t Share the Failure
Rung 2 is a separate Amazon Lex V2 bot, defined mostly by what it doesn’t depend on. Lex’s Assisted NLU and assisted slot resolution both invoke Amazon Bedrock models; enable them here and the fallback joins the failure domain it exists to escape. Keep them off, or prove with fault injection that the bot degrades cleanly without Bedrock. A separate bot also avoids the AI agent intent’s bot-level constraints, such as not sharing a locale with `AMAZON.QnAIntent`.
The intents are deliberately plain (`AddOrChangeItem`, `RemoveItem`, `ReviewCart`, `FinishOrder`, `SpeakToAgent`): one confirmed edit per turn, slower than Rung 1 and far more predictable.
A PrepareResume Lambda function first moves ownership to `LEX` at the expected version and builds a spoken summary for the bot’s opening prompt: “I have your order so far: six cases of 20-ounce Velmora Sparkling Water and two other items. You’d also asked to change Orbitta Cola to three cases. Should I do that?”
Product names are where directed voice bots struggle, so the code hook sends runtime hints (the store’s frequent items plus the lines already in the cart) before eliciting the product slot. Lex then prefers those phrases over similar-sounding values. Hints work with custom slot types on 8 kHz contact center audio, in English (US) and English (UK).
def handle_add_or_change(event):
state = event["sessionState"]
attrs = state.get("sessionAttributes") or {}
slots = state["intent"]["slots"]
if event["invocationSource"] == "DialogCodeHook" and not slots.get("Product"):
# Bias recognition toward this store's catalog before asking for the product.
phrases = cart_service.hint_phrases(attrs["storeId"], limit=50)
hints = {"slotHints": {"AddOrChangeItem": {"Product": {
"runtimeHintValues": [{"phrase": p} for p in phrases]}}}}
return elicit_slot(state, "Product", "Which product?", runtime_hints=hints)
if event["invocationSource"] == "FulfillmentCodeHook":
result = cart_service.set_line_quantity_by_name( # same service as Rung 1
store_id=attrs["storeId"], cart_id=attrs["cartId"],
product=slots["Product"]["value"]["interpretedValue"],
qty=int(slots["Quantity"]["value"]["interpretedValue"]),
expected_version=int(attrs["cartVersion"]),
rung="LEX", contact_id=attrs["contactId"])
attrs["cartVersion"] = str(result.version)
message = (f"Done. {result.spoken_line}. Anything else?" if result.validated else
f"Got it: {result.spoken_line}. A team member will confirm "
"price and availability. Anything else?")
return continue_ordering(state, attrs, message) # ElicitIntent: stay in the bot
return delegate(state)Because the hook calls the same Cart Service, a degraded CRM yields capture-mode lines here too. The response-envelope helpers, slot validation, and error handling are omitted; the flow passes `storeId`, `cartId`, `cartVersion`, and `contactId` as session attributes.
Continue reading: Check out Part 2 of this blog post, where we’ll dive deeper into the topic and explore additional insights.
Note: Product names, store identifiers, and dialog examples in this article are synthetic.
