An LLM can make an unfinished chatbot look remarkably complete. It can produce polished answers, follow a prompt, and make a prototype feel ready for customers. That fluency can also hide gaps in the product.
The difficult questions remain. What can the assistant reliably answer? What information does it need? When should it clarify, recover, or hand the conversation to a person? And how will the team know whether a change has improved the experience?
Conversation design gives teams a way to answer those questions before fluent responses disguise them.
This workflow covers six connected stages:
The stages are presented in sequence, but the work is still iterative. Testing exposes missing scenarios. Live conversations reveal language and behaviour the team did not anticipate. Those findings should be fed back into your design.
To make the workflow concrete, this guide follows ParcelPal, a fictional delivery assistant that helps customers track parcels, change eligible deliveries, and resolve missed, delayed or missing parcels.
Start by defining what the assistant can do, what it cannot do and who it serves. Capture this in a short, versioned capability statement with a clear owner.
“Helps customers with their deliveries” sounds reasonable but leaves most product decisions unresolved. Can ParcelPal track a parcel? Change delivery date? Change the address after dispatch? These actions belong to the same broad service, yet each relies on different rules, systems, and permissions.
The capability statement should distinguish between two kinds of exclusion:
Without these distinctions, a temporary gap can quietly become a permanent product rule.
The aim is not to describe every future scenario. It is to give the team a stable boundary for the version being designed. If the scope still produces frequent debate, the statement is probably too broad.
“Fluent copy cannot compensate for an unclear capability. It only makes the uncertainty harder to see.”
Once the boundary is clear, identify what customers are trying to achieve. A useful working definition is:
An intent is a customer goal that needs a distinct resolution.
For ParcelPal, common requests might include:
Build the inventory from evidence: service transcripts, search behaviour, customer research and existing chatbot logs. Keep the original customer language alongside the clean intent label. The label helps the team organise the work; the customer’s wording becomes essential when the system is evaluated.
Group requests by the resolution they need, rather than by similar words. “Where is my package?” and “Has it left the depot?” may both require tracking. “My parcel is missing” could refer to a delayed shipment, a parcel marked as delivered or a confirmed loss. Each case may lead to a different process.
For every intent, record:
This turns an intent list into something the product, design, engineering and business teams can work from.
Start with a plain-text dialogue. This keeps the team focused on the behaviour of the conversation before buttons, cards and visual polish enter the discussion.
Customer
I’m waiting for a delivery.
ParcelPal
I can help check it. Do you have the tracking number?
Customer
Yes, it’s PP482731.
ParcelPal
Your parcel is at the local depot and is expected to arrive tomorrow.
Customer
Do you know what time?
ParcelPal
The current delivery estimate is between 2 pm and 5 pm. This may change if the driver’s route is updated.
The dialogue is simple, but it resolves several design decisions. ParcelPal asks for the tracking number because it cannot identify the shipment. It gives the current location and delivery date first. It describes the time as an estimate. When the customer asks a follow-up question, it answers the new need without repeating the full tracking status.
Review each exchange for five things: whether it answers the question, whether the useful information comes first, whether every question is necessary, whether each turn follows naturally, and whether anything is repeated. Reading the dialogue aloud is still one of the quickest ways to expose long instructions and awkward transitions.
Once the main path works, add the less tidy cases: unclear requests, missing information, failed actions and handoff.
Responses should also change with context. The first answer may need enough detail to establish the situation. A follow-up should usually address only what is new. A clarification should ask for the smallest amount of information needed to continue.
This is where LLM conversation design departs from traditional scripting. The model can generate the sentence. The designer still defines what that sentence must achieve.
Keep three layers separate:
Every response depends on the current system's state: what is known, what is missing, what the customer has confirmed, and what the product is allowed to do next.
ParcelPal may know the tracking number and shipment status, but not whether the customer owns the shipment or has permission to change its address. It can display tracking information while still preventing an address change.
For each important state, define:
Design this model with engineering and business teams. A separate design version of product logic will drift quickly. Assumptions also need to be visible. Does a tracking link identify the customer? Can an address change after dispatching? Does the latest scan represent the parcel’s current location? Each assumption shapes what ParcelPal says and does.
System state also helps the assistant explain why something failed. If a customer asks to move a delivery to Friday, the request could fail because the parcel is unclear, a service is unavailable, the shipment is already out for delivery, or rescheduling is unsupported. A generic “Sorry, I can’t do that” hides the cause and gives the customer little help.
Use a small repair taxonomy instead:
For each repair, define the trigger, what the customer needs to understand, and how they can continue. The cause matters more than the apology.
Some requests should leave the assistant. Human support should be a designed route with observable triggers, rather than a catch-all for anything difficult.
A handoff may begin because the customer asks for a person, clarification repeatedly fails, the case needs human judgement, the assistant lacks authority, or security and fraud rules require review.
The conversation after that trigger matters. For a missing parcel, decide what shipment information and conversation history transfer, whether an investigation is already open, where the case goes, and what the customer should expect next. A handoff failed when the customer explained everything again.
Keep the reason for handoff separate from the route. The reason may remain the same while the route changes according to region, opening hours, queue or case type.
These behaviours can then become evaluation cases. Without a shared test set, teams tend to update a prompt, try a few questions, and debate whether the answers feel better. That makes review subjective and regression difficult to spot.
The evaluation set should include clear matches, ambiguous requests, near-matches, out-of-scope questions, restricted actions, and handoff scenarios. Define the expected behaviour rather than one approved sentence.
For the question “Is my parcel coming today?”, a passing response should use the latest shipment status and delivery estimate, distinguish an estimate from a confirmed event, and avoid inventing certainty. Several phrasings could meet those requirements.
Use real customer language wherever possible. Designed examples can fill gaps, but they should not replace evidence. Test the assistant’s full range of behaviour: when it should answer, clarify, refuse and hand off.
A good evaluation checks what the assistant does, not whether it repeats a preferred sentence.
Evaluation tests the scenarios the team knows about, and live conversations will reveal the ones it missed.
Customers combine needs, use unexpected language, and expose weak assumptions. They may describe a parcel as lost, missing, late or simply “not here”. Those phrases sound similar, but operations may treat the cases very differently.
Review a mix of successful, failed, and handed-off conversations. Then turn each finding into an owned change. “Customers are confused here” is mere observation, not something actionable. A useful review identifies what should change. Example of that could be:
These changes will be fed back into the workflow:
Transcript review → intents → conversation → system states and repairs → evaluation
The ownership of these items need to remain visible. A product team may own capability boundaries. Design may lead conversation behaviour. Engineering owns implemented state and technical behaviour. Business teams own the rules and information behind the answers. AI and data teams support the evaluation. And operations owns what happens after a human-in-the-loop handoff.
The split will vary between organisations. What matters is that each decision and change has a visible owner.
LLMs have greatly reduced and will continue to reduce the need to predict every customer phrase or script every response word for word. However, they do not decide what the product can do, which context matters, how failure should be repaired, when the AI assistant should stop, or what good behaviour looks like.
This six-stage workflow makes all those decisions visible. It gives teams a shared way to move from a fluent prototype to a conversation that can truly be built, tested, and improved as the product meets real customers.
