Draft and triage support replies from your knowledge base
An agent reads each incoming ticket, classifies and routes it, then drafts a reply grounded in your own help docs with citations — leaving your team to review, edit, and send. You own the whole pipeline and can fix or extend it without us.
Tools you'll use
Drafting and triaging support replies from your knowledge base is a "draft and triage" pattern, not auto-reply: an agent reads each incoming ticket, tags it by topic, urgency, and sentiment, routes it to the right queue, and writes a first-draft reply pulled from your own help docs — with links back to the source article so your agent can check it in seconds. A person still reviews and sends. This is now mainstream practice: a Gartner survey of 187 service leaders (conducted July–August 2024) found 85% would explore or pilot customer-facing conversational GenAI in 2025.
The mechanics are simple. Sort the inbox, then write the reply by finding the right answer in your help docs or past tickets and putting it in plain words. Both jobs are slow, repetitive, and mostly already documented somewhere — exactly the kind of work an agent does well. The published research is consistent: grounding drafts in your own docs (rather than the model's general training) is what keeps them accurate, and keeping a human on the send button is what keeps you safe.
Why it matters: first-response time drops sharply when a usable draft is already waiting, and agents handle more tickets because they edit instead of starting from a blank box. There is a useful byproduct, too. When the agent can't find an answer, that's a gap in your knowledge base worth filling — and gaps are common. The same Gartner survey found 61% of service leaders already have a backlog of articles to edit. The hard cap on results is the quality of your help docs, not the tool. Most teams find that out fast, which is a good thing.
Moriva's take
Gate 1 (Real work): clear pass — triaging the inbox and drafting replies is the daily job of every support team. Gate 2 (Owned): pass — built with an agentic coding tool, this is a script and a prompt your team holds, can read, and can change when a policy or product changes. Gate 3 (Measured): pass — first-response time, draft-acceptance rate, and tickets-per-agent are all easy to track before and after. We mark this CAREFUL, not GO, only because it touches customer-facing replies and customer data: keep a person on the send button, ground every draft in cited sources, and start with one ticket category before you widen it.
How do you draft and triage support replies from your knowledge base?
- 1
Pull your real tickets and docs into one place
Export a few hundred recent tickets (including the replies your best agents actually sent) and a current copy of your help center or internal docs. Point Claude Code or Codex at that folder. Ask it to read the tickets and report the top ten categories, the typical urgency mix, and which questions show up most often. This gives you a factual map of your inbox before you automate anything.
- 2
Build the triage classifier first, on its own
Have the tool write a small script that reads one ticket and returns a topic tag, an urgency level, a sentiment flag, and a suggested queue or owner. Run it across your exported tickets and compare its tags to how your team actually handled them. Aim for high agreement on the common categories before you trust it; tune the prompt where it disagrees. Triage is lower-risk than replies, so it is the right place to start.
- 3
Wire the draft to your knowledge base, with citations
Tell the tool to retrieve the most relevant help articles or past resolved tickets for a given question, then draft a reply using only that retrieved content — and to include the source link for every claim. Add a hard rule: if it cannot find a grounded answer, it must say so and flag the ticket for a human instead of guessing. Grounding plus a refusal path is what keeps drafts accurate and stops the model from inventing policy.
- 4
Keep the draft in a review step, never auto-send
Have the agent write the draft into your help desk as an internal note or a saved draft — somewhere an agent reads, edits, and sends manually. Start with one ticket category you understand well, like password resets or billing questions. The agent does the fetching and first writing; your person owns the judgment and the send. This is the human-in-the-loop design every credible deployment uses.
- 5
Add a confidence threshold and an escalation rule
Ask the tool to attach a confidence signal to each draft and to route anything low-confidence, angry, or about refunds, security, or legal straight to a senior agent untouched. The hardest tickets are the ones that escalate, so make sure they arrive with full context, not a half-finished draft. Decide these rules in plain English and keep them in the prompt where your team can edit them.
- 6
Turn misses into a knowledge-base gap report
When the agent can't find a grounded answer, log the question. Once a week, have the tool group those misses into a short list of missing or outdated articles, ranked by how often they came up. This closes the loop: the same system that drafts replies tells you exactly which docs to write next, which lifts the accuracy of every future draft.
- 7
Measure, then widen one category at a time
Track first-response time, the share of drafts agents accept with light edits, and tickets handled per agent — before and after, for the one category you started with. When the numbers hold and your team trusts the drafts, add the next category. Because the pipeline is yours, widening it is a prompt change your team makes, not a new project you hire out.
What could go wrong (and how to handle it)
The agent invents a policy or fact the customer then acts on.
Ground every draft in retrieved help-center content and require a source link for each claim. Add an explicit instruction to refuse and escalate when no grounded answer exists, rather than guessing. Keep a human reviewing before send.
Customer data (PII, account details) flows into the tooling without control.
Decide up front what data the agent may see, and redact or exclude sensitive fields before tickets reach it. Keep the pipeline pointed at your own systems, document what is sent where, and confirm it fits your privacy commitments and any regulations you operate under.
Over-automation: drafts get rubber-stamped and a wrong answer goes out at scale.
Never enable auto-send for customer replies in this pattern. Track the draft-acceptance and edit rate; if agents are editing heavily, the docs or the prompt need work. Start with one well-understood category and widen only when quality holds.
Bad triage routes urgent or angry tickets to the wrong place.
Validate the classifier against how your team actually handled past tickets before trusting it. Hard-route high-urgency, negative-sentiment, and sensitive topics (refunds, security, legal) to senior agents regardless of the draft. Review misroutes weekly and tune the rules.
Drafts stay confident even as your product or policy changes, so they slowly go stale.
Because drafts are grounded in your live docs, keeping the docs current keeps drafts current. Use the gap report to fix outdated articles, and re-run the classifier and a sample of drafts whenever a major policy or product change ships.
Results plateau because the help docs themselves are thin or contradictory.
Treat the knowledge base as the real product. Use the agent's misses and any conflicting-article flags to prioritize doc fixes. Outcomes are capped by content quality more than by the tool, so budget time for documentation, not just setup.
Prompts to get started
FAQ
Will this send replies to customers on its own?
No — not in this pattern. The agent triages and writes a first draft into a review step; a person edits and sends. That human-in-the-loop design is deliberate, because it is what keeps a wrong answer from going out at scale on customer-facing replies.
How do we stop it from making things up?
Two rules do most of the work. First, every draft is grounded in your retrieved help-center content and cites its sources, so your agent can verify it in seconds. Second, when no grounded answer exists, the agent is told to refuse and flag the ticket rather than guess. Grounding plus a refusal path is the difference between a useful draft and a confident fabrication.
What does it cost us in engineering time to own this?
Less than you'd expect, because the agentic tool writes the code. A focused operator can stand up triage and grounded drafting for one ticket category in roughly a week, then widen it category by category. The pipeline is a script and a prompt your team holds and edits in plain English — no consultant on retainer to change a routing rule.
What if our help docs are a mess?
Then you'll see it quickly, and that's valuable. Outcomes here are capped by content quality more than by the tool. The same system that drafts replies produces a ranked list of missing and outdated articles from the questions it couldn't answer, so the cleanup is targeted instead of guesswork.
How do we prove it's actually working?
Pick a baseline before you start: first-response time, the share of drafts agents accept with only light edits, and tickets handled per agent for one category. Compare the same numbers after a few weeks. If drafts are being heavily rewritten, the docs or the prompt need work — the metric tells you before a customer does.
Sources
- 85% of customer service leaders will explore or pilot a customer-facing conversational generative AI (GenAI) solution in 2025 (survey of 187 customer service and support leaders, conducted July–August 2024). — Gartner, 2024
- 61% of leaders say they have a backlog of articles to edit, and more than one-third have no formal process for revising outdated articles. — Gartner, 2024
More from Customer Support
Build an internal support knowledge agent
An agent that answers your support team's questions from your own help docs, macros, and past tickets, with a citation on every answer so reps can verify before they reply. It speeds up handle time and onboarding without putting an unverified bot in front of customers.
Ticket-trend analysis and macro suggestions
Turn your raw ticket exports into a weekly read of what customers are actually contacting you about, then use those clusters to spot missing or stale macros. The output is a report and a script your support team owns and runs itself.
Prep renewals and spot upsell openings
Build a repeatable agent that pulls usage, support, and account data into a per-customer renewal brief flagging churn risk and concrete upsell openings, 90 days ahead of each renewal date.
Want help shipping this?
We'll build it with your team on your real work — and leave you owning it, not renting it.