Conversations and Quality
The Conversations page shows every chat conversation between Jarvis and your website visitors. The Quality page gives you metrics on how well Jarvis is performing. Together, they help you understand what visitors are asking, how Jarvis is responding, and where your knowledge base needs improvement.
Conversations Page
Navigate to Widget Conversations in the sidebar to see all visitor chat conversations.
Searching and Filtering
The conversation list supports:
- Status filter - view All, Active (ongoing), Ended, or Escalated conversations
- Date range - filter by when the conversation happened
- Flagged - toggle to see only conversations you have flagged for review
- Search - find conversations by keyword
Reading a Conversation
Click any conversation in the list to open the detail panel. You will see:
- Conversation metadata - when it started, when it ended, number of turns (back-and-forth messages), and visitor email (if provided)
- Resolution outcome - how the conversation ended: Resolved, Escalated, Abandoned, or Active
- Message thread - the full conversation between the visitor and Jarvis
Visitor messages appear on the left side. Jarvis's responses appear on the right. If Jarvis used any tools during the conversation (like searching your knowledge base), you can expand the tool call details to see what it looked for and what it found.
A conversation detail panel showing visitor messages on the left and Jarvis responses on the right
Flagging Conversations
If you spot a conversation that needs team attention - maybe Jarvis gave an incorrect answer, or a visitor had a particularly complex question - click the flag icon to mark it. Flagged conversations can be filtered in the list so your team can review them.
Rating Conversations
Each conversation has a 5-star rating you can set. Use this to track quality over time. Click on a conversation where Jarvis performed well and give it 5 stars; conversations that need improvement can be rated lower. These ratings feed into your quality metrics.
Admin Notes
You can add notes to any conversation for internal context. Click "Add notes" in the conversation header to write a note that is visible only to your team - visitors never see admin notes.
Quality Dashboard
Navigate to Quality in the sidebar for an overview of Jarvis's performance.
Key Metrics
The dashboard shows several performance indicators:
Resolution Rate - the percentage of conversations where Jarvis resolved the visitor's question without needing human intervention. A healthy resolution rate depends on your content coverage, but generally:
- 70% or above is strong
- 50-69% means there are opportunities to improve your knowledge base
- Below 50% suggests significant content gaps
Escalation Rate - how often conversations are escalated to a human. Some escalation is normal and expected (complex questions, edge cases). A low escalation rate means Jarvis is handling most things on its own.
Average Turns - the average number of back-and-forth messages per conversation. Fewer turns generally means visitors are getting answers quickly.
Average Response Time - how quickly Jarvis responds to each message. This is usually very fast (a few seconds).
If you have fewer than 10 conversations, the quality metrics will show as "Insufficient data" rather than percentages. This is intentional - with a small sample size, percentages can be misleading. Once you have enough conversations, the colored indicators will appear.
Understanding Quality Flags
Jarvis automatically flags certain conversations that may need attention. These flags are tools for improvement, not alarms:
Possible inaccuracy - Jarvis may have said something that is not directly supported by your knowledge base. This does not necessarily mean the answer was wrong - it means you should check it. The most common cause is content gaps: the visitor asked about something your knowledge base does not cover well enough.
Visitor needed more help - the conversation shows signals that the visitor was not fully satisfied. Maybe they rephrased the same question multiple times, or the conversation ended without a clear resolution. Review these to see if better knowledge base content would have helped.
Repeated responses - Jarvis gave similar answers multiple times in the same conversation. This usually means the visitor was asking follow-up questions that Jarvis could not differentiate from the original question. Adding more detailed content on the topic usually resolves this.
Quality flags are normal, especially when you are first starting out. They are feedback tools, not failure indicators. Every flag is a signal about where your knowledge base can be improved. Over time, as you add content in response to these signals, the flags will decrease.
Knowledge Base Coverage Gaps
Navigate to Knowledge Base > Coverage Gaps to see topics that visitors asked about but your knowledge base could not fully answer.
Each gap shows:
- The query - what the visitor asked
- Category - the topic area
- Frequency - how often this gap has been encountered
- Best match score - how close the nearest knowledge base content was to answering the question
- Suggested topic - a recommendation for what to add to your knowledge base
Acting on Coverage Gaps
Review open gaps
Look at the gaps sorted by frequency. The most common gaps are the highest priority to fill.
Create knowledge base content
For each gap, create a new knowledge base article (or update an existing one) that covers the topic.
Mark as addressed
Once you have added the content, mark the gap as "Addressed." This removes it from the open list so you can focus on remaining gaps.
Coverage gaps are the fastest way to improve Jarvis's resolution rate. Each gap you fill means better answers for future visitors who ask the same question.
Building Quality Over Time
Quality improvement with Jarvis is an ongoing cycle:
- Review conversations - spot patterns in what visitors ask and how Jarvis responds
- Check quality flags - investigate conversations that were flagged
- Fill coverage gaps - add content for topics visitors ask about
- Rate conversations - track improvement over time with star ratings
- Refine personality - if the tone or style is not right, update your agent personality
The first few weeks are the most impactful. As you respond to quality signals and fill gaps, Jarvis gets noticeably better at handling your specific visitors and topics.
What's Next
To manage your account, team, and plan, head to Account and Billing.
Was this page helpful?