The QAs are right most of the time. But, there are chances they might misinterpret a conversation. The same applies for new QAs as well. In those cases, the right feedback should come from the agents whose conversations were evaluated. This will help the QAs understand what went wrong and will provide them an opportunity to learn from a conversation. The process of managing quality disputes is commonly referred as ‘rebuttal’ or ‘dispute management’
Level AI has a dispute management feature built within to bring agents and QAs on the same page.
If an agent is not satisfied by the QA evaluation, they can raise a dispute within Level AI. They can add comments to specific sections of the rubric and it will be forwarded to the QA who evaluated the conversation to initiate further discussion.
Raising a Dispute
Agents can raise a dispute from in the following ways:
- When they log into their Level AI dashboard, they’d be able to see the total number of their conversations evaluated by the QA team.
- They can click on the total evaluations widget, go through the list of conversations and raise a dispute for the conversation of their choice.
- When agents raise a dispute, they can also add comments mentioning the specific reasons behind the dispute.
- Once you add comments, click on the Dispute button.
Fixing the Dispute
If you’re a QA, you can get notified when an agent raises a dispute. You can review the disputes, read the comments added by the agents, and add your comments.
To view disputed conversations, click on view disputed evaluations from the dashboard.
You can click on a disputed evaluation to see the agent’s comments and add your comments/re-evaluate the question(s) and click on the Update button Agents can review new comments/scores and can then accept the evaluation.