Skip to content
English - United States
  • There are no suggestions because the search field is empty.

Navigating Scenario Studio: Rubric Considerations

This article provides some best practices to consider when creating a Rubric. 

Once you have crafted and fine tuned your Scenario Sample Conversation and you have optimized your prompt accordingly, the next step in the process is to create a Rubric, against which your Learners' simulations will be scored. While Rubrics are optional, this is a powerful way to assess your Learner's performance based on Skills and Criteria specific to the conversation being simulated. Whether or not you use Generate Rubric, the following will help guide you to create an applicable Rubric for your specific Scenario. 

  • Recognize that a Rubric is comprised of Skills comprised of Criteria. Criteria are binary. That is, the Learner either did or did not demonstrate in the simulation of the Scenario specified criteria.
  • Criteria must be phrased in the form of a question. For example, Did the Learner express empathy?
  • A Rubric works best with specific Criteria that can be met via a simulation of the associated Scenario. If a Criteria states, “Did the Learner present correct and complete information to the customer,” the Rubric will work best if that Criteria is more specific, based on what is crafted in the associated Scenario. You will need to modify that Criteria to be specific in this regard by stating precisely what constitutes said correct and complete information in the Scenario. For example, if the Scenario was crafted to provide for a simulation of a conversation which includes information about a life insurance policy for seniors in the amount of $10,000, then the Criteria should reflect such information, as follows (PURELY an example), “Did the Learner inform the customer regarding life insurance policy for seniors in the amount of $10,000?” 
  • A Rubric can only evaluate what is said in the simulator. A Rubric cannot evaluate any action which is conducted outside of Call Simulator, such as Update the CRM Record and After Call Work.
  • Any uncommon or domain-specific terms should be defined in the Rubric Context. For example, an acronym which might be uttered in the simulation and is specific to the associated Scenario should be clearly defined in Rubric Context, to help the Rubric properly score the simulation. 
  • Replicating your organization's QA Scorecard can best be accomplished through using Generate Rubric, and then compare what Call Simulator suggested could be evaluated based on the associated Scenario against your QA Scorecard. Remember, your organization's QA Scorecard is typically nonspecific because the conversation in a live call (or recorded live call) is unknown. Whereas, the conversation you have crafted in the Scenario Sample Conversation is known. Adjust the Skills and Criteria to align with your QA Scorecard but only to the extent that the Scenario Sample Conversation allows your Learner opportunity to exemplify said Criteria.

If you have any further questions regarding Rubric, Skills, and Criteria, please feel free to reach out to Support@CallSimulator.com. We are happy to help!