How AI interview scoring works, and what to ask a vendor
Oct 5, 2026 · 7 min read
Most AI interview scoring follows one pattern. The system takes criteria for the role, finds evidence for each criterion in what the candidate said, and produces a score with an explanation. This guide describes that pattern in general terms and gives you questions to ask any vendor.
How does AI interview scoring work?
It compares the candidate's answers with a rubric. The rubric lists what a good answer to each question should show, and the system rates how well the answer matches. A good tool then explains each rating in words so a person can check it.
People also ask how AI scoring works in video interviews. The principle is the same, but the input differs. In a spoken interview the input is what the candidate said. In some video tools the input also includes the picture, which brings extra risk. The next sections show why.
Where do the scoring criteria come from?
They should come from the role. A job description, a list of required skills and the areas you want covered are the starting point. If the criteria are generic, every candidate is judged on the same vague idea of a good employee, and the scores say little about the job in front of you.
- Job title and level, which set the depth of answer you expect.
- Required skills and experience from the job description.
- Areas to cover, such as customer handling, planning or a specific tool.
- What a strong, adequate and weak answer looks like for each area.
Before you rely on any scores, ask to see the rubric for a sample role. If a vendor cannot show what each score means, you cannot explain the result to a hiring manager.
What counts as evidence?
Evidence is something the candidate said that supports or weakens a score. The transcript is the usual source. A specific example, a number the candidate gave about their own work, a named tool or a clear explanation of a decision all count. A general claim with no example counts for less.
Written reasoning is how you see the evidence. A report that says a candidate scored well on planning, and then points to the part of the answer where they described how they sequenced work, lets you verify the score in seconds. A report with a number and no reasoning asks you to trust it.
What should the report show?
It should show a score for each question or area, with reasoning that refers to the candidate's own words. It should also let you reach the transcript or recording, so you can check the source yourself.
| Report element | What it tells you | Warning sign if missing |
|---|---|---|
| Per-question score | How the candidate did on each area | Only one overall number |
| Written reasoning | Why that score was given | Scores with no explanation |
| Transcript | What was actually said | No way to check the source |
| Recording or clips | How it was said, when a manager wants to hear it | Only a summary |
What should scoring not use?
Scoring should not use accent, appearance, speaking speed, or background noise. None of these predict how well someone will do most jobs, and each can disadvantage whole groups of candidates. Scores should depend on the content of the answer.
- Accent or the way a name is pronounced.
- Appearance, clothing, age, or the room behind the candidate.
- Speaking speed or the number of pauses, unless fluency is a stated job requirement.
- Audio quality or the candidate's device.
- Anything not tied to the role criteria.
Be careful with this question in demos. A vendor can say its tool is fair. What you need is evidence in the report, such as reasoning that refers to the answer and does not mention tone or looks. No tool is free of mistakes, so a person should review any score that affects a decision.
Who makes the decision?
A person should. A score is an input to a human decision, not the decision. A hiring team should be able to read the reasoning, disagree with it, and overrule it. Be cautious with any tool that rejects candidates automatically, because errors then go unseen.
Questions to ask a vendor about scoring
These questions work for any vendor. Ask them in writing and keep the answers.
- What is each score based on: the words of the answer, audio, video, or a mix?
- Can I see the rubric for my role before I run any candidates?
- Does every score come with written reasoning, and can I read it per question?
- Which factors does the scoring ignore on purpose, such as accent and appearance?
- Can I compare the score with the transcript and the recording?
- Can a person overrule a score, and is that recorded?
- Does the tool ever reject a candidate without a person acting?
- How do you check scoring for differences between groups of candidates, and can you describe the process?
The last question should get a description of a process. A vendor that answers with only a promise has not answered it. Treat your own pilot as a check too. Interview a small set of candidates you already know well, and see whether the reports match your judgment.
What does AI interview analytics add?
AI interview analytics usually means looking at results across many interviews. You might compare how candidates score across areas for one role, see which questions separate strong from weak answers, or spot a question that nobody answers well. That last case often means the question is unclear, not that the candidates are weak.
Use analytics to improve your process. Do not use it to rank people in ways that the underlying scores cannot support. A small difference between two scores is rarely a reason to prefer one candidate.
How can you check scoring in a pilot?
Pick five or six candidates whose strengths you already know. Run them through the interview and read the reports without looking at your notes. Then compare. If the scores and reasoning match your view for most people, and the differences can be explained, the scoring is useful. If they differ and the reasoning is thin, ask the vendor why.
Where AI Interview Agents fits
For each candidate, AI Interview Agents (AIIA) gives your team a transcript, an interview report with scores and written reasoning tied to what the candidate said, and a recording when one is available. Short clips of key answers are available too, so a manager can hear an answer that drove a score.
A person on your hiring team makes every hiring decision. AIIA scores and reports, and it does not reject candidates on its own. You can read a public sample report to see how scores and reasoning are laid out before you run anything.