On this page
The short version#
- You write the scorecard before anyone is interviewed: 4 to 6 skills, each with 4 written levels.
- Every skill score must quote the candidate’s own words, with the time in the interview. No quote, no score.
- The overall is a weighted average you set, shown out of 4. A must-have below 2 fails, whatever the average.
- A second AI model, from a different company, re-scores each interview without the candidate’s name. When they disagree on a must-have, a person decides.
- A person on our team reviews every finalist packet before you see it. You make the hiring decision.
You write the scorecard first#
Scoring starts before the first interview. When you create a role, an intake chat helps you write the scorecard. It pushes back on weak proxies like a big company name, an elite school or years of experience, and asks what the person must actually be able to do.
| Part | Rule |
|---|---|
| Skills | 4 to 6, set by you |
| Levels | 4 written levels per skill, from 1 to 4 |
| Weights | Whole numbers from 1 to 5 |
| Must-haves | At least one skill is a must-have |
| Knockouts | Up to 5 short yes or no checks |
| Pass line | From 2.0 to 3.5, in steps of 0.25 |
Each level is a sentence, not a number. For a skill like debugging, level 1 might read “Changes code before finding the cause” and level 4 “Narrows it fast and proves it with a test.” The AI picks the level that the evidence supports.
Every score needs a quote#
This is the rule the rest of the system is built on. For each skill, the AI must cite the candidate’s own words from the transcript. If it cannot, the skill is not scored.
- A quote must be at least 15 characters and an exact match to one part of the transcript. Case and punctuation must match.
- The time stamp comes from the transcript, never from the model.
- The interviewer’s own words cannot be evidence.
- Up to 4 quotes per skill.
- No valid quote means “insufficient evidence”, not a low score.
I would put the idempotency key on the request, not the row, so a retry after a timeout lands on the same ledger entry instead of charging the customer twice.
When you read the packet, you see the quote, the time and a short video clip at that moment. You can check the score yourself in under a minute.
How the overall score is computed#
Each skill gets a whole number from 1 to 4. The overall is a weighted average: each skill counts as much as its weight. It is rounded to 2 decimals and shown with one decimal, out of 4.
| Skill | Weight | Must-have | Score |
|---|---|---|---|
| Idempotency | 4 | Yes | 4 |
| Ledger design | 2 | No | 4 |
| Debugging | 4 | Yes | 3 |
| Communication | 2 | No | 3 |
overall = sum(score × weight) / sum(weight) # Example candidate, weights 4, 2, 4, 2 = (4×4 + 4×2 + 3×4 + 3×2) / (4 + 2 + 4 + 2) = 42 / 12 = 3.50 # shown as 3.5 / 4
With a pass line of 3.0, this example candidate passes: the overall is 3.5 and both must-haves are at 3 or above.
A must-have below 2 fails#
A high average cannot carry a missing basic. If any must-have skill scores 1, the candidate fails, even with 4s everywhere else. This is the rule founders ask about most, and it is on purpose.
When a role round is scored#
After the voice intro, most roles have a role round: live coding, a written case, a design walkthrough with screen share, or a sales role-play with an AI buyer. When that round is scored, its composite replaces the stored overall. The composite mixes parts such as tests passed, code quality and your scorecard skills, with weights you can change. Parts with no evidence drop out and the rest are rebalanced. The result is mapped back to the same scale out of 4.
A second model checks the first#
One AI model scores the interview. A second model, from a different company, scores the same interview again with the candidate’s name replaced by “Candidate”. A person reviews the interview when the two disagree in a way that matters:
- a skill score differs by 2 or more,
- one says pass and the other says fail,
- they land on different sides of 2 for a must-have,
- or the evidence fails the quote check.
What a person still decides#
A person on our team reviews every finalist packet before you see it. Every packet includes at least one reason to hesitate, so you are not only shown the good parts. You choose who becomes a finalist, and the hiring decision is always yours.
What we do not claim#
AI can be wrong. That is why every score cites a quote you can check. We run an internal name, gender and language check while we build. It is not a bias audit, and we do not call it one. Read the evidence before you decide.
