Talent SummonerCalibrating Recruiter and Manager Scoring
A decision guide for aligning recruiter and hiring manager scores around observable evidence, shared criteria and a human-owned next step.
Recruiter and hiring manager scores can differ even when both people are acting carefully. One reviewer may read a CV against the agreed requirements, while the other supplies operational context from the team. A different score is a signal to inspect the criterion and evidence, not proof that one reviewer is right.
Calibrating recruiter and manager scoring means agreeing what each criterion means, what evidence supports it and what to do when evidence is incomplete. The aim is comparable reasoning for one role, not identical opinions or an automatic hiring decision.
Define the role before comparing scores
Begin with the approved role brief, not the candidate who triggered the disagreement. List the tasks the person must perform and turn them into observable criteria. Label each as a must-have, nice-to-have or genuine operating boundary. Remove language such as "strong culture fit" unless the team can explain the job-related behaviour it refers to.
The US Office of Personnel Management (OPM) describes job analysis as examining tasks, required competencies and their connection. Its structured-interview guidance recommends selecting critical competencies from job analysis and confirming them with subject matter experts. Use this as general guidance; apply your jurisdiction's requirements and employment rules.
Write an evidence standard beside every criterion. "Has used a modern stack" is difficult to apply consistently. "Describes owning a production service change, including the decision and operating constraint" gives reviewers a narrower question. Define acceptable adjacent evidence and keep the limits visible. Do not add a requirement because a manager prefers one candidate's background.
Create a shared scoring contract
Before reviewing the shortlist, agree the record each person will complete. Separate evidence, interpretation and the action that follows. A compact contract can use these fields:
| Field | Shared question |
|---|---|
| Criterion | What approved task, competency or boundary are we checking? |
| Source | Which CV passage, profile detail or permitted record is being used? |
| Evidence status | Is the criterion supported, adjacent, contradicted or unknown? |
| Narrow interpretation | What does this source establish, and what does it not establish? |
| Score and reason | What local scale applies, and what evidence justifies the rating? |
| Next action | Who will verify, revise, hold or progress this item, and when? |
The scale belongs to your organisation. If you use numbers, define each point and do not present it as an objective measure of candidate quality. A status label is often more useful: supported means relevant context is present; adjacent means a bridge remains; unknown means the material does not answer the question; and contradicted means relevant sources conflict.
Keep the score separate from confidence in the evidence. A candidate may have a strong match to a requirement but low confidence because ownership or context is unclear. Conversely, a clearly documented experience may be confidently adjacent because the role requires a different boundary. The written reason should make that distinction visible.
Run separate first passes
Give the recruiter and manager the same brief, criteria and candidate material. Ask each to record evidence before seeing the other person's score. Include source wording or a precise reference, not only a summary such as "seems senior". Review the same criteria for every candidate.
The recruiter can flag whether evidence maps to the brief; the manager can explain relevant work context. Neither perspective should silently rewrite the requirement. Add any material constraint to the criterion record and confirm it with the decision owner before applying it to candidates.
Do not hide an unknown by guessing what the person probably did. A title, employer name or familiar technology may be useful context, but it does not establish ownership, scope, results or current availability on its own. Record the smallest claim supported by the source and leave the rest for a comparable check.
Diagnose the disagreement
Compare scores criterion by criterion, not as two competing totals. Ask each reviewer to point to the source and state the assumption behind the rating. Common disagreements fit these categories:
| Disagreement | What to ask | Calibration action |
|---|---|---|
| Definition | Are we using the same meaning of the criterion? | Rewrite the criterion in observable terms and version the brief. |
| Evidence scope | Does the source show exposure, contribution or ownership? | Record the narrowest supported claim and keep missing scope unknown. |
| Relevance | Is adjacent experience acceptable for this role? | Agree the bridge required and apply it to comparable candidates. |
| Priority | Are we treating a preference as a gate? | Confirm the class and approval of the requirement before reordering. |
| Missing information | What fact could change the next action? | Ask the same focused question or use the same check for relevant candidates. |
Do not resolve every disagreement by averaging the numbers. An average can conceal that one reviewer found a missing must-have while the other assumed it was present. Resolve the underlying interpretation first. If the criterion remains ambiguous, pause the score, assign the role owner to revise it and create a new brief version rather than editing one candidate's record in isolation.
Turn calibration into a decision rule
After discussing a difference, write one disposition and its reason. Progress means the evidence supports the next stage. Verify identifies a focused question, work sample or permitted check. Hold names a dependency and owner. Revise records a change to the role brief or evidence standard. Close states the approved, job-related basis for ending consideration; a low rank by itself is not that basis.
For unknown evidence, name the fact that is missing, the person responsible for checking it and the stop condition. Use comparable questions for comparable candidates. Preserve the original source and append the new evidence with its date. If the reviewers still disagree after the check, record the disagreement and escalate it to the decision owner instead of treating seniority or confidence of expression as a tie-breaker.
Keep the calibration record with the brief version and shortlist review. Revisit it when a material requirement changes, a new reviewer joins or repeated disagreements reveal unclear wording. This explains changed scores without claiming to predict performance or remove bias.
Where Talent Summoner fits
Talent Summoner is our product. Its current Candidate Ranking tool, checked on 3 September 2026, accepts a pasted or uploaded job description and up to 50 CVs in PDF, DOCX, MD or TXT format. It returns a ranked shortlist with plain-English reasoning and provides a shareable report without requiring an account. Use that report as an input to the recruiter and manager's separate evidence passes; it does not define your calibration scale or make the hiring decision.
When discovery is the gap, candidate sourcing, also checked on 3 September 2026, starts from a role brief and searches LinkedIn, GitHub and other public professional sources across 200M+ profiles. It returns ranked results with must-haves, nice-to-haves and reasoning. Treat a public profile as discovery material to verify, not proof of ownership, interest, availability or fit. Keep the same criterion contract after sourcing.
Why do recruiter and manager scores differ?
They may be applying different definitions, evidence standards or priorities to the same requirement. Compare the source and assumption criterion by criterion before deciding whether the brief or a score needs to change.
Should the recruiter or manager have the final score?
Neither role should win by default. The decision owner should approve the criteria, resolve material interpretation disputes and record the human-owned disposition. A recruiter's process view and a manager's work context are inputs to that decision.
How should missing evidence affect a score?
Mark the criterion unknown, state what is missing and assign a comparable verification step. Do not treat silence as failure or use a title, employer or keyword as proof of unrecorded scope.
Does Talent Summoner calibrate recruiter and manager scores?
No. Our product provides ranked CV results and plain-English reasoning against supplied requirements. Your team defines the scale, compares evidence, resolves disagreements and owns the next action.
For the next shortlist, freeze the role brief, give both reviewers the same criterion record, compare their evidence notes before their totals and assign an owner to every material unknown. Then use candidate ranking for supplied CVs or candidate sourcing when discovery is the gap.
