Talent SummonerReviewing Candidate Evidence from Open-Source Work
A decision guide for comparing open-source work across a close shortlist, separating visible evidence from assumptions and choosing the next check.
Open-source work can make a close shortlist look easier to compare. One candidate links to a repository, another has pull requests, and a third has no public code. Ask what the source shows and what needs checking.
Use it after discovery, when comparing a small set of candidates against the same criteria. It is evidence review, not a reward for public activity.
Freeze the criterion before opening repositories
Write the job-related requirement in observable terms. "Strong open-source contributor" is not a useful gate. "Can review a change, explain a trade-off and maintain production-quality code in a collaborative team" gives the reviewer questions. Classify each part as must-have, nice-to-have or approved boundary before looking at names.
For each criterion, define what evidence could support it and what it cannot. A pull request discussion might support exposure to review and collaboration, but not production ownership, full contribution or current proficiency. A README may describe a project while leaving authorship, operating scale and decisions unknown.
Keep a short record for each source:
Candidate / criterion / role brief version:
Repository, issue or pull request URL:
Date checked and source context:
What the source visibly establishes:
Candidate contribution: direct / shared / unknown:
What remains unknown or conflicts:
Verification question / owner / stop condition:
Decision: progress / verify / hold / close:Compare contribution, not activity totals
Review the work in the same order for every candidate. Start with the artifact most relevant to the criterion, then ask:
- What was the problem? Identify the user, system, constraint or decision in the source.
- What did the candidate do? Separate an authored change from a review comment, issue, documentation edit or team-owned repository.
- What context is visible? Note tests, trade-offs, review responses or constraints the source shows.
- What outcome is supported? Record a stated result and context. If the page only says "improved" or "faster", mark it unverified.
- What cannot be known? Write missing ownership, scope, recency or collaboration detail as unknown.
Use the same labels across the shortlist:
| Evidence label | What the source supports | Next action |
|---|---|---|
| Direct | The described work and material context closely match the criterion, with contribution visible enough to inspect. | Verify the remaining scope or outcome, then compare on the same gate. |
| Adjacent | The work is relevant, but the role, environment or ownership bridge is still material. | Ask one focused question about that bridge. |
| Unknown | The source does not answer the criterion or contribution is not attributable. | Do not add credit or reject; request comparable evidence. |
| Contradicted | A reliable source conflicts with the candidate's stated role or another material record. | Preserve both traces and assign a human resolution. |
These labels describe evidence, not predicted performance. Fewer visible commits may still be stronger evidence; a highly active profile may be unclear without ownership.
Read GitHub signals with their limits
GitHub says a personal profile shows work, contributions and information a person chooses to share publicly. Its contribution reference lists conditions for what appears in the graph and explains that some actions only sometimes count. A graph is a selective activity record, not a measure of capability. A quiet profile cannot show that a candidate lacks experience, and a busy graph cannot establish authorship of every result.
Inspect the repository, issue or pull request where permitted. Look for a participant role, the change, review responses and evidence of the decision the criterion names. Do not infer seniority from stars, followers, employer names, language counts or popularity. Treat dates as context, not a universal freshness threshold. Current availability and interest remain separate.
Licence and privacy boundaries matter. GitHub's licensing guidance says a public repository is not automatically free of copyright restrictions; without a licence, default copyright rules apply. Do not copy code or personal details into a hiring record merely because a page is public. Capture the minimum job-related trace your process permits and follow approved access and retention rules. The ICO's recruitment-AI guidance, published 6 November 2024, highlights lawful basis, transparency, fairness and limiting personal information to what is necessary. This is general information, not legal advice.
Work through a fictional comparison
Suppose the requirement is: review and maintain a shared service, explaining technical trade-offs to collaborators. Candidate A has a pull request that changes the service, responds to review comments and links a test. The evidence is direct for review participation; production ownership and the result still need checking.
Candidate B maintains a personal library with documentation and issue responses, but the repository does not show the service context. That is adjacent evidence. Ask which parts they owned and how they handled a team-owned change. Candidate C appears in an organisation repository, but the page does not identify their contribution. That is unknown. Ask for a specific change they can explain or use another assessment.
Do not force a finer ranking before the same checks are applied. Record the source trace, label, question and owner. If two candidates remain indistinguishable, keep the tie and move to a comparable check.
Where Talent Summoner fits
Talent Summoner is our product. Its current candidate-ranking tool, checked 2 September 2026, accepts a pasted or uploaded job description and up to 50 CVs in PDF, DOCX, MD or TXT format, requires no account and returns a ranked shortlist with plain-English reasoning and shareable reports. Use it to organise CV review; it does not verify repository authorship, interpret licence rights or decide whether evidence meets your standard.
When the existing pool is too small, candidate sourcing, also checked 2 September 2026, starts from a role brief and searches LinkedIn, GitHub and other public professional sources across 200M+ profiles, returning ranked results against must-haves and nice-to-haves with plain-English reasoning. A surfaced profile or repository is a discovery lead, not proof of ownership, availability or interest. Keep the evidence record alongside product output.
Does open-source activity prove a candidate can do the job?
No. It can provide a work trace, but ownership, scope, context and outcomes may remain unknown. Use it to choose a verification question.
Should a candidate without public code be rejected?
No. Work may be private or collaborative. Treat missing public evidence as unknown and use an approved work sample or interview question.
Are commit counts or repository popularity useful ranking criteria?
They provide source context, but do not establish capability, authorship or job fit. Prefer evidence tied to the approved requirement and record what the source cannot show.
Can Talent Summoner verify GitHub authorship or licensing?
No. Our product can discover public-source profiles or organise supplied CVs. Your team must inspect the source, resolve contribution questions, follow privacy and licence rules and decide.
Choose one must-have and review one relevant repository, issue or pull request for each shortlisted candidate. Record one source trace, evidence label and verification owner. Then use candidate ranking for supplied CVs or candidate sourcing when discovery is the gap.
