Adverse-Impact Questions for Small Teams

When a small hiring team sees different progression rates between groups, the first task is to troubleshoot the process, not announce a finding. This checklist helps define the stage, preserve the denominator, investigate data quality and decide who can pause the workflow. It is general educational information, not legal advice. Rules vary by jurisdiction; get qualified advice for the relevant context.

Start with the exact selection question

Write down the role, version of the brief, selection stage and outcome before looking at group rates. A sourcing pool, CV shortlist, interview invitation, offer and hire are separate outcomes. Ask:

What is the unit: person, application, CV, interview or offer?
What is the entry rule and what exactly counts as a positive outcome?
What changed in the role brief, reviewer instructions, ranking configuration or threshold?
What dates define the observation window, and who owns a pause decision?

The OPM Assessment Decision Guide, issued 6 June 2007, connects job analysis, assessment content, administration and evidence. Ask whether each criterion measures a documented job requirement. The EEOC's Employment Tests and Selection Procedures guidance, issued 1 December 2007, is US guidance and does not decide a team's legal position elsewhere.

Make cohorts and denominators visible

For every stage, record eligible count, positive count and selection rate for each approved subgroup, plus the comparison group. State the inclusion rule, duplicate handling, time window, role family and whether a person can appear more than once. Do not compare a pilot's top-10 shortlist with a full year's hires as one cohort.

Preserve missing records and exclusions. Ask whether people without an optional attribute were excluded, placed in an unknown bucket or included in the total. Record its purpose, access controls and deletion or review date. Do not infer a sensitive trait from a name, photo, school, postcode, accent, language or public profile. If the denominator cannot be reconstructed, label the result UNKNOWN.

Treat four-fifths as a screening signal

The four-fifths ratio is the selection rate for one group divided by the highest selection rate. A result below 0.80 can prompt investigation, but it is not a legal conclusion and a result at or above 0.80 does not prove that a process is fair. The EEOC's 2023 Annual Performance Report, published 23 February 2024, describes the agency's May 2023 technical assistance and cautions against treating the rule as a guarantee.

Ask what else could explain the signal: small cells, missing data, job-relevant experience, application channel, stage timing, reviewer assignment, accessibility barriers, a changed threshold or a confounded cohort. Report counts beside rates and record an uncertainty method or why a result is suppressed. A ratio invites inspection; it does not permit labelling a group or automatic rejection.

Check privacy, accessibility and human review

Use the minimum subgroup data needed for the review. The ICO's recruitment AI considerations, dated 6 November 2024, highlights purpose, minimisation, transparency, responsibilities and impact assessment. Restrict access, separate identity from analysis where practical, document provenance and retain only for the approved period.

Review whether the process was accessible. WCAG 2.2 became a W3C Recommendation on 12 December 2024 and provides testable web-content criteria. Check keyboard access, focus, text alternatives, contrast, zoom, timing, captions and the route for requesting an adjustment. WCAG conformance does not prove a fair hiring outcome. A person must review job-related evidence, overrides and unanswered questions; a tool output must not silently become an automatic rejection.

Use the NIST AI RMF 1.0, published 26 January 2023, as a voluntary structure for governing, mapping, measuring and managing risk, not a certification. Assign process, data or privacy, and decision owners. Record evidence, open question, action, rollback or manual fallback, due date and monitoring date.

Apply a clear gate and correction path

EVIDENCE: stage, outcome, cohorts, denominators, data source and owner are documented.
UNKNOWN: a label, subgroup, missingness pattern, accessibility condition or confounder is unclear.
HOLD: a material question needs an owner, approved data decision or re-test before use continues.
STOP: an unapproved sensitive proxy, exposed data, inaccessible route, automatic rejection or
       missing accountable owner requires containment and manual review.

Corrective action may mean revising a job-related criterion, checking collection consistency, changing a threshold, retraining reviewers, adding an accessible route or reverting to a prior configuration. Keep the original result, record the change and re-test the same defined cohort where possible. Do not promise that one correction removes all bias.

Fictional example

In this fictional example, a four-person team reviews 80 applicants for a data analyst role. At the interview stage, group A has 10 positive outcomes from 50 eligible applicants and group B has 3 from 20; 10 records have no approved subgroup value. The team does not publish a disparity finding. It first checks whether the 10 unknown records were systematically excluded, whether one reviewer handled most group B cases and whether the interview task had an inaccessible timed component. Because the missingness and accessibility conditions are unresolved, the gate is HOLD. The team uses a manual review, assigns owners, preserves the data decision and sets a re-test date. This is an example, not a benchmark.

Where Talent Summoner fits

Talent Summoner is our product for candidate sourcing and CV ranking. Candidate sourcing starts from a role brief and searches public professional sources; candidate ranking organises supplied CVs for human review. Our product does not establish legal compliance, create approved subgroup data, administer adjustments, manage an application pipeline or automatically reject candidates. Your team owns the criteria, evidence, privacy decisions and final outcome.

Use the checklist for one defined stage at a time, and keep legal, privacy and accessibility questions with the qualified people responsible for them.