AI hiring and screening tool claims: what should buyers ask?
Last reviewed May 30, 2026
When an AI hiring or screening tool claims it evaluates candidates objectively, fairly, and consistently, employers and procurement teams need to know what bias audit records, candidate disclosure practices, and legal applicability analysis back that claim. NYC Local Law 144 requires a public bias audit before deployment. Court cases under the FCRA show that candidates have the right to know an algorithm was used. This page shows what evidence to request before relying on an AI hiring tool's fairness or efficiency claims.
Evidence buyers verify
- A publicly available bias audit report with named methodology, auditor, date, and subgroup performance data.
- Confirmation of whether the tool triggers NYC LL144 or equivalent state-level AEDT obligations.
- An FCRA applicability analysis: whether the tool's output qualifies as a consumer report requiring candidate disclosure.
Opens the checker for this claim type. Paste your vendor's exact wording there. Evidence questions only — not a blacklist or fraud detector. Not sure what a result looks like? See a sample receipt.
Sources this guide draws from
- · July 5, 2023
NYC Local Law 144 enforcement page. Requires annual bias audit before use of AI hiring tools in NYC, public publication of audit results, and 10-day advance notice to NYC-resident candidates.
- · 2023
Official FAQ on what qualifies as an AEDT, who the law applies to, what the bias audit must cover, and how candidate notice must be provided.
- · September 2025
Reports on Kistler v. Eightfold AI (FCRA theory: AI scored applicants without telling them the algorithm existed) and Mobley v. Workday (vendor held as agent liable for age discrimination).
Public claims with documented evidence gaps
"ensuring consistent, fair evaluation for every candidate"
Compliance / Safety- Source and date
- Eightfold AI product page (AI Interviewer) · 2025
- Evidence signal
- Fairness and consistency claim without a visible bias audit record or candidate disclosure process.
- Evidence gap
- NYC Local Law 144 requires that any employer using an AI screening tool in NYC publish a bias audit result before deployment and give NYC-resident candidates 10 business days' notice. A 'consistent, fair evaluation' claim without a public bias audit, stated audit methodology, and demographic subgroup performance data carries an unverified evidence burden.
- Buyer question
- Has the vendor published a bias audit under NYC LL144 or equivalent, and which subgroups and performance metrics did the audit cover?
Match each claim pattern to the evidence buyers need
| Claim pattern | Evidence needed | Buyer question |
|---|---|---|
| Consistent, fair, or objective candidate evaluation | Published bias audit report, audit methodology, subgroup performance data (race, gender, age), auditor independence, audit date, and which protected characteristics were tested. | Where is the bias audit report published, which protected groups were tested, and what were the performance gaps between the best- and worst-performing subgroups? |
| Reduces bias or produces bias-neutral screening results | Bias definition used, comparison to previous process, statistical disparity analysis by protected class, residual bias disclosure, and post-deployment monitoring plan. | Compared to what baseline, and does the vendor define bias the same way our legal and HR teams would in a disparate-impact analysis? |
| AI scales screening or reduces time to hire | Scope of automated steps versus human-review steps, missed-candidate rate, false-positive and false-negative rates, FCRA applicability analysis, and candidate disclosure requirements. | Which steps involve no human review before a candidate is rejected, and has the vendor assessed whether its output triggers FCRA consumer-report obligations? |
| Compliant AI hiring tool or LL144-ready | NYC LL144 bias audit publication link, audit date and auditor name, applicable jurisdictions covered, candidate notice template, and update cadence for the audit. | Where is the publicly available bias audit, when was it last conducted, and which jurisdictions does the compliance claim cover? |
| AI that improves hiring quality or predicts job success | Outcome validation study, time period, job role coverage, comparison to alternative screening method, adverse impact analysis, and whether the vendor's contract caps liability for hiring decisions made using the tool. | What outcome data supports the job-success prediction claim, and does the vendor's contract hold them responsible for discriminatory screening outcomes? |
Evidence to request
- A publicly available bias audit report with named methodology, auditor, date, and subgroup performance data.
- Confirmation of whether the tool triggers NYC LL144 or equivalent state-level AEDT obligations.
- An FCRA applicability analysis: whether the tool's output qualifies as a consumer report requiring candidate disclosure.
- A candidate notice process giving NYC-resident applicants 10 business days' notice before the tool is used.
- Vendor contract terms specifying whether the vendor or employer holds liability for discriminatory screening outcomes.
- A post-deployment monitoring plan showing how bias is tracked after the tool goes live.
Questions to put in front of the vendor
- Where is the published bias audit report, and which protected groups and performance metrics did it cover?
- Does using this tool in our jurisdiction trigger NYC LL144, Illinois AEDT law, or other state-level bias audit requirements?
- Has the vendor assessed whether its output qualifies as a consumer report under the FCRA, requiring candidate disclosure and dispute rights?
- Which steps of our hiring process would involve no human review before a candidate is rejected or advanced?
- Does the vendor warrant that the tool produces non-discriminatory outcomes, or does the contract cap their liability at monthly subscription fees?
- How often is the bias audit updated, and does a new audit run when the model or training data changes?
Wording boundaries to compare against
- Bias audit conducted by [auditor name] on [date] covering [protected classes]; results published at [URL]; audit methodology: [named method].
- Screening step automates [named tasks]; candidate rejection requires human review; NYC-resident candidates notified [X] business days before use.
- FCRA applicability review completed [date]; tool output [does / does not] qualify as a consumer report under [legal basis]; candidate rights: [disclosure/dispute process].
- Time-to-screen reduced from [baseline] to [outcome] for [role type]; human review required before [named decision points]; missed-candidate rate: [X%] on [validation set].
Have your vendor's exact claim wording ready?
Check an AI hiring tool claim How the evidence method works