AI, Equity, and Bias: Evaluating Algorithmic Decision-Making

AI, Equity, and Bias: Evaluating Algorithmic Decision-Making

Grades 9–12 · Cybersecurity & AI Education · NYS 9-12.IC.1 · 60 Minutes

NYS-Aligned Standard

9-12.IC.1 — Evaluate the impact of computing technologies on equity, access, and influence in a global society. NYS Computer Science and Digital Fluency Learning Standards (2020)

Learning Objectives — “I Can” Statements

• I can explain how algorithmic decision-making systems are used in high-stakes settings (hiring, lending, policing, admissions).
• I can evaluate how an AI system affects equity, access, and influence for different groups.
• I can recommend changes that mitigate unintended bias and increase fairness.

Essential Question

When an algorithm makes a consequential decision, how do we evaluate whether it is fair — and who decides?

Lesson Sequence

Hook / Warm-Up (8 min)

Prompt: “An AI screens job applications and rejects 80% before a human sees them. Is that fair? What would you need to know to decide?” Capture criteria students raise (accuracy, transparency, who’s affected).

Direct Instruction (16 min)

1. Algorithmic decision-making: a model trained on historical data produces a score/decision.
2. Where bias enters: unrepresentative training data, proxy variables (e.g., zip code standing in for race), feedback loops, and unequal access to appeal.
3. Evaluation lenses: equity (are outcomes fair across groups?), access (who can use or contest the system?), influence (who holds power over its design and deployment?).

Structured Evaluation (26 min)

Teams analyze one original scenario (AI résumé screener, predictive policing tool, loan-approval model, college-admissions predictor) using an Impact Evaluation Frame: intended benefit → who gains access/influence → who could be excluded or harmed → likely source of bias → two concrete mitigations (e.g., audit outcomes by group, human review, transparency, diverse training data, appeal process).

Closure — Evidence-Based Claim (10 min)

Students write a claim: “This system increases/decreases equity because ___ . To make it fairer, designers should ___ , because ___ .”

SDI & Differentiation Block

Supports for MLLs/ELLs

Entering/Emerging (NYSESLAT Levels 1–2):

• Provide the evaluation frame with icons (scale = equity, door = access, megaphone = influence).
• Sentence frame: “This AI helps ___ but could exclude ___ .”

Transitioning/Expanding (NYSESLAT Levels 3–4):

• Pre-teach: algorithm, bias, equity, access, influence, proxy, mitigate, audit.
• Provide the claim scaffold with the first clause modeled.

Supports for Students with IEPs

SDI Adaptation Dimensions: methodology, delivery

• Methodology: Assign one scenario; provide a partially completed Impact Evaluation Frame.
• Delivery: Read scenarios aloud; allow verbal or recorded responses; extend time.

Suggested Placement: ICT

Answer Key / Model Responses

Sample (AI résumé screener): Intended benefit — faster, consistent screening. Access/influence — employers gain influence; applicants have little visibility or appeal. Excluded/harmed — qualified candidates from groups underrepresented in past hires. Bias source — training on historical hiring data that embedded past inequities; proxies like school or zip code. Mitigations — audit selection rates by group, add human review, publish criteria, allow applicants to appeal.

Claim model: “The screener decreases equity because it learns from biased historical hiring and rejects candidates before human review. To make it fairer, designers should audit outcomes by group and require human review of borderline cases, because this catches discriminatory patterns the model would otherwise repeat.”

Alignment Record