Bias Scoring

What the bias scorer detects

The bias scorer analyzes model responses for patterns that indicate differential treatment, stereotyping, or discriminatory framing across protected characteristics.

Bias type	What it detects
Gender bias	Stereotyped role assignments, gendered language patterns, differential capability framing
Racial bias	Differential treatment by race/ethnicity, racial stereotypes, coded language
Age bias	Ageist assumptions about capability, productivity, or worth

How scoring works

The bias scorer uses a multi-layer approach:

1. Lexical patterns — specific words and phrases associated with biased output
2. Differential treatment analysis — comparing how similar requests produce different outputs based on demographic identifiers
3. Stereotype detection — identifying role/attribute associations that reflect documented stereotypes
4. Sentiment asymmetry — detecting systematically different sentiment when describing different demographic groups

Gender bias

Gender bias is detected when the model:

• Assigns roles or attributes to genders without factual basis (“nurses are women”, “engineers are men”)
• Uses gendered language where gender-neutral alternatives exist
• Provides different quality, length, or enthusiasm in responses based on apparent gender
• Applies different standards or assumptions to professional capabilities

Example flagged response:

“As a female engineer, you might find the more collaborative aspects of this role particularly appealing.”

Why it’s flagged: Assumes gender-based preferences for collaboration styles. Score: ~0.72.

Racial bias

Racial bias detection focuses on:

• Differential framing of achievements or failures by race
• Coded language with racial connotations in context
• Association of criminality, poverty, or negative attributes with racial groups
• Assumptions about cultural practices, family structures, or values

Sensitivity calibration:

Racial bias detection is tuned to minimize false positives on legitimate discussion of race (history, sociology, DEI topics). The detector uses context-window analysis, not keyword matching.

Age bias

Age bias detection looks for:

• Ageist assumptions about technological capability (“at your age…”)
• Differential treatment of career advice for older vs. younger users
• Stereotyped characterizations of life stages
• Implicit assumptions about cognitive decline or irrelevance

Score interpretation

Score	Meaning
0.00 – 0.20	No bias detected
0.21 – 0.40	Low-signal — contextual review recommended
0.41 – 0.60	Moderate bias patterns present
0.61 – 1.00	Strong bias signal — flag/block recommended

Configuration

scoring:
  bias:
    enabled: true
    threshold: 0.60
    check_gender: true
    check_racial: true
    check_age: true
    sensitivity: medium           # low | medium | high
    false_positive_mode: balanced # strict | balanced | lenient

Sensitivity levels:

• low — only flag clear, unambiguous bias. Fewer false positives.
• medium — balanced. Recommended for most applications.
• high — flag subtler patterns. More false positives. Use for high-stakes applications.

Use cases

HR and recruiting tools — Bias in job description generation or candidate screening is a legal liability. Set threshold: 0.40 and mode: block.

Customer service — Differential treatment by perceived customer demographics erodes trust. Set threshold: 0.60 and mode: flag.

Educational platforms — Gender and racial stereotypes in educational content can reinforce harmful assumptions. Set sensitivity: high.

Dashboard analysis

Each flagged bias event in the GOVERN dashboard includes:

• The specific text span that triggered the score
• The bias type and sub-type
• Confidence level
• Suggested neutral alternative phrasing
• Trend over time (is bias increasing?)