Chapter 5 of 6
Bias and Representation
What the training data contained, the output will echo.
6 min read
Where bias enters
These systems learn patterns from text written by people, mostly in a few languages, mostly published in particular places. So the output leans towards the dominant voice of that data: whose names sound professional, whose English sounds correct, which histories are told, which examples come first.
What this looks like in a classroom
- Generated example names cluster in one culture.
- Feedback on writing marks multilingual students' correct home dialects as errors.
- Historical summaries adopt the framing of whoever wrote the most.
- Careers, families, and abilities appear in stereotyped combinations.
A short classroom protocol
Ask students to generate ten example characters for a story, then count. Who appears? Who is missing? What does the tool assume when a detail is not given? This is a fifteen-minute activity that teaches more about statistical bias than an hour of definitions.
The teacher's obligation
If a tool influences assessment, its bias becomes your grading policy. Do not let generated feedback stand as a mark. Read it, correct it, and take responsibility for what goes on the page.
Practice exercises
0 of 3 complete1.Bias in these systems mainly comes from…
2.If generated feedback influences a grade, who is responsible for its bias?
3.Run the ten-character activity in your head: what patterns would you expect in the names, jobs and families the tool produces?