--- blogpost: true date: August 19th, 2026 author: Jessica Woodgate category: Write Up tags: AI, ML, bias, data feminism, data labelling, fairness, intersectionality, history, racism, standpoint theory --- # Data Ethics Club: [The Categorization of Race in ML is a Flawed Premise](https://proceedings.mlr.press/v267/doh25a.html) ## Article Summary Machine learning (ML) research often relies on datasets that label race as a categorical variable, treating it as a “ground truth” feature that can be predicted. Similarly, algorithmic fairness works tend to characterise race as a protected attribute. A lot of this research relies on predefined classifications without critical engagement or justification. However, the concept of separate human races is anthropologically contested. Scholars across disciplines have emphasised that racial classifications are neither genetically discrete nor reliably measurable and are better understood as social constructs than biologically discrete entities. Presenting racial categories as computer-readable attributes gives the appearance that they are measurable and biologically real, as well as entrenching them in digital infrastructures. The use of racial categorisation in algorithmic fairness research builds on the US-centric doctrine of disparate treatment or impact. Unlike the US, legal and institutional frameworks in Europe tend to avoid racial classification and restrict the collection of ethnic or racial data to narrowly defined conditions. The paper argues that, in line with European standards, racial categories should be abandoned in ML research whenever possible. There are two key problems with racial categorisation in ML: firstly, using it means that ML frameworks fail to represent mixed-race identities; and secondly, it enforces reductive classifications. Mixed race individuals navigate multiple cultural and racial contexts which singular classification cannot capture. Reductive classification occurs via the process of transforming abstract ideas (such as race) into concrete entities (such as programmable features). In response to these issues, the paper proposes that ML research should abandon racial taxonomies and focus on developing alternative methods, such as using composite ethnicity variables that combine a range of classifications or context-specific mapping of individual characteristics into a racial space. Participatory AI methods, which require direct collaboration with affected parties and domain experts, also help support fairness interventions to be effective and contextually grounded. ## Discussion Summary ### How is race currently treated as a variable in datasets you have seen/used/created? Race labels exist in datasets we’ve worked with but are often messy. In ML research, we have seen race used in the way that the paper describes, where categories are applied uncritically and without question. In clinical settings, we’ve encountered datasets with race labels such as “black/white/other”. We found that a model trained on this data performed well for “black” and “white”, but not for “other”. We’ve also worked with image datasets containing skin tone labels, some of which only existed for a subset of the data and did not correspond well with the actual images. In other cases, we have seen proxies like language, location, or name used but no explicit label of race. The way race is handled depends on the domain. In quantitative social science research in Brazil and the US, it is normal to consider and measure race. The data is often acquired through self-identification. Qualitative sociology research views the concept as flawed but still in use. A critique raised in this domain is that race is sometimes absorbed into national identity and is quite tied to racial inequality. Sociology of health remains unclear about which part of race categorisation is biologically helpful. Social work data frequently incorporates race as a category, but the way it is categorised can change depending on whether the categories are decided by the government, local bodies, censuses, or something else. Within government itself, there is often a hesitation to create racial categories and a patchy view over what the correct approach is. ### How should fairness benchmarks be adapted when deploying models across different geographic, legal, and socio-cultural settings? Racial categories don’t necessarily line up across contexts. Differences in the history and perception of race mean that results from one country will not necessarily be comparable to another, for example, in the US compared to Australia. In the US, there is a sense in which it is reasonable to measure race because of historical segregation, where those that are mixed race were considered (legally) black. In places like Brazil, measuring race is more complicated due to government [racial whitening policies](https://en.wikipedia.org/wiki/Racial_whitening). Race is not a universally defined concept, so it is impossible to collect it perfectly. The same race column in different datasets might represent different things, possibly as a result of collapsing and aggregating a large amount of information into a single column. Diversity in how race is understood over different settings means that signposting and explaining what categories were used, why, and how, is crucially important, especially if categorisation has been conducted using AI. People have very different problems in mind when they think about race. Datasets are shaped by how they were created, as [all forms of knowledge are situated insofar as they are produced by specific people in specific circumstances](https://dataethicsclub.com/write_ups/DataFeminism_writeup.html#id5). We’d like to see authors in ML research and fairness benchmarks describe the racial categorisations that they are using and why they have chosen them. A standard declaration as part of the methods section could be an appropriate way to address this. Doing so would help make the framework more clear and enable greater comparability across contexts. This draws parallels to the Data Feminism chapter “[What Gets Counted Counts](https://dataethicsclub.com/write_ups/DataFeminism_writeup.html#data-feminism-chapter-4-what-gets-counted-counts)”, which highlights the importance of being clear about the context within which research has been created and who has been included in it. ### Does the failure of discrete categories for mixed-race identities point to a fundamental limitation of classification algorithms in social domains? Disagreements over racial categorisation have existed as long as the modern concept of race. Race is not the only contested concept, as related features such as ethnicity are also highly questioned, as well as whether there is even a meaningful difference between these concepts. These ideas are shaped by institutional and historic racism. Flaws in how ML research handles race go beyond fairness concerns to implications such as comparability and generalisability. Technologists like to try and squash complexity into neat variables that can be manipulated. When people from non-diverse backgrounds are the ones making decisions about what is relevant and what isn’t, [they often end up suppressing important features which don’t fall into their own experiences](https://dataethicsclub.com/write_ups/DataFeminism_writeup.html#data-feminism-chapter-5-unicorns-janitors-ninjas-wizards-and-rock-stars), such as aspects of identity that are vital for the self-identification of diverse groups. Self-identification can function as a decent variable and is favoured in social science, but it is not always perfect and can be very noisy. The labels that are used don’t always reflect the experience of the person behind it. Those of us that are mixed race feel like our experience can’t really be captured through ML. Even if there were appropriate categorisation mechanisms, we aren’t sure when it would be necessary to have a precise model of the complexity. ### What change would you like to see based on this piece? Who has the power to make that change? Reading the paper, we were sometimes torn between the desire to find efficient technical solutions and the challenges that come along with unpacking the complicated concept of racism. Race-related data does have uses, such as in areas around disproportionality. Some of us have experienced the impact of training data which isn’t diverse; incorporating notions of race can help mitigate this. When examining disproportionality, race is just one element. It’s vital to look at a range of intersections, such as immigration, religion, poverty, class, and language. It seems that many proposed solutions tend to rely on “human in the loop”, however, what this exactly is meant by this remains unclear. Decisions that are made about how to categorise people have tangible impacts. Today, AI technologies that exacerbate biases are ubiquitous. It is one thing to create models of the world and mess about with coefficients, but making decisions about individuals means that some outcomes will be unfair and impact real peoples’ lives. The paper hints at societal impacts of race and racism but focuses more on ML technicalities. Whilst this positioning may be appropriate for the paper’s venue (an ML conference), it’s important to think about why the current categories look the way they do. This in turn necessitates a discussion of racism. ## Attendees - Huw Day, Data Scientist, University of Bristol: [LinkedIn](https://www.linkedin.com/in/huw-day/), [BlueSky](https://bsky.app/profile/huwwday.bsky.social) - [Jessica Woodgate](https://jessica-woodgate.github.io/), PhD Student, University of Bristol - [Melanie Stefan](https://melaniestefan.net/), Computational Neuroscientist, Medical School Berlin - Joe Carver, Data Scientist, Brandwatch - Vanessa Hanschke, Researcher in XAI, University of Bristol