How often do you encounter race statistics in daily life? From public polls to scientific publications to online debates, there seems to be a widespread societal desire to pin down and compare the traits and psychologies of “racial groups” (I explain the scare quotes below).
This desire is accompanied by an implicit assumption that the survey measurement and analysis practices researchers use to produce race statistics are meant to capture the objective Truth about what different “racial groups” truly believe or feel about a topic.
I question this logic in my recent paper:
what if instead of uncovering the Truth, research methods are merely (re)producing caricatures of the people that we study as “racial groups”?
Rather than measuring the “racial” realities they claim to capture, survey practices may instead be contributing to the process of racialization.
The Paper
Surveys that report psychological race differences typically expose people who are self or externally classified as different race categories to stimuli that they judge on some trait. For example, this could look like rating the trustworthiness of different faces, expressing their beliefs or opinions about a particular topic, or assessing whether a given scenario is racist. Following standard analytic training in the social sciences, responses are often averaged within race categories (each person’s data gets pooled with data from other people who share a race classification) and these race-based averages are then compared across race categories. Observed gaps between race-based averages (statistically significant or otherwise) are then used as evidence that “racial group” differences exist in the trait being measured. These race-based averages are then reported to various audiences (the public, the scientific literature) as if they represent the quantified psychological reality of collections of people who share a race classification. (I describe this standard approach broadly because it is achieved using all sorts of different survey designs and descriptive or inferential statistics).
You’ve likely encountered this in many familiar examples. Reports on “The latino/black/white vote” produce an image that race categories are singular political actors operating on an internally-unified psychology. Similarly, reports on large “racial gaps” in the perspectives of white vs. black vs. non-white people on consequential issues produces an image of society that looks like Neapolitan ice-cream: a racial arena of psychologically sortable, predictable, and bounded blocs of people. Each person’s worldview neatly separated by their predetermined racial lane. My paper examined whether and how this Neapolitan image of society is reinforced by the standard approach (summarizing psychological/survey data using race averages ).
In a series of studies, over 1200 people who self-classified as black, white, or latinx/e/a/o rated how racist they perceived 50 anti-immigrant tweets to be. The tweets were collected in 2017 during the initial Muslim ban and concurrent discussions of building a wall along the Mexico-U.S. border. (Example tweet: “We’re very close to having Pakis banned from America. Love it!”)
Pause for a moment: do you think people mostly agreed or disagreed in their racism ratings, and did any disagreements primarily fall along racial lines — as racial gaps?
I compared the conclusions we would draw from using the standard average comparison approach vs. a variance mapping approach I helped develop. Variance mapping shifts analytic focus from averages to quantifying the extent of variation in ratings and where that variation comes from.
Standard average comparison approach
The following plots show the average (and confidence intervals) of racism ratings provided by participants classified as black, white, or latinx/e/a/o across 4 studies. As often predicted by race talk in social and political psychology, the white sample consistently rated the tweets as less racist on average than the black and latinx/e/a/o sample. Through slippery interpretations of the data, this kind of “racial gap” gets transformed into evidence of a wide & persistent difference between racially grouped cognitions that reflect its members. Because psychology often places the individual as the primary unit of analysis, these averages are then used to make inferences about individuals’ psychologies who are portrayed as “belonging” to each racial category- a wholesale racial capture of a person’s mind. This kind of inferential leap, often driven by a well-meaning impulse in analyses of the psychological impact of racism, may not be supported by the data. Making this leap requires looking beneath category averages to heterogeneity between individuals’ responses. Do these race averages truly represent distinct racial cognitions as standard interpretations suggest? or might there be substantial variation among individuals that gets overlooked?
Variance mapping approach
The following figure shows the variance mapping analysis of the same data used in the average analyses above. It shows the extent to which variation in racism ratings arises from agreement or disagreement among individuals — whether considering all participants together (“All”) or examining subsamples based on shared race categories (“Black,” “Latinx,” “White”). Average analyses that emphasize racial differences would imply that there should be relatively more agreement than disagreement within race categories.
Turns out Around 50% (!) of the variation in ratings — within the full sample and within each race category (!!!)— came from disagreements between individuals’ overall rating tendencies (red bar). For example, I might have a higher overall average because I rate most of the tweets as highly racist (maybe I’m more practiced or have more at stake in perceiving racism). Someone else might rate more tweets as not very racist at all and have a low overall average. Then, around 20% of the variation came from idiosyncratic disagreements in how individuals ranked specific tweets (purple bar). For example, I might share the same overall average rating with someone else but I consistently see one tweet as more racist than another; whereas the other person might perceive the opposite ranking. These disagreements within every race category were as large as when analyzing the full sample together, despite significant average differences between race categories (as shown above). Finally, less than 10% of the rating variation came from agreement (blue bar), within the full sample AND within each race category. (the same pattern appeared across all 4 studies)
Putting it together
Variance mapping revealed that disagreements between individuals overshadowed average race differences. Contrary to standard interpretations, racial gaps in measures of perceptions do not always reflect distinct & divergent group psychologies. Instead, the distinct racial cognitions these gap analyses refer to might simply reflect subsets of individuals within each race category who drive the average differences. This can be obscured by how researchers using the standard average approach interpret and report their results in groupist ways that resemble Neapolitan ice cream.
This groupist tendency then trickles back down to how individual psychology is portrayed, where race averages are transformed into a property of how individuals supposedly perceive the world too. I consider this a scientific form of racialization, which is not the scientific construction of actual races but a scientifically produced illusion that people belong to real races, which, in turn, shapes why they’re analyzed as such. Racialization is reinforced analytically by the overapplication of race averages that are too internally heterogeneous to meaningfully describe the psychology of individuals who must inhabit, accept, refuse, transform, or navigate imposed race categories.
At the end of this process, average caricatures get portrayed as racial truths. This is why I named the paper analytic racecraft (a tribute to Barbara and Karen Fields’ must-read book Racecraft): quantitative survey practices can assume, produce, and act on the illusion that races are real social, biological, or demographic groupings (much like witchcraft practices produce and act on the belief that witches are real).
More examples
Consider two recent papers from respected scientific journals. Both use the standard average comparison approach to report large racial gaps in perceptions of police behavior and of racial discrimination across the decades— both contexts where racial gaps are expected. However, when I reanalyzed their publicly available datasets using the variance mapping approach, their own data told a very different story from the one they published.
The first analysis in this paper explores racial gaps in personal fear ratings of police behavior (searching, yelling, hitting, pepper spraying, tapering, killing). Although they provide visuals of the rating variance, their interpretation instead narrowly focuses on the magnitude of the black-white average race difference. This common analytic decision led the authors to interpret and report the data as evidence that “the American racial divide in police-related fear is not a temporary artifact […] but a persistent social fact in the United States”. My variance mapping re-analysis, on the other hand, revealed huge disagreements in personal fear ratings within each race category — there was basically no within-race agreement at all. There was too much heterogeneity to consider this a “racial divide” (much less a widespread one). To acknowledge this does not deny that fear of police can and does exist among U.S. Americans racialized as black, but places that fact alongside the empirical reality that the magnitude and presence of fear as measured in this paper largely depends on the individual person, it is not overdetermined by the race category someone is classified as.
This second paper examines how people classified as black vs. white perceive the magnitude of racial discrimination in the U.S. across the decades. Throughout this paper (body, abstract, and even the title), the authors repeatedly claim that zero-sum thinking about racial discrimination is a psychological tendency of white, but not black, people (“liberal, moderate, and conservative White Americans alike believe that racism is a zero-sum game with gains for Black people meaning losses for White people […] Liberal, moderate, and conservative Black Americans continue not to see racism as a zero-sum game at all”). Relying on various average-focused analyses, their report emphasizes large racial gaps in perceptions of discrimination and of zero-sum thinking. These statistical gaps serve as the foundation for their theoretical conclusions that people who inhabit different race categories have distinct perceptual tendencies (even if they try to nuance it a bit with political orientation).
Against their interpretations and firm conclusions about racial differences, my variance mapping re-analysis instead revealed that perceptions of general racial discrimination across the decades were largely shared within and across race categories. Agreement between people accounted for 30–50% of the variation, and this agreement transcended racial boundaries. I further show how their analysis of zero-sum thinking relies on racecraft because it is analyzed as a property of the “racial group” (it was calculated between people), rather than of the individual. When I account for this in my re-analysis, zero-sum thinking ends up existing in individuals within both race categories in ways that are more nuanced than their approach allowed them to recognize.
Both papers focused on reporting racial gaps. Yet, one dataset revealed substantial individual disagreements, while the other showed broad agreement across all participants. What are the consequences of framing these studies around large and persistent racial differences? What psychological fictions do we end up believing and psychological realities do we end up missing when we focus on averages over heterogeneity? And how did both papers arrive at the same conclusions despite such starkly different variance patterns in their data?
Rethinking the (social) nature of race and statistics
The problem runs deeper than researchers simply ignoring the variation behind race category averages. I trace these misleading practices to how assumptions about the nature of race shape the focus of analyses and interpretation of findings.
Many social scientists adopt a race realist stance: while race is not biological, it is considered real as a social construct. This perspective underlies research on racial identity, “race relations”, cultural differences, and intergroup prejudice—where racism is framed as the product of antagonisms between real “racial groups”. A common analogy in this space compares race to money: just as physical money becomes valuable and significant through economic practices, races are thought to become real through social practices that infuse race categories with social significance. However, as Adam Hochman points out, this analogy falls apart: “When we collectively agree that money is valuable it becomes valuable, but when groups were racialized they did not take on the key properties they were believed to possess”. In other words, social practices do not create actual races. Races are not entities in the world and do not become real through our race-talk/racecraft. This insight exposes an inverted causal logic: racism is not the result of antagonisms between real racial groups, instead, racist practices generate civic/social/legal double standards between people with different ancestries. These practices and targeted consequences give rise to race-talk and racecraft, which reinforce beliefs in the reality of race(s) and cyclically work to naturalize racism. As Mo Torres succinctly put it: “racism is not discrimination based on race. Rather, race is an ideology created to justify racism. Racism came first and race followed”. With this corrected causality, differential outcomes that are typically understood as the “effect of race” are actually the effect of racist practices that spread beliefs in race.
When researchers treat races as socially constructed and real (through a racist alchemy that equates race with “identity” or “culture”), it would seem intuitive to summarize everyone assigned to a race category with group averages. These averages are often interpreted as capturing the true psychology of a real “racial group”, enabling comparisons between supposedly distinct racial psychologies. This race realism which mistakes race categories for agentic groups is why the two papers above, despite their wildly different variance patterns, can arrive at the same misleading conclusion.
Having lived most of my life as an illegalized immigrant and racialized as latinx in the united states, I understand the appeal of this groupist impulse: unique perspectives do emerge from experiencing a life structured by a specific societally imposed position and people do build self and collective understandings in response to the violence of racism. However, no matter how well-intentioned the groupist impulse is, people who occupy the same societal position (or even the same intersecting positions) are not automatically imbued with those emergent psychologies. Racially-clustered psychologies cannot be expected a priori, their emergence depends on how people react to and are embedded within dynamic infrastructures of racialization, socialization, consciousness-raising, epistemic manufacturing, and other heterogeneous forces (violent and exploitative economies, elite & local political discourse, information & affective ecologies, idiosyncratic life experiences, research methods, individuations, and more). The groupist impulse sidesteps the deeper issue: race is an ideological fiction, one that our research practices help to maintain.
Scholars like Barbara and Karen Fields have shown how beliefs in race are continually re-created in everyday societal and scientific practices, while scholars like Sylvia Wynter and Denise Ferreira da Silva have shown how modernity itself is structured by ongoing material and symbolic orders that (re)produce and (re)define race and The Human. Without undoing this global epistemic structure that compels us to see and treat each other as racial beings, our attempts to analyze and counter racism often revert to reinforcing racecraft, or what da Silva calls the racial dialectic. In this process, we repeatedly perform a conjuring trick: transforming “racism, something an aggressor does, into race, something the target is, in a sleight of hand that is easy to miss”. The above papers are examples of this – but they are far from the only ones. This conjuring trick is everywhere (once you know to look for it)!
What researchers ultimately miss is how deeply our methods are entangled in the same epistemic & material orders that produce and sustain racial thinking. Just as political and social institutions racialize people, so do our scientific practices… especially when we fail to interrogate & disconnect our own research methods from their colonial histories & political economies. By relying on uninterrogated methods, we become active participants in the racialization process, shaping the very knowledge that is then used to define and govern social life. This is not an neutral act of survey measurement—it is an act of creation. Through colonially-inherited quantification, we extend the social domains where people get sorted into consequential race categories, reproducing the ideologies we claim to study & resist.
I hope this motivates researchers to look inward at how they may be entrenching racecraft in their fields, and do the hard work of imagining & developing approaches that better account for their political embeddedness and the complexity of how people make sense of the world. A world where power & perceptions are dynamically (re)formed.
To learn more about this perspective, check out my related papers: