Responsible Language Model Design and Personalization for Minority Users
Block by Block: Research at Work is a research spotlight series that highlights the innovative work being done by CMU researchers through the Block Center, showcasing how their projects are driving impactful solutions at the intersection of technology and society.
During a recent conversation with the Block Center, Dr. Maarten Sap discussed how his research on Responsible Language Model Design has evolved since we last spoke, thanks in part to the hard work of students.
Responsible Language Model Design and Personalization for Minority Users
Imagine if AI could adapt to you. Learning your preferences, your communication style, even your dialect, and responding in a way that feels natural and familiar. When Dr. Maarten Sap posed this idea to his students in his Spring class "Ethics, Safety, and Social Impact in NLP and LLMs", their reactions differed significantly. One Indian student welcomed the idea of AI adapting to her dialect and background, while an African American student expressed discomfort with the idea of AI learning more about her or attempting to mimic her identity.
Dr. Sap describes this moment as a catalyst for his research on the cultural adaptation of AI. As he explained, “there’s a clear tension between personalization and profiling or stereotyping, and that tension is shaped by the relationship individuals have with dominant social groups.” For some users, particularly those whose cultural norms are underrepresented in AI systems, adaptation can feel empowering. For others, especially those from historically marginalized groups, it can raise concerns about bias and misuse. These differing perspectives highlight how power dynamics shape user experiences with AI systems.
Seeking to approach cultural adaptation in an ethical and responsible way, Dr. Sap launched Responsible Language Model Design and Personalization for Minority Users, a project funded in part by a Block Center Seed Fund.
To further explore cultural differences in these perspectives, Dr. Sap expanded this work into three recent papers: “Not Like Us, Hunty: Measuring Perceptions and Behavioral Effects of Minoritized Anthropomorphic Cues in LLMs”, “Perceptions in Black American English Interactions with LLMs“, and “Rejected Dialects: Biases Against African American Language in Reward Models”. Together, these studies examine how large language models (LLMs) interact with minoritized language and identity cues, and how users perceive and respond to those interactions.
The first paper investigates how LLMs adopt stylistic cues associated with minoritized communities (such as African American English) and how these cues influence user trust, engagement, and perceptions of authenticity. The findings reveal a complex dynamic. While some users view these adaptations as relatable or engaging, others perceive them as inappropriate or even harmful, particularly when they appear to mimic identities without lived experience. This underscores the tension between personalization and stereotyping in AI design.
The second paper investigates how multi-turn interactions with Black American English (BAE) producing LLMs affect not just how BAE speakers perceive the model, but how they perceive themselves. Studying both speech-based and text-based conversations, the researchers found a significant change in participant self-esteem following interactions with a BAE-producing LLM, along with notable differences between BAE LLM and Standard American English (SAE) LLM interactions.
The third paper examines how reward models (used to evaluate and fine-tune AI outputs) may systematically disadvantage African American English (AAE). The research shows that outputs written in this dialect are more likely to be rated as lower quality or less appropriate, even when they convey the same meaning as standard English. These results highlight how existing social and linguistic biases can become embedded in AI systems, reinforcing inequalities at multiple stages of development.
Throughout our conversation, Dr. Sap emphasized the important role students play in advancing this work. Reflecting on the impact of Block Center support, he shared how seed funding enabled him to bring on a student researcher:
“I used Block funding to support Mikayla as an intern for this project, and she was incredible. She came from a linguistics background and had never done AI research or coding before. With this opportunity, she learned how to conduct user studies and even used LLMs to help write code for her research. Her first paper was accepted on the first try at a top-tier HCI conference… I was so proud of her.”
This experience reflects a broader theme of the project: the value of interdisciplinary collaboration. By bringing together perspectives from linguistics, computer science, and social science, Dr. Sap’s work not only advances research on AI systems but also creates meaningful opportunities for students to engage with the societal impacts of emerging technologies.