CommunityAI: Supporting Community-Centered Evaluation of Social Service AI via the AI Risk Reports
Block by Block: Research at Work is a research spotlight series that highlights the innovative work being done by CMU researchers through the Block Center, showcasing how their projects are driving impactful solutions at the intersection of technology and society.
During a recent conversation with the Block Center for Technology and Society, Dr. Hong Shen discussed how her research on social service AI has evolved since we last spoke.
CommunityAI: Supporting Community-Centered Evaluation of Social Service AI via the AI Risk Reports
Artificial intelligence is rapidly expanding, particularly through tools designed for use in social services. However, during a previous conversation with the Block Center, Dr. Hong Shen highlighted a persistent disconnect between AI developers and the communities these systems are meant to serve. Researchers building AI tools are often not members of the communities affected by them, and community stakeholders frequently have little input during design and deployment. This gap has led to systems that, at best, fail to address real needs and, at worst, harm vulnerable populations.
In response, Dr. Shen and her collaborators launched the CommunityAI project in 2023, supported by Block Center Seed Funding. The project aims to create participatory methods that allow communities to meaningfully evaluate and shape AI systems used in social services.
Working alongside the University of Pittsburgh’s School of Social Work and local social service organizations, Dr. Shen’s team developed AI Failure Cards, a hands-on tool designed to build AI literacy while centering community expertise. By gathering direct input from frontline workers and social service providers, researchers can better understand how AI systems function in practice and refine decision-making tools to align with real community needs.
AI Failure Cards are a card-based toolkit that helps community members identify, document, and reflect on potential risks and failures in AI systems. Rather than requiring technical expertise, the cards present realistic scenarios—such as biased recommendations, incorrect eligibility assessments, or a lack of transparency—and prompt participants to discuss how these failures might appear in their daily work.
The cards transform abstract technical risks into accessible, experience-based discussions, allowing social workers and impacted communities to contribute practical knowledge that developers might otherwise overlook and propose bottom-up strategies to mitigate potential harms.
The CommunityAI project has evolved into a set of generalizable, open-access methods that organizations can adapt to their own local contexts. Dr. Shen regularly delivers student lectures and workshops demonstrating how the toolkit works in practice. For example, students at the University of Pittsburgh School of Social Work are learning how to engage directly with the communities they serve and integrate stakeholder feedback into AI development, creating an ongoing feedback loop between users and designers.
After initial local pilots, the project expanded nationally through additional funding from the National Science Foundation’s Civic Innovation Challenge, enabling new partnerships with community organizations across the country.
One such partner is Collaborative Support Programs of New Jersey (CSPNJ), a statewide peer-led nonprofit focused on mental health services. Building on CommunityAI’s participatory design framework, Dr. Shen’s team collaborated with CSPNJ to better understand the needs of peer support workers and the individuals they serve. This collaboration resulted in PeerCoPilot, a chatbot designed to help mental health peer providers support clients’ wellness goals, develop action plans, and check benefits eligibility.
As Dr. Shen explained, “We are taking a participatory, community-centered AI design method and working closely with peer-run organizations to build this ‘peer copilot’ to support frontline providers serving people receiving public social services.”
The work continues to expand nationwide, with similar collaborations starting to emerge in New York and Utah.
During our conversation, Dr. Shen emphasized that this research would not be possible without the contributions of students and interdisciplinary collaborators, including faculty members Hoda Heidari, Ken Holstein, Fei Fang, Haiyi Zhu from Carnegie Mellon University, Nev Jones from the University of Pittsburgh, and a group of dedicated students. . The project brings together expertise from computer science, social work, public policy, and community organizations—all united by a shared commitment to ensuring technology serves real human needs.
“We care deeply about the real-world impact of the technologies being deployed,” she noted. “Our goal is to empower the people most affected by these systems to participate directly in their design and deployment.”
Dr. Shen encourages students interested in this area to explore multiple dimensions of AI research—from policy and governance to machine learning and community engagement. Because technological innovation often moves faster than social institutions can adapt, interdisciplinary collaboration is essential. Community trust, she emphasized, remains central: when researchers are perceived as outsiders, meaningful collaboration becomes difficult, and the goals of CommunityAI cannot be achieved.
Growing interest in community-centered AI development continues to shape the future of Dr. Shen’s work, demonstrating how participatory approaches can help ensure AI systems are accountable, equitable, and grounded in lived experience.