Humans are increasingly delegating agency to machines such as language models in social contexts, giving rise to social agents. I lead research advancing the study of social agents through three main directions:
Deploying social agents at scale for useful social simulations that inform real-world practice.
arXiv'25a
What Makes LLM Agent Simulations Useful for Policy? Insights From an Iterative Design Engagement in Emergency Preparedness
Yuxuan Li, Sauvik Das, Hirokazu Shirado
Under review at CHI 2026 paper / cite
LLM agent simulations can be genuinely useful for policy. We work closely with policymakers over 16 months to design and build a 13,000-agent simulation system that directly informed and improved policy at CMU.
Citation
@article{li2025makes,
title={What Makes LLM Agent Simulations Useful for Policy? Insights From an Iterative Design Engagement in Emergency Preparedness},
author={Li, Yuxuan and Das, Sauvik and Shirado, Hirokazu},
journal={arXiv preprint arXiv:2509.21868},
year={2025}
}
Spontaneous Giving and Calculated Greed in Language Models
Yuxuan Li, Hirokazu Shirado
EMNLP 2025 Main | The 2025 Conference on Empirical Methods in Natural Language Processing Oral Presentation | Extensive Media Coverage paper / video / selected media coverage / cite
Reasoning models are greedier. We find that reasoning models consistently exhibit lower cooperation and reduced norm-enforced punishment, mirroring human tendencies of "spontaneous giving and calculated greed".
@inproceedings{li-shirado-2025-spontaneous,
title = "Spontaneous Giving and Calculated Greed in Language Models",
author = "Li, Yuxuan and
Shirado, Hirokazu",
editor = "Christodoulopoulos, Christos and
Chakraborty, Tanmoy and
Rose, Carolyn and
Peng, Violet",
booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.emnlp-main.267/",
doi = "10.18653/v1/2025.emnlp-main.267",
pages = "5271--5286",
ISBN = "979-8-89176-332-6",
abstract = "Large language models demonstrate strong problem-solving abilities through reasoning techniques such as chain-of-thought prompting and reflection. However, it remains unclear whether these reasoning capabilities extend to a form of social intelligence: making effective decisions in cooperative contexts. We examine this question using economic games that simulate social dilemmas. First, we apply chain-of-thought and reflection prompting to GPT-4o in a Public Goods Game. We then evaluate multiple off-the-shelf models across six cooperation and punishment games, comparing those with and without explicit reasoning mechanisms. We find that reasoning models consistently reduce cooperation and norm enforcement, favoring individual rationality. In repeated interactions, groups with more reasoning agents exhibit lower collective gains. These behaviors mirror human patterns of ``spontaneous giving and calculated greed.'' Our findings underscore the need for LLM architectures that incorporate social intelligence alongside reasoning, to help address{---}rather than reinforce{---}the challenges of collective action."
}
Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models
Yuxuan Li, Hirokazu Shirado, Sauvik Das
FAccT 2025 | ACM Conference on Fairness, Accountability, and Transparency paper / video / cite
LLMs are increasingly implicitly biased. We find that when simulating human behavior (actions), more advanced models exhibit stronger sociodemographic biases, even though these biases appear reduced when measured explicitly through Q&A (words).
Citation
@inproceedings{li2025actions,
title={Actions speak louder than words: Agent decisions reveal implicit biases in language models},
author={Li, Yuxuan and Shirado, Hirokazu and Das, Sauvik},
booktitle={Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency},
pages={3303--3325},
year={2025}
}
HiddenBench: Assessing Collective Reasoning in Multi-Agent LLMs via Hidden Profile Tasks
Yuxuan Li, Aoi Naito, Hirokazu Shirado
Under review at ICLR 2026 paper / cite
Multi-agent LLM systems fail at collective reasoning, much like human groups. We demonstrate this by formalizing the Hidden Profile paradigm from social psychology and constructing a scalable 65-task benchmark based on this formalization.
Citation
@article{li2025assessing,
title={Assessing Collective Reasoning in Multi-Agent LLMs via Hidden Profile Tasks},
author={Li, Yuxuan and Naito, Aoi and Shirado, Hirokazu},
journal={arXiv preprint arXiv:2505.11556},
year={2025}
}
A Human-Computer Collaborative Tool for Training a Single Large Language Model Agent into a Network through Few Examples
Lihang Pan*, Yuxuan Li*, Chun Yu, Yuanchun Shi (*denotes equal contribution)
arXiv 2024 paper / cite
We make it easy for novices to build powerful multi-agent LLM systems (MAS). We introduce EasyLAN, a human–AI collaborative system that helps users construct MAS using only a few examples.
Citation
@article{pan2024human,
title={A human-computer collaborative tool for training a single large language model agent into a network through few examples},
author={Pan, Lihang and Li, Yuxuan and Yu, Chun and Shi, Yuanchun},
journal={arXiv preprint arXiv:2404.15974},
year={2024}
}
Say Your Reason: Extract Contextual Rules In Situ for Context-aware Service Recommendation
Yuxuan Li, Jiahui Li, Lihang Pan, Chun Yu, Yuanchun Shi
arXiv 2024 paper / cite
We make it easy to control service recommendations on users’ mobile phones in situ. We introduce SayRea, an interactive system that helps extract contextual rules for personalized, context-aware service recommendations in mobile scenarios using LLMs.
Citation
@article{li2024say,
title={Say Your Reason: Extract Contextual Rules In Situ for Context-aware Service Recommendation},
author={Li, Yuxuan and Li, Jiahui and Pan, Lihang and Yu, Chun and Shi, Yuanchun},
journal={arXiv preprint arXiv:2408.13977},
year={2024}
}