A research team led by Professor Li Ning, Flextronics Chair and Chair of the Department of Leadership and Organizational Management at Tsinghua University's School of Economics and Management, has achieved a major breakthrough in AI-assisted social science research. Their paper, "A Large-Scale Replication of Scenario-Based Experiments in Psychology and Management Using Large Language Models," was published in the August 2025 issue of Nature Computational Science (Vol. 5, No. 8), a sub-journal of Nature. The first author is PhD student Cui Ziyan, with Li Ning as corresponding author and postdoctoral fellow Zhou Huaikang as a key contributor.

Traditional psychology experiments require hundreds of human participants to make decisions, answer questions, or engage in scenario simulations in laboratories. The Tsinghua team instead created a "digital laboratory," presenting 156 classic experiments from 5 top psychology and management journals to three large language models—GPT-4, Claude 3.5 Sonnet, and DeepSeek V3—allowing them to simulate human participants. This method, called "silicon-based replication," enables countless "digital humans" to independently think and make decisions, replicating human experimental behavior.

The results are striking: replication rates reached 73%–81%. In other words, if a human experiment found that "rewards can increase work motivation," the AI models had over a 70% probability of reaching the same conclusion. This provides social science researchers with a "rapid laboratory" for quickly testing hypotheses, saving significant time and cost.

The study establishes a standardized workflow for using AI in psychology experiments and proposes quantitative metrics—such as replication success, directional consistency, and effect size comparison—to guide future research. It represents a leading international effort in applying large language models to social sciences and offers systematic empirical evidence for AI-assisted research.

This work also aligns with China's "AI+" Initiative, which encourages human-AI collaborative research in philosophy and social sciences. By innovating experimental methods and research organization, the study not only broadens the research perspective of social sciences but also provides an empirical path for understanding AI’s impact on cognition, judgment, and ethics.

Nature Computational Science, published by Nature Publishing Group, focuses on computational science and interdisciplinary research, with a 2024 impact factor of 18.3.

Editor: Ren Zhongxi