A new study published in Nature found that large language models can forecast outcomes of social science experiments with accuracy rivaling pooled human experts. The research used an archive of 70 preregistered, nationally representative survey experiments encompassing 469 distinct treatment effects and responses from nearly 120,000 American participants. By prompting GPT-4 to simulate how representative samples would react to experimental stimuli, the team inferred treatment effects strongly correlated with real-world results, even for experiments not published before the model’s training data cutoff. The model was not fine-tuned on outcomes; it relied solely on parametric knowledge from pretraining. Predictions from GPT-4 achieved a correlation with actual treatment effects nearly indistinguishable from pooled human forecasts. However, the analysis revealed a systematic tendency for the LLM to overestimate effect sizes, meaning simulated participants often showed larger differences between conditions than real humans did. To further stress-test the approach, the team compiled a secondary archive of 15 megastudies containing 606 treatment effects.