diff --git a/content/assets/selfplay.png b/content/assets/selfplay.png new file mode 100644 index 000000000..c4586e739 Binary files /dev/null and b/content/assets/selfplay.png differ diff --git a/content/research/SPIRAL - Letting LLMs Teach Themselves Through Self-Play.md b/content/research/SPIRAL - Letting LLMs Teach Themselves Through Self-Play.md index b26438fec..69da8901e 100644 --- a/content/research/SPIRAL - Letting LLMs Teach Themselves Through Self-Play.md +++ b/content/research/SPIRAL - Letting LLMs Teach Themselves Through Self-Play.md @@ -8,8 +8,14 @@ tags: - reinforcement - learning --- + +![[selfplay.png]] +*Source: [Liu, Guertler et al., 2025](https://arxiv.org/abs/2506.24119)*. + ## TL;DR -*We collaborated with the TextArena team to develop SPIRAL, a novel RL framework that allows LLMs to develop complex reasoning capabilities by playing text-based games against themselves. Using SPIRAL on a simplified variant of poker with no mathematical content, a 4B-parameter Qwen model improved its performance on math and reasoning benchmarks by 8.6% and 8.4% respectively. It does this by learning specific strategies, such as case-by-case analysis and expected value calculation, that generalize beyond poker better than simple game heuristics. We're excited to explore whether self-play on social deduction games like Mafia can lead to general improvements in LLMs' social cognition.* +_We collaborated with the TextArena team to develop SPIRAL, a novel RL framework that allows LLMs to develop complex reasoning capabilities by playing text-based games against themselves. Using SPIRAL on a simplified variant of poker with no mathematical content, a 4B-parameter Qwen model improved its performance on math and reasoning benchmarks by 8.6% and 8.4% respectively. It does this by learning specific strategies, such as case-by-case analysis and expected value calculation, that generalize beyond poker better than simple game heuristics. We're excited to explore whether self-play on social deduction games like Mafia can lead to general improvements in LLMs' social cognition._ + +--- ## Teaching Social Cognition Through Games At Plastic Labs, one of our key research interests is improving language models' social cognition: their ability to represent people's mental states, predict users' behaviors, and navigate complex social dynamics. This capability is essential for creating AI systems that can genuinely understand and adapt to individual users, yet it remains underdeveloped compared to technical abilities and so-called "hard skills" like reasoning and coding.