Reflection-Bench: Evaluating Agency in Large Language Models

Lingyu Li 1,2,3, Yixu Wang 1, Haiquan Zhao 1, Shuqi Kong 1,2,3, Yan Teng 1, Chunbo Li 2,3, Yingchun Wang 1
1 Shanghai Artificial Intelligence Laboratory
2 Shanghai Jiao Tong University
3 Shanghai Mental Health Center
Paper Code arXiv

Reflection, a fundamental process underlying agent-environment interaction, integrates various cognitive components. To achieve desired outcomes, the agent needs to predict the future states and behavioral consequences based on prior beliefs. Post-action, discrepancies between prediction and observation are perceived, prompting an update of prior belief via recalling previous events and counterfactual thinking.

Abstract

The increasing deployment of large language models (LLMs) as autonomous agents necessitates rigorous evaluation of their fundamental agency capabilities in complex real-world environments. Drawing on cognitive psychology, we introduce Reflection-Bench, a novel benchmark that conceptualizes agent-environment interaction as an integrated reflection process encompassing seven interrelated cognitive dimensions: predictive processing, decision-making, perception, memory, counterfactual thinking, belief updating, and meta-reflection. Distinguished by its parameterized design and contamination-free nature, Reflection-Bench enables systematic assessment across varying difficulty levels while minimizing training data overlap. Our comprehensive evaluation of 15 current LLMs using three prompting strategies reveals a clear performance hierarchy and identifies critical limitations in current models. While leading LLMs demonstrate basic agency capabilities, they struggle with meta-reflection and cross-functional cognitive integration. These findings highlight the gap between current LLM capabilities and the requirements for reliable autonomous agents, suggesting directions for future development in strengthening core cognitive functions, improving cross-functional coordination, and enhancing adaptive processing mechanisms.

BibTeX

@misc{li2024reflectionbenchprobingaiintelligence,
        title={Reflection-Bench: probing AI intelligence with reflection}, 
        author={Lingyu Li and Yixu Wang and Haiquan Zhao and Shuqi Kong and Yan Teng and Chunbo Li and Yingchun Wang},
        year={2024},
        eprint={2410.16270},
        archivePrefix={arXiv},
        primaryClass={cs.AI},
        url={https://arxiv.org/abs/2410.16270}}