Abstract: 

Large language models (LLMs) have demonstrated remarkable capabilities in generating coherent text and completing various natural language tasks. Nevertheless, their ability to perform complex, general reasoning has remained limited. In this talk, I will describe OpenAI’s new o-series models, which are trained via reinforcement learning to generate a hidden chain of thought before its response. We have found that the performance of these models consistently improve with more reinforcement learning compute and with more inference compute. The latest model, o3, surpasses previous state-of-the-art models in a variety of benchmarks that require reasoning, including mathematics competitions, programming contests, and advanced science question sets. I will discuss the implications of scaling this paradigm even further.