Key Points

1. Large Language Models (LLMs) are trained on a combination of natural language and formal language (code), enhancing their programming and reasoning capabilities.

2. Code-empowered LLMs have demonstrated improved performance as decision-making hubs for intelligent agents, enabling them to handle a wider range of complex tasks in multi-agent environment simulation and AI for science.

3. The article presents an overview of the benefits of integrating code into LLMs' training data, highlighting how code training enhances LLMs' reasoning abilities and task performance.

4. LLMs trained with expressions formulated within a defined set of symbols and rules exhibit advantages similar to those trained with programming languages, expanding the definition of code to incorporate homogeneous training corpora.

5. Code training broadens the scope of LLMs' tasks beyond natural language, enabling diverse applications including code generation for mathematical theory, general programming tasks, and data retrieval.

6. Code-LLMs demonstrate improved performance on structured reasoning tasks and complex reasoning capabilities related to markup, HTML, and chart understanding.

7. LLMs trained with code exhibit strong code generation abilities, benefiting applications such as database administration, game design, and website generation.

8. LLMs themselves can serve as state-of-the-art code evaluators, analyzing and scoring human or machine-generated codes for various tasks, including code fault localization, code evaluation, and automatic bug reproduction.

9. Code-empowered LLMs have led to the emergence of intelligent agents (IAs) in complex real-world tasks, enhancing IAs' decision-making, planning and execution skills, and facilitating self-correction and self-improvement.

Summary

The research paper explores the integration of code in large language models (LLMs) and its impact on enhancing the models to act as intelligent agents (IAs). It presents an overview of the benefits of integrating code into LLMs' training data, such as unlocking reasoning ability, producing structured intermediate steps, and taking advantage of code compilation and execution environment. The paper also discusses how these advantages facilitate the functioning of LLMs as IAs, especially in complex real-world tasks, and presents several key challenges and future directions in this field.

The paper focuses on various aspects of LLM training with code, such as defining code, methods for LLM code training, and advantages of integrating code into LLM training. It discusses how LLMs benefit from code training in terms of programming proficiency, complex reasoning capabilities, and structured knowledge comprehension. Additionally, it highlights how code-LLMs are employed in real-world scenarios, such as robotics, autonomous driving, and complex reasoning tasks.

Addressing Challenges and Future Directions
The study also addresses the challenges and future directions in this domain, including the need to understand the precise influence of specific code properties on LLMs' reasoning abilities, exploring alternative data modalities and training objectives to further enhance the reasoning capabilities of LLMs, and the potential of reinforcement learning as a more effective approach for utilizing feedback and improving LLMs.

Comprehensive Understanding
The paper provides a comprehensive understanding of how code enhances LLMs as intelligent agents and offers insights into the future directions and challenges in this rapidly evolving field of research.

Reference: https://arxiv.org/abs/2401.00812