FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (AI summary)

Key Points

1. The emergence of generative pre-trained models, such as GPT-4, has led to the synthesis of high-quality text, but has also posed challenges in identifying factual errors in the generated text.

2. Challenges in factuality detection include a wider range of tasks facing an increasing risk of self-check errors when handled by generative models, the length and lack of clearly defined granularity for individual facts in generated texts, and a scarcity of explicit evidence available during the fact-checking process.

3. The paper proposes FAC T OOL, a task and domain-agnostic framework for detecting factual errors in texts generated by large language models, and experiments on four different tasks show the efficacy of the proposed method.

4. The research emphasizes the need for a more comprehensive factuality detection and verification framework that is versatile given the remarkable versatility of tasks and domains handled by LLMs.

5. The paper introduces a new, more challenging yet practical task setting for factuality detection without explicit claims or evidence, and proposes a framework capable of addressing this challenge across a variety of scenarios.

6. The framework leverages various tools, including Google Search, Google Scholar, code interpreters, Python, or even LLMs themselves, to gather evidence about the factuality of the generated content.

7. The paper evaluates the factuality of modern chatbots, including GPT-4, ChatGPT, Claude-v1, Bard, and Vicuna-13B, using FAC T OOL powered by GPT-4, and demonstrates that GPT-4 has the best factuality across almost all scenarios.

8. Experimentation is conducted across diverse tasks such as knowledge-based QA, code generation, math problem solving, and scientific literature review writing to demonstrate the potential of incorporating tools like Google Search, Google Scholar, code interpreters, Python, and LLMs in factual error detection.

9. Overall, the research aims to address the challenges of factuality detection in generative AI and proposes a comprehensive and adaptable framework that can be extended to more scenarios.

Summary

The paper proposes a task and domain-agnostic framework, FACTOOL, for detecting factual errors in texts generated by large language models (LLMs) such as ChatGPT. It identifies the challenges introduced by generative AI technology, such as inaccuracies and lack of verifiable evidence, and aims to address the limitations of LLMs. The framework leverages various tools, including Google Search, Google Scholar, and LLMs themselves, to gather evidence about the factuality of the generated content and employs the reasoning abilities of LLMs to assess the factuality of the content. Experimental results demonstrate the efficacy of FACTOOL across knowledge-based QA, code generation, math problem solving, and scientific literature review writing.

The paper also reports a comparison of the proposed framework with self-check baselines, highlighting the superior performance of FACTOOL powered by GPT-4. Furthermore, the paper evaluates the factuality of modern chatbots using FACTOOL and demonstrates the effectiveness of GPT-4 in achieving the highest weighted claim-level factual accuracy and response-level accuracy compared to other chatbots such as ChatGPT, Bard, Claude-v1, and Vicuna-13B. The study offers insights into the potential of incorporating a comprehensive factuality detection and verification framework to improve the reliability of generated content by LLMs.

Reference: https://arxiv.org/abs/2307.13528v2