Greetings,
Welcome to the 14th edition of the State of AI. This issue transports us into a fascinating exploration of AI and robotics, where we examine how large language models are lending their planning capabilities to robots and prompting them to ask for help. We then delve into the intriguing domain of AI psychology, exploring how personality traits are manifesting in large language models.
Continuing our journey, we look at scalability and marvel at how transformers are being scaled to handle an unprecedented 1,000,000,000 tokens. The safety and reliability of large language models come under our microscope next, as we study why and how LLM safety training sometimes fails. Finally, we turn our attention towards the intriguing interplay between AI and mental health, as we consider how models are being trained to generate, recognize, and reframe unhelpful thoughts.
Each of these topics illustrates the breadth and depth of AI's potential and provides a fascinating insight into the state of the field. We invite you to join us on this journey of discovery and hope you enjoy this issue.
Best regards,
Contents
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
Personality Traits in Large Language Models
LongNet: Scaling Transformers to 1,000,000,000 Tokens
Jailbroken: How Does LLM Safety Training Fail?
Training Models to Generate, Recognize, and Reframe Unhelpful Thoughts
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
Authors: Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu, Dorsa Sadigh, Andy Zeng, Anirudha Majumdar
Source & References: https://arxiv.org/abs/2307.01928
Introduction
Large language models (LLMs) have made tremendous advancements in recent years and continue to demonstrate remarkable capabilities, such as step-by-step planning and commonsense reasoning. These traits make LLMs an attractive solution for enabling robots to perform complex tasks. However, despite their potential, LLMs still suffer from a tendency to generate confidently wrong outputs, or "hallucinations." In this research paper, the authors present KNOW NO, a framework designed to align the uncertainty of LLM-based planners and help robots recognize when they don't know the solution, prompting them to ask for assistance when necessary.
Aligning Uncertainty with Conformal Prediction
To tackle the problem of hallucinations in LLMs, the authors of the paper utilize Conformal Prediction (CP), a method for aligning the uncertainty of LLM-based planners. CP provides statistical guarantees on task completion while minimizing the need for human intervention. This approach is built on the observation that language-instructed robot planning can be treated as a multiple-choice Q&A (MCQA) problem. By transforming the planning problem into MCQA and applying CP, the framework is able to provide calibrated confidence and minimize help from humans.
With CP, the robot generates a set of plausible next steps and their corresponding confidence scores. If there is more than one option in the set generated by CP, the robot asks for help, improving efficiency and autonomy compared to modern baselines that require extensive prompt tuning.
Theoretical Guarantees and Minimal Help
The authors prove theoretical guarantees on calibrated confidence for both single-step and multi-step planning problems. These guarantees ensure that, with a user-specified probability level (1-ϵ), the robot successfully performs tasks in which it seeks help when necessary. Additionally, CP minimizes the average size of prediction sets, helping to achieve the goal of minimal help.
Evaluating KNOW NO in Simulated and Real-World Environments
The researchers evaluated the KNOW NO framework across a variety of simulated and real-world robot setups, involving tasks containing different types of ambiguity, such as spatial or numeric uncertainties and human preferences. Their experiments demonstrated that KNOW NO significantly improves efficiency and autonomy compared to traditional baselines.
In these settings, the robot first generates multiple candidate plans for the next step and ranks them based on confidence scores. If there is more than one option and the robot is uncertain, it seeks help from a human, who can choose one of the options provided or halt the operation if necessary. This setup ensures that the robot's task completion rate aligns with the probability level specified by the user.
Addressing Ambiguous Instructions
In real-world scenarios, natural language instructions often contain inherent or unintentional ambiguity. Instead of following an incorrectly constructed plan that could lead to undesirable or unsafe actions, a robot using KNOW NO would ask for help or clarification when necessary. This ability to identify when it doesn't know the best course of action allows the robot to operate more safely and effectively.
Enabling Efficient and Autonomous Robots
Overall, KNOW NO represents a significant step toward enabling robots to make intelligent decisions when faced with ambiguous or uncertain situations. By aligning the uncertainty of LLM-based planners and providing calibrated confidence and minimal help, the framework allows robots to improve their efficiency and autonomy. With further development and integration, this approach has the potential to significantly advance the field of robotics and create safer, more reliable, and more capable machines.
LONGNET: Scaling Transformers to 1,000,000,000 Tokens
Authors: Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Furu Wei
Source & References: https://arxiv.org/abs/2307.02486
The Pursuit of Scaling Sequence Length
A recent development in the world of machine learning and natural language processing comes in the form of LONGNET, a Transformer variant capable of scaling up to a staggering one billion tokens without sacrificing its performance on shorter sequences. Conceived by researchers at Microsoft Research, this accomplishment addresses a persistent challenge in the development of neural networks—balancing computational complexity and model expressivity, particularly when it comes to increasing sequence lengths. Scaling up sequence length introduces a host of benefits, such as enabling interaction with humans and exploring the limits of in-context learning.
Keep reading with a 7-day free trial
Subscribe to State of AI to keep reading this post and get 7 days of free access to the full post archives.