State of AI

Week 2, August 2023

Greetings,

Welcome to the 18th edition of the State of AI. This issue takes you on an enlightening journey through the ever-evolving world of AI. We commence with a fascinating exploration into the self-repair capabilities in language models, aptly named "The Hydra Effect". Transitioning into the cooperative realm, our focus then shifts to the innovations brought by MetaGPT, emphasizing the synergy of meta programming in multi-agent collaborative systems.

As we venture further, we uncover the prowess of ToolLLM, which has remarkably equipped large language models to proficiently navigate more than 16,000 real-world APIs. Next, we spotlight the strides in training scalability with Deepspeed-chat, a testament to the seamless melding of efficiency and scale in training chat models. Concluding our journey, we embrace the notion of modeling our diverse world through the lens of language, underpinning the limitless potentialities of AI.

Every article in this edition is a testament to the boundless horizons AI continues to shape, inviting readers to a deeply insightful expedition. Dive in and relish the marvels!

Best regards,

State of AI

---

Contents

1. The Hydra Effect: Emergent Self-repair in Language Model Computations

2. MetaGPT: Meta Programming for Multi-Agent Collaborative Framework

3. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

4. Deepspeed-chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales

5. Learning to Model the World with Language

---

The Hydra Effect: Emergent Self-repair in Language Model Computations

Authors: Thomas McGrath, Matthew Rahtz, János Kramár, Vladimir Mikulik, Shane Legg

Source & References: https://arxiv.org/abs/2307.15771v1

---

Introduction

A recent research paper from Google DeepMind takes an in-depth look at large language models (LLMs) such as GPT-3 and unveils a phenomenon they call the Hydra Effect. The Hydra Effect refers to the ability of LLMs to not only demonstrate redundancy but also adapt and self-repair their computations when certain layers are ablated or "removed." This intriguing behavior raises new questions about how neural networks encode and process information and opens new avenues for interpretability.

Motivation & Background

Understanding the internal structure and computation of large language models has always been a challenging task. Researchers often rely on ablation studies where parts of a trained neural network are removed or altered to investigate their impact on the model's performance. However, such modifications do not always lead to the expected degradation in performance. The Hydra Effect reveals the complex nature of neural network computations and the fact that removing certain components can trigger compensatory mechanisms in the model, making them more resilient to network disruptions.

Model Architecture

The researchers focus on an autoregressive Transformer-based language model similar to GPT-3, with 7 billion parameters. Autoregressive language models use the past sequence of input tokens to predict the probability distribution of the next token. Transformer architectures consist of multiple layers and have two main components: attention layers (Attn) and multi-layer perceptrons (MLP). Attention layers help encode dependencies between input tokens, while the MLP layers refine the representation by adding non-linear transformations.

Dataset & Methods

To analyze the model's factual recall, the Counterfact dataset is used. The dataset contains a series of factual statements about subjects and their relations to various objects. Using this dataset allows the researchers to measure the importance of each layer by calculating its impact on the model's logits (pre-softmax values).

Two primary methods are employed to determine a layer's importance:

1. Unembedding: This method maps layer outputs to the logits using the model's logit lens, a function that calculates the output logits by applying the layer normalization (LayerNorm or RMSNorm) followed by the unembedding matrix.

2. Ablation: This method is more invasive, as it requires replacing a layer's output with another value (e.g., a vector of zeros) and observing the effect on the model's performance.

Results & the Hydra Effect

The researchers find that ablation-based and unembedding-based importance measures disagree in most layers. While the former method generally results in a more significant impact on model performance, the latter method reveals that the removed components have less importance. This discrepancy is attributed to the Hydra Effect, where removing a layer prompts other downstream layers to compensate for the lost functionality.

This effect is not observed in all layers but seems to be localized to specific attention and MLP layers. Additionally, some downstream MLP layers seem to perform an erasure or memory-management function, attenuating or accentuating their impacts depending on the importance of the upstream attention layers.

Neural Networks as Causal Models

One way to understand the complex relationships within neural networks is by viewing them as structural causal models. These models describe how variables influence each other and can help determine the underlying casual mechanisms. In the context of large language models, the compute graph – the architecture that outlines the flow of information through the model – can be treated as a causal graph.

Using causal models, the research highlights the difference between direct and total effects, linking unembedding-based impact measures to direct effects while ablation-based measures correspond to total effects. This approach allows for more granular insights into the behavior of LLMs and their self-repair abilities.

Implications & Future Work

The discovery of the Hydra Effect has several implications for interpreting and analyzing neural network computations. First, it challenges the assumption that removing important components should degrade model performance and may cause cascading failures. Second, the effect complicates our understanding of component importance as different methods can yield different results. Finally, the phenomenon offers clues into how language models naturally develop redundancy and adaptation capabilities, which could be expanded upon in future work.

In conclusion, the Hydra Effect identified by Google DeepMind researchers in large language models like GPT-3 opens up new possibilities for understanding neural networks, as they adapt and self-heal despite disruptions to their internal structure. This line of research has the potential to further our knowledge in interpreting and analyzing the complex world of neural network computations and offers a valuable tool for future interpretability efforts.

---

MetaGPT: Meta Programming for Multi-Agent Collaborative Framework

Authors: Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan LIN, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu

Source & References: https://arxiv.org/abs/2308.00352

---

Introduction

Many recent breakthroughs in machine learning have leveraged large language models (LLMs) to solve problems more efficiently than ever before. However, with increased complexity in real-world applications, the need to create more efficient programming systems that can understand and solve complicated tasks has become more critical. MetaGPT is a framework designed to improve LLM-driven multi-agent collaboration. By incorporating human-like workflows and standardized operating procedures, this innovative approach aims to provide more organized, coherent, and effective solutions for complex problem-solving.