Chameleon, WavCraft, ALPINE, RLHF Workflow & more on LoRA's

Week 3, May 2024

Greetings,

Welcome to the latest edition of the State of AI. In this issue, we explore the cutting-edge developments in multimodal AI models with Chameleon, delving into its early-fusion capabilities that blend image and text generation seamlessly. Next, we navigate through the innovative WavCraft, an advancement in audio editing and generation powered by large language models.

We also uncover the planning capabilities within autoregressive learning models with the intriguing ALPINE, and delve into the comprehensive workflow of RLHF, from reward modeling to online reinforcement learning. Finally, we examine the intriguing findings of LoRA, revealing how it learns and forgets less in the intricate world of language models.

Each of these topics showcases the remarkable strides being made in the AI realm, promising a rich and engaging read. Sit back, relax, and indulge in the fascinating world of AI advancements with us.

Best regards,

State of AI

---

Contents

1. Chameleon: Mixed-Modal Early-Fusion Foundation Models

2. WavCraft: Audio Editing and Generation with Large Language Models

3. ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models

4. RLHF Workflow: From Reward Modeling to Online RLHF

5. LoRA Learns Less and Forgets Less

---

Chameleon: Mixed-Modal Early-Fusion Foundation Models

Authors: Chameleon Team, FAIR at Meta

Source and references: https://arxiv.org/abs/2405.09818v1

---

Introduction

The intersection of text and image processing in machine learning has gained considerable traction, but a common challenge persists: how to seamlessly integrate and generate mixed-modal content. Enter "Chameleon," a ground-breaking family of early-fusion token-based mixed-modal models designed to understand and generate images and text in any arbitrary sequence. Developed by the Chameleon Team at FAIR, Meta, these models are debuting with remarkable capabilities in visual question answering, image captioning, text generation, and more. Let's dive deeper to understand what makes Chameleon a significant leap forward in this space.

A Unified Approach to Multimodal Understanding

Traditional multimodal models often separate text and image processing, leading to suboptimal integration and generation. Chameleon flips this notion by employing a unified architecture that treats images and text as discrete tokens. Imagine images encoded as tokens analogous to words in textual data. This is made possible by quantizing images, enabling the application of a single transformer architecture to sequences of both image and text tokens. Such an early-fusion approach allows Chameleon to smoothly reason and generate mixed-modal documents, a feat that has proved challenging until now.

The Secret Sauce: Stability and Training Innovations

One key challenge in developing such a model is maintaining stability during training, especially when dealing with large datasets and diverse modalities. Chameleon's development team tackled this by introducing architectural innovations like query-key normalization (QK-Norm) and careful placement of layer norms. These adjustments, among others, ensure stable training even as the model scales, addressing the complex divergences encountered in multimodal settings.

Part of the stability recipe also includes sophisticated training techniques. By adapting supervised fine-tuning approaches from text-only language models to the mixed-modal setting, Chameleon achieves strong alignment and performance. This robust approach allows the model to be trained on an enormous dataset of around 10 trillion tokens, including texts, images, and code, making it one of the most comprehensively trained models to date.

Performance Across Multiple Benchmarks

Chameleon's capabilities shine when evaluated across a variety of tasks. On visual question answering and image captioning benchmarks, it outperforms state-of-the-art models like Flamingo, IDEFICS, and Llava-1.5. Not just confined to mixed-modal tasks, Chameleon also holds its ground in text-only benchmarks, competing effectively with models like Mixtral 8x7B and Gemini-Pro in commonsense reasoning and reading comprehension tasks.

The model's prowess is not limited to benchmarks. It also sets new standards in generating high-quality images and text, the latter being a domain traditionally dominated by unimodal language models. Chameleon’s integrated approach offers holistic performance across a blend of vision and text, unlike any other single-modal or multimodal model to date.

Unprecedented Human Judgments in Mixed-Modal Generation

Static benchmarks, while useful, often fail to capture the nuanced capabilities of advanced models in real-world scenarios. To tackle this, the Chameleon team conducted an extensive human evaluation to measure the quality of mixed-modal long-form responses to open-ended prompts. Impressively, Chameleon-34B outperformed strong baselines like Gemini-Pro and GPT-4V, achieving a 60.4% preference rate against Gemini-Pro and 51.6% against GPT-4V in pairwise comparisons.

Architectural Novelty

Underlying Chameleon's success are several architectural innovations. The use of query-key normalization ensures controlled growth of norms, addressing the instability often encountered in mixed-modal training. Furthermore, Chameleon employs RMSNorm for normalization, the SwiGLU activation function, and rotary positional embeddings, refining the transformer architecture for optimal performance.

An additional trick up Chameleon’s sleeve is dropout adjustment—implemented after attention and feed-forward layers to ensure norm growth remains stable. For the larger model, Chameleon-34B, an innovative reordering of normalization techniques within the transformer block is used, which combined with previous adjustments, ensures stable and effective training.

A Glimpse at the Future of Multimodal Models

So, what does Chameleon's advent mean for the future of multimodal machine learning? It's a step towards truly unified foundation models capable of seamlessly reasoning and generating mixed-modal content. The implications are vast, from more intuitive AI systems that can understand and generate complex documents blending text and imagery, to advanced applications in creative industries where mixed-modal content is king.

Through its unified approach and advanced architecture, Chameleon promises a future where AI models are not hindered by the artificial separation of different types of data. Instead, they can fluidly handle a mixture of text, images, and beyond, paving the way for the next generation of intelligent systems.

Conclusion: New Horizons in AI Research

"Chameleon" represents a significant stride in the journey towards truly integrated multimodal AI models. Its early-fusion, token-based approach, combined with innovative training and stability techniques, marks a new era in AI research. Excelling across various benchmarks and proving its mettle in human evaluations, Chameleon showcases what's possible when models are designed to understand and generate across diverse data types from the ground up.

As the tech landscape continues to evolve, models like Chameleon will play a crucial role in shaping the future, enabling seamless integration and generation of multimodal content. For those of us keenly watching the AI space, Chameleon is a glimpse into the remarkable possibilities that lie ahead.

So next time you marvel at an AI's ability to generate a detailed image from a text prompt or produce a coherent, context-rich story interwoven with relevant visuals, remember—it's innovations like Chameleon's early-fusion model making it all possible. Keep an eye on Chameleon; this is just the beginning.

---

WAVCRAFT: Audio Editing and Generation with Large Language Models

Authors: Jinhua Liang, Huan Zhang, Haohe Liu, Yin Cao, Qiuqiang Kong, Xubo Liu, Wenwu Wang, Mark D. Plumbley, Huy Phan, Emmanouil Benetos

Source and references: https://arxiv.org/abs/2403.09527

---

Unleashing the Power of LLMs in Audio Editing

Welcome to the world of WAVCRAFT, a groundbreaking system for audio editing and generation brought to you by the collective efforts of researchers from prestigious institutions around the globe. WAVCRAFT doesn't just aim to transform audio content creation; it leverages the power of large language models (LLMs) to seamlessly connect task-specific modules for an unprecedented level of user control and creativity. Today, we’re diving into what makes WAVCRAFT a game-changer in the audio domain.