Bi-Weekly AI Research Roundup

Latest research summaries in ML/DL, Robotics, CV, NLP and GenAI

Contents

1. CodeV: Empowering LLMs for Verilog Generation through Multi-Level Summarization

2. A call for embodied AI

3. IDs for AI Systems

4. SegPoint: Segment Any Point Cloud via Large Language Model

5. EnergyDiff: Universal Time-Series Energy Data Generation using Diffusion Models

6. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

7. Scaling Granite Code Models to 128K Context

8. Open-Source Conversational AI with SpeechBrain 1.0

9. KNOWNET: Guided Health Information Seeking from LLMs via Knowledge Graph Integration

10. Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation

11. RoboGolf: Mastering Real-World Minigolf with a Reflective Multi-Modality Vision-Language Model

12. Hyp2Nav: Hyperbolic Planning and Curiosity for Crowd Navigation

13. ExploreGen: Large Language Models for Envisioning the Uses and Risks of AI Technologies

14. StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions

CodeV: Empowering LLMs for Verilog Generation through Multi-Level Summarization

Authors: Yang Zhao, Di Huang, Chongxiao Li, Pengwei Jin, Ziyuan Nan, Tianyun Ma, Lei Qi, Yansong Pan, Zhenxing Zhang, Rui Zhang, Xishan Zhang, Zidong Du, Qi Guo, Xing Hu, Yunji Chen

Source and references: https://arxiv.org/abs/2407.10424v3

---

Introduction

This paper introduces CodeV, a series of open-source instruction-tuned large language models (LLMs) for Verilog generation. The increasing complexity and high costs associated with modern processor design have led to a surge in demand for processor design automation, and LLMs have shown promise in automatically generating code for general-purpose programming languages. However, existing methods fail to perform well on hardware description languages (HDLs) like Verilog due to the scarcity of high-quality instruction tuning data.

Key Points

• The authors observe that Verilog code collected from the real world has higher quality than those generated by LLMs, and that LLMs excel in summarizing Verilog code rather than generating it.

• Based on these observations, the paper introduces CodeV, which employs a reverse instruction generation approach to construct high-quality instruction-tuning datasets from real-world Verilog code.

• The CodeV framework uses multi-level summarization to bridge the semantic gap between high-level requirement descriptions and low-level Verilog code, and fine-tunes base LLMs to create SOTA Verilog generation models.

• The authors plan to open-source the CodeV models and the instruction tuning dataset containing 165K high-quality description-code pairs.

Methodology

The CodeV framework first collects and filters high-quality Verilog modules from open-source codebases. It then uses GPT-3.5 to generate multi-level summaries of the Verilog code, pairing high-level descriptions with corresponding modules to create a high-quality instruction-tuning dataset. Finally, the authors fine-tune base LLMs, including CodeLlama, DeepSeekCoder, and CodeQwen, on this dataset to create the CodeV series of models.

Results and Findings

Experimental results show that the CodeV series achieve state-of-the-art (SOTA) results on the VerilogEval and RTLLM benchmarks. CodeV-CodeQwen achieves 77.6% pass@1 on the VerilogEval-machine benchmark and 53.2% on the VerilogEval-human benchmark, outperforming GPT-4 and the previous SOTA model BetterV. The CodeV models also achieve a 93.1% syntax pass rate and 55.2% function pass rate on the RTLLM benchmark, outperforming the previous SOTA model RTLCoder.

Implications and Conclusions

The CodeV framework and models represent a significant advancement in the field of Verilog generation, demonstrating the effectiveness of reverse instruction generation and multi-level summarization approaches. The open-sourcing of the CodeV models and the instruction tuning dataset is expected to support the advancement and collaboration within the LLM, electronic design automation (EDA), and programming language communities.

---

A call for embodied AI

Authors: Giuseppe Paolo, Jonas Gonzalez-Billandon, Balázs Kégl

Source and references: https://arxiv.org/abs/2402.03824v3

---

Introduction

This paper proposes Embodied AI (E-AI) as the next fundamental step in the pursuit of Artificial General Intelligence (AGI), in contrast to the current advancements in Large Language Models (LLMs).