Welcome to this week’s edition of State of AI 👋
This week’s crop splits along a few lines. Efficiency work is everywhere: an architecture that gives prefill and decode their own compute budgets, a memory scheme that unlocks capacity as sequences grow, and diffusion accelerations worth up to 7× with no retraining. Robots get both sides of the story, with a hierarchical foundation model that spends extra compute only when it’s unsure, and two security papers showing how easily the state those planners trust can be poisoned. And a healthy batch of negative results made the cut too: guidance tricks that fail to beat plain CFG, interpretability tools that add nothing to behavior prediction, and LLMs that waste 2–6× the compute of a grid search when tuning physics simulations.
Here’s what caught our attention:
τ₀-VLA: A Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation: separating high-level subtask planning from low-level motor control, with compute allocated only at low-confidence decisions, gets robots through complex household manipulation across multiple embodiments at 45–90% success rates.
TDD-Agent: Test-Driven Reasoning for Code Generation: writing executable tests before implementation, then letting tests and code evolve together, reaches 78–90% pass rates on repository-level tasks.
Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation: an architecture that separates prompt processing from token generation, so the two inference phases can be optimized independently, with consistent quality gains across dense and sparse scaling.
Proteus: Incremental Memory Activation for Long-Context Sequence Modeling: progressively expanding memory capacity as sequences grow beats static allocation on needle-in-haystack retrieval and long-context understanding, with zero added parameters.
When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents: compromising the environmental state a planner reads can redirect robot behavior toward adversarial goals at 99% planning-level attack success, with user instructions and planner logic untouched.
Spectral Progressive Diffusion for Efficient Image and Video Generation: up to 7× speedup on images and 2.5× on video by riding the frequency spectrum diffusion models already generate implicitly, with zero architectural modifications.
SMA: Auditing Membership Leakage in Retrieval-Augmented Generation Systems: source-aware membership inference that tells you whether leaked content came from pretraining data, external retrieval, or user input, working in semi-black-box settings with no gradient access.
Le Critique: Privileged Value Functions for LLM Reinforcement Learning: feeding value functions task information the policy never sees (reference solutions, other trajectories) cuts variance in LLM RL while keeping policy gradients unbiased.
Let’s get into it 👇
Contents
Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation
SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controlling
Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models
Spectral Progressive Diffusion for Efficient Image and Video Generation
An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models
SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMs
Le Critique: Privileged Value Functions for LLM Reinforcement Learning
Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
Proteus: Incremental Memory Activation for Long-Context Sequence Modeling
Memory Efficient Audio Synthesis with Decoupled Temporal Depth Diffusion Transformers
τ0τ_0 τ0-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation
When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents
Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation
Authors: Liming Liu, Mingze Wang, Tuo Zhao
Source and references: https://arxiv.org/abs/2608.12385v2
Introduction
LLM serving has a built-in mismatch: prefill is compute-bound, decode is memory-bound, and conventional models allocate identical computation to both. The Decode-Branch Transformer splits them. A primary path alone processes prompts and owns the KV cache, while a lightweight decode branch adds continuation computation on top, so each phase can be provisioned for what it actually costs.
Key Points
Phase-decoupled architecture: the primary path determines the persistent KV cache; the decode branch adds computation during generation while writing no cache state and leaving the primary path untouched, which makes the two phases independently tunable.
Shared-weight efficiency: primary and decode paths share all major attention, MLP, and output matrices, with separate embeddings and learned coupling vectors, so extra decode computation avoids proportional growth in memory traffic during the memory-bound phase.
Consistent quality gains: across NanoGPT token scaling, dense LLaMA-style scaling, and sparse MoE configurations, Decode-Branch reaches lower validation loss than baseline Transformers under matched token budgets.
Router replay for MoE: the decode branch reuses the primary path’s selected expert set with independent mixture weights, keeping routing cost flat while allowing phase-specific expert allocation.
A three-way trade-off surface: primary and branch expert fan-outs become independently configurable knobs, exposing trade-offs among prefill cost, decode cost, and model quality that can be tuned per serving workload.
Methodology
The core mechanism is asymmetric shared-KV attention: the primary path performs standard causal self-attention, and the decode branch generates queries that attend to the primary path’s keys and values through learned coupling. Training combines predictions from both paths with weights set by the primary distribution’s confidence. Evaluation runs three axes: controlled data scaling with NanoGPT (five token budgets), dense LLaMA-style scaling from 0.125B to 1B parameters at 80× parameter count in tokens, and sparse MoE scaling from 0.25B to 1B with independent routing or router replay, all under matched token budgets.
Results and Findings
On NanoGPT, Decode-Branch beat the standard Transformer at all five token budgets, with fitted asymptotes of 2.9013 versus 2.9416. In an approximate compute-matched comparison, a D=10 Decode-Branch beat a D=20 Transformer on validation loss with equivalent backbone computation. Dense scaling showed persistent gains at every size from 0.125B to 1B. MoE runs improved consistently with router replay, and fully independent routing added only modest further gains. The allocation sweeps are the most useful result for practitioners: fixed-prefill sweeps show monotonic loss improvement as the branch expert budget grows, and fixed-decode sweeps peak near three-quarter prefill fractions, mapping the trade-off surface directly.
Implications and Conclusions
The idea lands exactly where serving economics hurt, since decode compute is what you pay for on every generated token while prefill amortizes over the prompt. The evidence so far is validation loss at 1B scale and below, though, and the paper stops short of the number that would seal the argument: measured end-to-end throughput on a production serving stack, where continuous batching and cache pressure decide what actually helps. Until someone publishes that, treat this as a well-mapped design space awaiting its deployment test.
TDD-Agent: Test-Driven Reasoning for Code Generation
Authors: Hongyue Yu, Kefan Li, Jiakun Li, Hongzheng Chai, Yuan Yuan, Rui He, Junyi Wei
Source and references: https://arxiv.org/abs/2608.16742v1
Introduction
TDD-Agent applies test-driven development to LLM code generation. The model writes executable unit tests before any implementation, then code and tests refine each other iteratively based on execution feedback. Prior pipelines treat generated tests as static validators applied after the fact; here test generation works as a reasoning step that forces the model to pin down expected behaviors, input constraints, and edge cases up front.
