Multiverse Turns LLM Pruning Into Ising Optimization, Keeps Llama 3.3 Near 77 MMLU at Half Depth
Multiverse Computing maps LLM block pruning to an Ising problem, keeping Llama-3.3-70B near 77 MMLU after removing 50% of its depth without retraining.
Summary
On September 21, 2026, Multiverse Computing researchers Antonio Tiene, Ali Hashemi, David Jansen and Roman Rausch presented “LLM Compression by Block Removal with Constrained Binary Optimization.” CBO maps each transformer block to 0, keep, or 1, remove, then uses a second-order loss expansion to build a Hessian containing individual importance and pairwise couplings. It minimizes xᵀH⁰x while deleting exactly M of N blocks, equivalent to an all-to-all Ising glass with fixed magnetization. The Hessian needs one set of forward and backward passes on a small calibration dataset; candidates then require only cheap energy calculations, and one Hessian serves multiple compression targets.
A single GPU can enumerate millions of configurations in seconds, but testing roughly 29 billion ways to remove 8 of Llama-3.3-70B-Instruct’s 80 blocks took about two days. Larger searches convert to QUBO for classical, quantum or quantum-inspired methods including quantum annealing, QAOA, branch-and-bound and tabu search; an open-source tabu solver found verified lowest-energy states within seconds. Because energy imperfectly predicts quality, CBO returns several low-energy states. With 16 of 32 Llama-3.1-8B-Instruct blocks removed, the 17th excited state cut an early block and beat the ground state across several benchmarks after light retraining.
Without retraining, Llama-3.3-70B-Instruct scored 82.2 MMLU originally, 76.6 versus block influence’s 59.3 after removing 32 of 80 blocks, and 76.9 versus 54.0 at 40 of 80, an almost 23-point advantage and wins across every tested benchmark. Qwen3-14B led block influence by about 10 MMLU points at 12 of 40 blocks removed. On hybrid NVIDIA-Nemotron-3-Nano-30B-A3B-FP8, CBO removed 2 to 3 MoE layers or 2 attention layers and beat block influence on AIME25 and GPQA without retraining. The open-source code supports compression pipelines combining block removal with quantization, low-rank or SVD compression, width pruning and knowledge-distillation-based healing.
Positives
- 76.9 MMLU at 40 of 80 blocks removed kept Llama-3.3-70B-Instruct near its 82.2 original score without retraining.
- Almost 23 MMLU points separated CBO from block influence at 50% depth, with CBO winning every tested benchmark.
- About 10 MMLU points favored CBO on Qwen3-14B after removing 12 of 40 blocks.
- Seconds were enough for an open-source tabu solver to reach verified lowest-energy states on the hardest checkable searches.
- The 17th excited state beat the ground state across several benchmarks after light retraining of Llama-3.1-8B-Instruct.
- AIME25 and GPQA improved over block influence when CBO pruned NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 without retraining.
Risks & concerns
- Roughly 29 billion configurations required about two days on one GPU when removing 8 of Llama-3.3-70B-Instruct’s 80 blocks.
- Exponential growth makes exact enumeration impractical for deeper pruning targets and larger models.
- The lowest Ising energy does not always identify the highest-quality pruned model, requiring evaluation of multiple low-energy states.
- Removing the wrong interacting blocks can collapse model quality, making independent block rankings unreliable.
- At lighter compression, CBO was only comparable with existing block-influence methods because pairwise couplings mattered less.