Dharma-AI GPU Allocator Boosts Utilization 33 Points Over FIFO
Dharma-AI's GPU allocator lifted utilization by up to 33 points and priority-weighted output 105.1% versus FIFO, with decisions in 1 to 15 ms per request.
Summary
On August 17, 2026, Dharma-AI reported that its constraint-aware allocator beat FIFO across seven scenarios on identical hardware and workloads. Utilization gained up to 33 percentage points. Priority-weighted output improved in all seven, peaking at 105.1% and averaging 52% across five contended tests. It schedules contiguous, nonpreemptible training, batch inference and quantization blocks alongside elastic real-time demand, with capped timestep swaps and one job per GPU.
Results, listing FIFO to allocator utilization, weighted output, gain and latency, were: mixed control, 8 GPUs and 10 jobs, 51.6% to 72.4%, 7,093 to 10,980, 54.8%, 1 ms; real-time contention, 8 and 8, 75.0% to 80.2%, 3,233 to 4,029, 24.6%, 1 ms; training-heavy, 8 and 16, 53.6% to 87.0%, 8,553 to 17,545, 105.1%, 2 ms; large mixed, 14 and 16, 76.8% to 82.7%, 13,977 to 20,101, 43.8%, 2 ms; oversubscribed, 8 and 9, 85.4% to 87.5%, 4,311 to 5,760, 33.6%, 1 ms; scale, 64 and 30, 44.9% tied, 44,233 to 51,248, 15.9%, 15 ms, with both completing 27 of 30 jobs; uniform priority, 14 and 16, 76.8% to 87.5%, 25,219 to 31,052, 23.1%, 2 ms.
The NP-hard model prices unmet real-time demand at 5 to 10 times equal-priority batch allocation. A legal-by-design heuristic powers fast mode, while full mode refines its grid. It optimizes 24 hours, commits one timestep, reruns every 30 to 60 minutes and pins active jobs. Training forecasts use 22 features and 10 variants covering full tuning, LoRA, SFT, DPO, RLHF, RLVR and CPT. LoRA cuts trainable parameters up to 10,000 times and memory roughly 3 times; DPO removes RLHF’s reward model and sampling loop. Quantization models bitsandbytes, AWQ and GPTQ, while real-time forecasting rebuilds weekly profiles from hourly traffic. Forecast errors remain the central risk.
Positives
- Training-heavy utilization rose from 53.6% to 87.0%, while priority-weighted output increased 105.1% with a 2 ms scheduling latency.
- All seven scenarios improved priority-weighted output, and six increased utilization while the 64-GPU scale test tied at 44.9%.
- The 64-GPU, 30-job scale test delivered 15.9% more weighted value in 15 ms while both schedulers completed 27 jobs.
- Uniform-priority scheduling raised utilization from 76.8% to 87.5% and value 23.1%, showing horizon planning adds gains without priority differences.
- Fast-mode decisions took 1 to 2 ms in contended tests, supporting reallocation on every incoming request.
Risks & concerns
- Incorrect duration and traffic forecasts can undermine allocation because required GPU-hours and real-time demand are predictions rather than fixed inputs.
- The training-heavy headline result reflects a single FIFO baseline ordering, limiting what the 33-point comparison alone establishes.
- The 64-GPU scale test left utilization unchanged at 44.9%, despite producing 15.9% more priority-weighted value.
- FIFO peak reservations can strand four GPUs all day when an application needs six at midday but only two at 4am.
- NP-hard combinatorial allocation cannot meet per-request latency directly, requiring a heuristic on the hot path instead of continuous formal optimization.
