OpenAI Jalapeño Chip Beats Nvidia Blackwell in First Inference Benchmarks
OpenAI’s Jalapeño beats Nvidia Blackwell on inference speed and power efficiency, but tiny 2026 volumes push broader deployment into 2027 for customers.
Summary
OpenAI unveiled Jalapeño’s first benchmark results at Hot Chips on Tuesday, August 25, 2026. On SemiAnalysis’ InferenceX benchmark, the inference system delivered more tokens per user and greater throughput per kilowatt than currently available state of the art processors, including an Nvidia Blackwell system. Hardware chief Richard Ho said Jalapeño can return responses with low latency while serving more customers per unit of power.
First announced in October 2025, Jalapeño was developed with Broadcom, with OpenAI models assisting its creation. OpenAI plans a multigenerational platform coordinating AI products, models, chips and memory. Its full stack architecture targets prefill and communication bottlenecks by reducing data movement, keeping model state and the response generating KV cache local, and activating the appropriate compute, memory and networking for each inference phase. Deployment is expected in very small volumes at the end of 2026, followed by more significant deployment in 2027, when competing processors may have advanced.
Positives
- SemiAnalysis’ InferenceX benchmark placed Jalapeño ahead of currently available state of the art inference processors in tokens per user and throughput per kilowatt.
- Richard Ho said Jalapeño combines low latency with the power efficiency needed to serve many customers.
- OpenAI and Broadcom jointly developed Jalapeño, with OpenAI’s models assisting the design process.
- Jalapeño keeps model state and the KV cache local to reduce data movement and communication delays.
- OpenAI plans to coordinate products, models, chips and memory across multiple Jalapeño generations.
Risks & concerns
- Very small deployment volumes at the end of 2026 will limit Jalapeño’s near term impact.
- Significant deployment is not expected until 2027, giving competing inference processors time to advance.
- Jalapeño’s benchmark lead is measured against currently available Nvidia Blackwell hardware, which may not represent the frontier by broad deployment.
- The disclosed InferenceX comparison provides no numerical benchmark scores for independently assessing the size of Jalapeño’s lead.