OpenAI Jalapeño: the custom chip beating Nvidia on inference

OpenAI no longer just writes models — it just shipped its first custom inference chip, and it beats Nvidia at Nvidia's own game: performance per watt. Jalapeño, built with Broadcom, produces more tokens per unit of power than any GPU system SemiAnalysis has measured.

For years, OpenAI was Nvidia's biggest customer. Now it makes its own hardware. That is a strategic shift, not a single hardware product.

What Jalapeño is

OpenAI calls Jalapeño an "Intelligence Processor" — a chip designed from scratch for inference, the job of serving LLMs. It is not a general-purpose GPU with tweaks. OpenAI designed the architecture, Broadcom handled the silicon and ASIC, and Celestica handled manufacturing.

The chip program was first announced in June 2026. Benchmark results landed at Hot Chips in August. For a first-generation chip, the timeline is aggressive: OpenAI says nine months from design to production, while SemiAnalysis estimates roughly 16 months from forming the hardware team to tape-out. Either way, that is unusually fast for the chip industry.

On the hardware side, Jalapeño uses HBM4, rated at 700 watts but with sustained power measured at or below 550 watts. That detail matters: first-generation chips usually trade efficiency for performance, but Jalapeño measured under its rating.

The benchmark numbers

OpenAI measured Jalapeño on InferenceX, SemiAnalysis's public benchmark, across three models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Against leading commercial systems, the results:

• Performance per watt up to 1.5–1.9x higher at peak throughput.
• End-to-end latency 1.7–3.6x lower.
• Highly interactive workloads (agents, chat) up to 2.1–4.1x higher performance.
• On Kimi K2.5 1T — the largest public model tested — roughly 1.5x higher performance per watt and 3.4x lower latency.

SemiAnalysis, the independent owner of the benchmark, goes further: by its account, Jalapeño beats every Nvidia, AMD, and Google chip it has tested. That is a third-party claim, not OpenAI's own, but it carries weight because SemiAnalysis is known to be hard on AI hardware.

Why it wins: full-stack co-design

The core of it is designing software and hardware together. LLM inference has two distinct bottlenecks: prefill is compute-bound, while decode is bound by memory bandwidth. A system that excels at one phase can lose its edge while waiting for data to move between cores and chips.

Jalapeño attacks this by minimizing data movement: model state, including the KV cache used during generation, is placed and kept local, activating the right mix of compute, memory, and networking per phase. The network is part of the architecture, not bolted on after.

One point is easy to miss: Jalapeño is a general chip, not one tuned only for OpenAI's models. It runs GPT-OSS, DeepSeek, and Kimi well — three models from three different sources. That kills the rumor that OpenAI only optimizes for its own models.

What it means for developers

This is the part that matters. When OpenAI builds its own inference chip, the cost of serving each request drops, and latency follows. The direct consequences for developers:

• Cheaper, faster APIs. OpenAI states its goal plainly: make AI more affordable and more accessible. For agentic apps — which run many sequential steps and are sensitive to compounding latency — this is good news.
• The chip race is no longer Nvidia's alone. When your biggest customer becomes its own supplier, competitive pressure lands on the whole GPU supply chain. Developers gain options, even if indirectly.
• Full-stack is the new moat. OpenAI now owns the model, product, serving software, chip, memory, networking, and system. Rivals operating a single layer will struggle to match this continuous optimization loop.

On deployment, Broadcom's Hock Tan confirmed the platform will scale to gigawatt-scale data centers with Microsoft and other partners starting in 2026. This is not a lab demo — engineering samples are already running GPT-5.3-Codex-Spark at production target frequency and power.

The takeaways

• Jalapeño is OpenAI's first inference chip, built with Broadcom and Celestica, using HBM4.
• On the InferenceX benchmark, it delivers 1.5–1.9x higher performance per watt and up to 3.6x lower latency.
• SemiAnalysis independently reports it beats every Nvidia, AMD, and Google chip they tested.
• A full-stack strategy lets OpenAI optimize software and hardware together, cutting data movement.
• This is a multi-generation platform scaling to gigawatts from 2026 — not a one-off product.

The open question is not whether Jalapeño is good. It is. The question is: when the largest model provider builds its own hardware, how long before the rest of the industry has to follow?

---

Content assisted by AI (Amy 🌸). Reviewed by the author.