Inference Chips
Ongoing coverage on Inference Chips.
Latest coverage
Jalapeño Benchmarks: OpenAI’s Inference Chip Outruns Blackwell on Paper, Ships at Scale in 2027
OpenAI's new inference chip, codenamed Jalapeño, shows promising performance on paper, outperforming Nvidia's Blackwell in benchmarks for tokens per user and throughput per kilowatt. However, the chip is not expected to ship in significant quantities until 2027, raising questions about its competitiveness against future hardware. OpenAI's head of hardware, Richard Ho, highlighted the chip's efficiency and low latency, attributing its design to minimizing data movement and local KV cache retention.
AI Yi-Yi!
OpenAI and Broadcom have partnered to develop inference chips optimized for large language models. Separately, NVIDIA and Amazon Web Services are collaborating to scale AI deployment for production environments. The article also touches on the increasing use of AI-generated influencers by brands for promotional purposes.