Etched’s sohu chip shakes up AI inference, threatening nvidia’s reign with 20x throughput leap

By mid-2026, Etched revolutionized transformer-specific AI hardware by pioneering a holistic inference solution that extends beyond chip design to encompass racks, power delivery, boards, and interconnects, enabling optimized transformer model inference at scale. Recognizing thermal constraints that cap FLOP utilization on GPUs, Etched innovated low-voltage operation inspired by industries like Bitcoin mining, demonstrating that cutting voltage significantly reduces power consumption and thermal throttling, thereby boosting throughput without the traditional performance penalties.

Etched’s Sohu chip marks a transformative leap in efficiency by hardcoding the transformer attention mechanism directly into silicon, achieving an unprecedented 90% compute utilization compared to the 30-40% typical of GPUs. This specialization enables a single eight-chip Sohu server to process Llama 70B at 500,000 tokens per second—vastly outperforming eight Nvidia H100s at 25,000 tokens per second and eight Blackwell B200s at 43,000 tokens per second—though this architectural rigidity limits compatibility with non-transformer or variant models like DeepSeek V4 and Qwen3-235B-A22B.

Etched’s innovation extends beyond silicon to cluster-scale memory interconnects and integrated inference ecosystems, exemplified by the Sohu chip’s 144 GB of HBM3E memory fabricated on TSMC’s advanced N4P four-nanometer process. This combination of cutting-edge memory bandwidth and low-latency design, paired with full rack-scale inference clusters under over $1 billion in contracts, underscores Etched’s strategic commitment to delivering scalable, infrastructure-level optimization tailored exclusively for transformer workloads.

By mid-2026, Etched had rapidly emerged as a formidable challenger to Nvidia’s AI inference hardware dominance, securing $800 million in cumulative funding that propelled its valuation from a modest $34 million in 2023 to an impressive $5 billion—a staggering 147-fold increase. This surge was underpinned by over $1 billion in signed customer contracts even before large-scale deliveries began, signaling robust market demand and confidence in Etched’s specialized hardware solutions. High-profile investors such as Peter Thiel, Jane Street, and Nobel laureate Geoffrey Hinton further underscore the industry’s belief in Etched’s disruptive potential.

Etched’s aggressive vertical integration strategy sets it apart in a market where competitors often focus solely on chip sales. By designing and manufacturing its Sohu ASIC chip—tailored exclusively for Transformer architectures—and delivering fully integrated rack-scale AI inference clusters complete with hardware, interconnection, and software, Etched simplifies deployment for enterprise customers. This end-to-end control, including production facilities in Taiwan and a 2MW data center in San Jose targeting gigawatt-scale deployment by 2027, enables faster product cycles and addresses the costly complexity of setting up AI infrastructure, directly challenging Nvidia’s established ecosystem.

Technologically, Etched’s Sohu chip exemplifies its competitive edge by embedding Transformer computational logic directly into hardware circuits, achieving approximately 20 times higher inference throughput compared to Nvidia GPUs. Their eight-chip server configuration claims parity with the performance of 160 Nvidia H100 GPUs, offering enterprises a compelling solution to slash infrastructure costs while boosting efficiency. This performance leap, combined with a 400+ person team drawn from Nvidia, Google TPU, TSMC, and other industry leaders, positions Etched not just as a hardware vendor but as a cohesive AI inference ecosystem innovator.

By mid-2026, Etched has decisively shifted from selling standalone AI chips to offering fully integrated rack-scale inference clusters, a move that redefines the commercial relationship with customers by packaging chips, custom racks, interconnection hardware, and software into turnkey AI inference ecosystems. This approach not only simplifies deployment—allowing customers to run large Transformer models immediately without complex configuration—but also locks in long-term infrastructure contracts that are far less flexible than traditional GPU swaps, signaling a strategic bet on ecosystem adoption and scale.

Etched’s Sohu chip exemplifies the broader industry trend away from general-purpose GPUs toward specialized silicon purpose-built for Transformer-based AI workloads, hardcoding the attention mechanism directly into hardware to achieve utilization rates around 90%, compared to 30-40% on Nvidia’s H100 GPUs. This specialization enables a 'different class of machine' optimized for inference throughput, latency, and power efficiency, underscoring the competitive advantage of tightly integrated hardware-software solutions in the emerging AI inference market.

Etched’s vertically integrated strategy—spanning chip design, production in Taiwan, and system integration in a 2MW San Jose data center—has been fueled by a rapid talent acquisition spree, including senior engineers from Nvidia, Google TPU, and Broadcom, growing to over 400 employees within three years. This full-stack control accelerates development cycles and supports the company’s ability to fulfill large-scale orders exceeding $6.8 billion, positioning Etched as a formidable rival to Nvidia by delivering optimized, scalable AI inference clusters that meet the demands of next-generation AI workloads.

By mid-2026, Etched’s vertically integrated approach—spanning chips, racks, software, manufacturing, and production—has dramatically lowered AI inference costs and accelerated time-to-market, enabling support for cutting-edge models like GPT-5.6. This end-to-end co-design strategy contrasts sharply with peers who take over seven years to bring products to market, positioning Etched as a pivotal player in the shift from training-focused AI development to inference-centric deployment, where latency, cost, and scalability define competitive advantage.

Etched’s breakthrough ASIC chip, Sohu, tailored exclusively for Transformer architectures, delivers roughly 20 times the throughput of general-purpose GPUs, enabling massive concurrency and scalability in inference workloads. With over $1 billion in backlog orders and production underway with TSMC’s 4nm process, Etched’s rapid milestones—including first-try chip success—underline its capacity to meet the surging market demand for efficient, high-volume AI inference.

The economics of AI are rapidly evolving as inference now consumes 80% to 90% of a model’s total lifecycle cost and has overtaken training in data center revenue contribution. Etched’s cost reductions are thus not merely incremental but transformative, unlocking new AI applications such as agent-based tasks that require 5 to 30 times more tokens than typical interactions, fundamentally reshaping the scalability and profitability landscape for AI labs and enterprises alike.