Etched unveiled Sohu: the world's first transformer-specific AI chip

Ellie Ramirez-Camara -

As the demand for computing power rises, and NVIDIA's dominance increases, alternative AI chip companies have emerged and tech firms have turned to developing in-house inferencing chips specialized in running AI models. Etched, however, is unlike those other companies because it is developing algorithm-specific AI chips (ASICs) designed to run transformer models exclusively. This bold move represents a significant bet on the continued dominance of transformer architecture in AI, given that if transformer models were to be replaced as the dominant architecture by any other, the chips would become useless.

Sohu, Etched's first chip, boasts unprecedented performance for transformer models. Etched claims that Sohu is significantly faster and cheaper than its competitors, including NVIDIA's next-generation Blackwell B200 GPUs. This claim is backed by Sohu's 500,000 tokens per second throughput for Llama 70B, an unthinkable feat for any current or next-generation GPU, and one that enables Sohu to support otherwise impossible applications. Key features of Sohu include:

  1. Specialized architecture: Traditional GPUs devote most of their area to programmability to gain flexibility and support multiple architectures. Since Sohu is transformer-specific, it can dispense with this requirement and fit more math blocks, the ones yielding the computing power. With this approach, Sohu achieves over 90% FLOPS utilization, compared to ~30% on GPUs.
  2. Impressive scalability: Because of its unique architecture, an 8xSohu server reportedly replaces 160 H100 GPUs.
  3. Wide compatibility: Sohu supports all major transformer models from companies like Google, Meta, Microsoft, and OpenAI, and will handle enhancements and modifications to future transformer-based models.

Etched argues that the increasing scale and cost of AI model training and inference make specialized chips inevitable. They believe transformers will continue to dominate the AI landscape, justifying their algorithm-specific approach. Only time can tell whether Etched's bet will pay off.

In the meantime, the company has secured a partnership with TSMC on their 4nm process, talent hailing from major AI chip projects, and early customers reserving funds for Etched hardware. Moreover, the company opened a registration form to grant interested individuals and organizations access to the Sohu Developer Cloud and is looking to grow its team.