Would You like a feature Interview?
All Interviews are 100% FREE of Charge
As generative AI permeates more and more industries, the companies that make the chips that run the models stand to gain big. Estimation Cloud providers, which account for 70% to 95% of the AI chip market, wield great influence. Meta To Microsoft Wary of falling behind in generative AI, it’s spending billions on NVIDIA GPUs.
There’s understandable reason why generative AI vendors aren’t satisfied with the status quo: Their success depends in large part on the whims of the major chipmakers, so they, along with opportunistic venture capitalists, are on the hunt for promising startups to challenge the incumbents of AI chips.
etching is one of many alternative chip companies vying for position in the space, but it’s also one of the most interesting. Just two years old, Etched was founded by Harvard dropouts Gavin Uberti (ex-OctoML and ex-Xnor.ai) and Chris Zhu, who, along with Robert Wachen and former Cypress Semiconductor CTO Mark Ross, set out to build a chip that could do one thing and one thing only: run AI models.
This is not unusual. Many startups and large tech companies are developing chips specifically for AI models, also known as inference chips. Meta has MTIA, Amazon has Graviton and Inferentia, etc. But Etched’s chips are teeth It is unique in that it only runs one type of model: Transformers.
Proposed by a team of Google researchers in 2017, Transformers are by far the dominant generative AI model architecture to date.
Transformers are the basis of OpenAI’s video generation model Sora, they are at the core of text generation models such as Anthropic’s Claude and Google’s Gemini, and they are also used in generative art tools such as the latest version of Stable Diffusion.
“In 2022, we bet that Transformers will rule the world,” Etched CEO Uberti told TechCrunch in an interview. “We’ve reached a point in the evolution of AI where specialized chips that perform better than general-purpose GPUs are essential, and tech decision makers around the world know this.”
Etched’s chip, “Sohu,” is an ASIC (application-specific integrated circuit), a chip customized for a specific use, in this case to operate a transformer. TSMC’s 4nm processUberti claims that Sohu can deliver significantly better inference performance than GPUs and other general-purpose AI chips while consuming less power.
“Sohu is orders of magnitude faster and cheaper than Nvidia’s next-generation Blackwell GB200 GPUs when running text, image and video transformers,” Uberti said. “One Sohu server is the equivalent of 160 H100 GPUs. … Sohu will be a more affordable, efficient and environmentally friendly option for business leaders who need specialized chips.”
How does Sohu achieve all this? There are a few ways, but the most obvious and intuitive is a streamlined inference hardware and software pipeline. Because Sohu does not run non-Transformer models, the Etched team was able to eliminate hardware components that are not related to Transformers, while simultaneously reducing the software overhead traditionally used to deploy and run non-Transformers.
Etched arrives at a tipping point in the race for generative AI infrastructure: Beyond cost concerns, the GPUs and other hardware components currently required to run large models are dangerously power-hungry.
Goldman Sachs Predict AI is predicted to increase data center electricity demand by 160% by 2030, leading to a significant increase in greenhouse gas emissions. Meanwhile, researchers at the University of California, Riverside say: estimate Global AI use could lead to data centers consuming 1.1 trillion to 1.7 trillion gallons of freshwater by 2027. Impact on local resources(Many data centers use water to cool their servers.)
Uberti optimistically (or hyperbolically, depending on how you interpret it) pitches Sohu as a solution to the industry’s consumption problems.
“The bottom line is that our future customers will have no choice but to switch to Sohu,” Uberti said. “Companies are betting on Etched because speed and cost are essential for the AI products they’re building.”
But assuming Etched achieves its goal of bringing Sohu to the mass market within the next few months, can it succeed with so many other companies following close behind?
Etched doesn’t have any direct competitors at the moment, but AI chip startup Perceive recently Previewed the processor It has hardware acceleration for the Transformer, and Groq has also invested heavily in Transformer-specific optimizations for the ASIC.
Competition aside, what if one day Transformers fell out of favor? Uberti says that in that case, Etched would naturally design a new chip. That’s reasonable, but it would be a pretty drastic step back given how long it took to perfect Sohu.
None of these concerns have deterred investors from pumping huge amounts of money into Etched.
Today, Etched announced it has closed a $120 million Series A funding round co-led by Primary Venture Partners and Positive Sum Ventures. This brings Etched’s total funding to $125.36 million and included participation from big-name angel investors such as Peter Thiel (Uberti, Zhu, and Wachen are Thiel Fellowship alumni), GitHub CEO Thomas Dohmke, Cruise (and Bot Company) co-founder Kyle Vogt, and Quora co-founder Charlie Cheever.
These investors likely believe there’s ample opportunity for Etched to expand its server sales business. And it probably will: Uberti claims that unnamed customers have reserved “tens of millions of dollars” worth of hardware so far. Uberti suggested that the upcoming release of Sohu Developer Cloud, which lets customers preview Sohu through an online interactive playground, should drive further sales.
But it remains to be seen whether this is enough to propel Etched and its 35-person team into the future its co-founders envision. The AI chip field can be unforgiving even in good times. Graphcoreand relatedly, Funding for AI chip ventures to plummet in 2023.
But Uberti has a strong pitch: “Video generation, audio-to-audio conversion, robotics, and other future AI use cases wouldn’t be possible without faster chips like Sohu. The entire future of AI technology will be shaped by whether our infrastructure can scale.”