Etched Is Making the Biggest Bet in AI
Etched invests in AI with Sohu, a specialized chip for transformers, surpassing traditional models like DLRMs and CNNs. Sohu optimizes transformer models like ChatGPT, aiming to excel in AI superintelligence.
Read original articleEtched is making a significant investment in AI by developing Sohu, the world's first specialized chip for transformers. This chip is designed to outperform traditional AI models like DLRMs and CNNs, specifically excelling in running transformer models such as ChatGPT. The company believes that scaling is crucial for achieving superintelligence in AI models, with a focus on providing more compute power and better data. The shift towards specialized chips for transformers is seen as inevitable due to the increasing demand for transformer inference in various AI applications. Sohu's design allows for higher FLOPS utilization compared to GPUs, enabling it to handle transformer models more efficiently. The company's approach involves optimizing hardware and software specifically for transformers, aiming to lead the market in transformer-specific inference. Etched's bet on transformers is supported by the current dominance of transformer architectures in AI applications, making Sohu a key player in the evolving AI hardware landscape.
Related
Optimizing AI Inference at Character.ai
Character.AI optimizes AI inference for LLMs, handling 20,000+ queries/sec globally. Innovations like Multi-Query Attention and int8 quantization reduced serving costs by 33x since late 2022, aiming to enhance AI capabilities worldwide.
AI is exhausting the power grid
Tech firms, including Microsoft, face a power crisis due to AI's energy demands straining the grid and increasing emissions. Fusion power exploration aims to combat fossil fuel reliance, but current operations heavily impact the environment.
TSMC experimenting with rectangular wafers vs. round for more chips per wafer
TSMC is developing an advanced chip packaging method to address AI-driven demand for computing power. Intel and Samsung are also exploring similar approaches to boost semiconductor capabilities amid the AI boom.
Intel's Gaudi 3 will cost half the price of Nvidia's H100
Intel's Gaudi 3 AI processor is priced at $15,650, half of Nvidia's H100. Intel aims to compete in the AI market dominated by Nvidia, facing challenges from cloud providers' custom AI processors.
Finnish startup says it can speed up any CPU by 100x
A Finnish startup, Flow Computing, introduces the Parallel Processing Unit (PPU) chip promising 100x CPU performance boost for AI and autonomous vehicles. Despite skepticism, CEO Timo Valtonen is optimistic about partnerships and industry adoption.
This seems to be a novel definition of "smarter" - one could also argue that the printed answer key for a standardized test is smarter than most humans.
I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?
I mean, are they? Seems like the industry would prefer these things to become commodities, especially if it helps with portability and reproducibility.
Related
Optimizing AI Inference at Character.ai
Character.AI optimizes AI inference for LLMs, handling 20,000+ queries/sec globally. Innovations like Multi-Query Attention and int8 quantization reduced serving costs by 33x since late 2022, aiming to enhance AI capabilities worldwide.
AI is exhausting the power grid
Tech firms, including Microsoft, face a power crisis due to AI's energy demands straining the grid and increasing emissions. Fusion power exploration aims to combat fossil fuel reliance, but current operations heavily impact the environment.
TSMC experimenting with rectangular wafers vs. round for more chips per wafer
TSMC is developing an advanced chip packaging method to address AI-driven demand for computing power. Intel and Samsung are also exploring similar approaches to boost semiconductor capabilities amid the AI boom.
Intel's Gaudi 3 will cost half the price of Nvidia's H100
Intel's Gaudi 3 AI processor is priced at $15,650, half of Nvidia's H100. Intel aims to compete in the AI market dominated by Nvidia, facing challenges from cloud providers' custom AI processors.
Finnish startup says it can speed up any CPU by 100x
A Finnish startup, Flow Computing, introduces the Parallel Processing Unit (PPU) chip promising 100x CPU performance boost for AI and autonomous vehicles. Despite skepticism, CEO Timo Valtonen is optimistic about partnerships and industry adoption.