Tech | Visa | Scholarship/School | Info Place

Meta launches latest customized artificial intelligence chip to catch up

Meta is spending billions of dollars on its own AI research as it strives to catch up with its competitors in the field of generative artificial intelligence. Some of those billions will be used to recruit artificial intelligence researchers. But a larger portion is devoted to developing hardware, specifically the chips used to run and train Meta’s artificial intelligence models.

Today, a day after Intel announced its latest artificial intelligence accelerator hardware, Meta unveiled the latest results of its chip development efforts. Dubbed the “next generation” Meta Training and Inference Accelerator (MTIA), the chip is the successor to last year’s MTIA v1, and it runs models that include ranking and recommending display ads on Meta’s properties, such as Facebook.

Compared to MTIA v1, which was built on a 7nm process, the next generation MTIA uses a 5nm process. (In chip manufacturing, “process” refers to the size of the smallest component that can be built on a chip.) The next generation of MTIA is a physically larger design, equipped with more processing cores than its predecessor. While it consumes more power (90W vs. 25W), it also has more internal memory (128MB vs. 64MB) and runs at a higher average clock speed (up from 800MHz to 1.35GHz).

Meta said that the next generation of MTIA is currently in use in 16 of its data center regions and has improved overall performance by 3 times compared to MTIA v1. If the “3x” statement sounds a little vague, you’re not wrong – we think so, too. But Meta only volunteered that the number came from testing the performance of “four key models” of both chips.

“Because we control the entire stack, we can achieve higher efficiencies compared to commodity GPUs,” Meta wrote in a blog post shared with TechCrunch.

Meta’s hardware reveal — which comes just 24 hours after a press conference on the company’s various ongoing generative AI initiatives — was unusual for a few reasons.

First, Meta revealed in a blog post that it is not currently using next-generation MTIA to handle generative AI training workloads, although the company claims to have “multiple projects underway” to explore this. Second, Meta acknowledges that the next generation of MTIA will not replace the GPU used to run or train models, but will complement it.

Reading between the lines, Meta is moving slowly—perhaps slower than it wants to.

Meta’s AI team will almost certainly be under pressure to cut costs. The company expects to spend $18 billion on GPUs to train and run generative AI models by the end of 2024, and — with cutting-edge generative models costing tens of millions of dollars to train — the in-house hardware offers attractive alternatives.

I suspect that while Meta’s hardware was underperforming, its competitors were ahead, which shocked Meta’s leadership.

Google this week launched the TPU v5p, its fifth-generation custom chip for training artificial intelligence models, to Google Cloud customers, as well as its first dedicated chip for running models, Axion. Amazon has multiple lines of custom AI chips. Microsoft also joined the race last year with the Azure Maia AI accelerator and Azure Cobalt 100 CPU.

Meta said in a blog post that the next generation of MTIA took less than nine months “from first chip to production model,” which to be fair is shorter than the typical window between Google TPUs. But Meta still has a lot of work to do if it hopes to achieve a level of independence from third-party GPUs and match its fierce competition.

#Meta #launches #latest #customized #artificial #intelligence #chip #catch

Leave a Reply

Your email address will not be published. Required fields are marked *