News Meta is reportedly testing its first RISC-V based AI chip for AI training

[Admin]

Meta is evaluating its AI training accelerator, which could reduce its reliance on Nvidia's AI GPUs for training.

Meta is reportedly testing its first RISC-V based AI chip for AI training : Read more

 

[Jame5]

RSIC-V? Is that an offshoot of RISC-V?

 

[usertests]

RSIC-V? Is that an offshoot of RISC-V?

A very sickly offshoot.

 

[COLGeek]

Fixed in the forums. Hopefully the author will see and adjust on the article side (title).

 

[ezst036]

Gotta fix the messed up URL also.

tomshardware.com/tech-industry/artificial-intelligence/meta-is-reportedly-testing-its-first-rsic-v-based-ai-chip-for-ai-training

 

[COLGeek]

Gotta fix the messed up URL also.

tomshardware.com/tech-industry/artificial-intelligence/meta-is-reportedly-testing-its-first-rsic-v-based-ai-chip-for-ai-training

I saw that, but that would break the link from the forum side.

 

[bit_user]

One interesting thing to note is that MTIA's accelerators for inference use open-source RISC-V cores. This enables Meta to customize instruction set architecture as it wishes to meet its requirements at its cadence, but on the other hand, it does not need to pay royalties to any third party. It is unclear whether MTIA's training accelerator is also based on the RISC-V ISA, but this is possible.

The authors on this site continue to struggle with language around the open standard that is RISC-V.

Not knowing whether it's even using RISC-V, the author then walks onto a speculative branch of assuming the cores were designed in-house, which is pretty much the only way they could've completely avoided paying for them. It seems most likely they were designed by Broadcom or maybe SiFive, in either case meaning that Meta would've paid something for them (whether it was a one-time up-front charge or an ongoing royalty is more of a footnote).

 

[bit_user]

Edit: I think it's a different chip, but they discussed their next-gen inference accelerator at Hot Chips 2024, where they went into a little detail about its use of RISC-V (but sadly not the cores' microarchitecture or saying whether they were internal or licensed):

Meta AI Acceleration in the Next-Gen Meta MTIA for Recommendation Inference

The next-gen Meta MTIA is a custom RISC-V accelerator for the company's recommendation model AI inference workloads deployed this year

www.servethehome.com

In terms of memory, that presentations claims 256 MiB of on-die SRAM and 16-channel LPDDR5, providing 128 GiB at ~205 GB/s. So, they must mean 16-bit channels, because that'd be a 256-bit datapath which aligns with the capacity and bandwidth numbers. I've got to say that's rather underwhelming for an ASIC made on TSMC N5 that's 421 mm^2.

I think the key thing is that they're relying on batching (via the on-die SRAM) and model parallelism to minimize the chip's external DRAM bandwidth requirements. They show 2 processors + 256 GB LPDDR5 per PCIe card, with a single server hosting 12 cards via two PCIe switches.

More detail, although beware that this page covers a bunch of other presentations, as well. There's a table of contents, at the top. The Meta chip is 2nd.

https://irrationalanalysis.substack.com/p/hot-chips-2024-irrational-recap
​

 

[Mr Majestyk]

Every company that can is looking for ways to wed themselves off Leatherman's proprietary iron grip. As much as I despise Meta, good to see them make this move.