What Can You Do With 400,000 SLACers?

Cerbras WSE Likes Big Core Counts

There is something strange going on at Hot Chips this year, brought to us by Andrew Feldman, the man that created and later sold Seamicro to AMD; the largest chip on the planet which goes by the name Cerbras.  The chip is an impressive 46,225 square millimeters and contains 400,000 Sparse Linear Algebra Cores, specifically designed for neural network computation.   How they managed to create a a single chip interconnected on a single wafer without impurities ruining at least some of those cores is a question we would like to see answered beyond the current reply of redundancy; hopefully we will be filled in at Hot Chips.

As for the numbers, you are looking at a total of 1.2 trillion transistors on the wafer/chip with 18GB of SRAM memory offering 9PB/s of bandwidth with communication handled by Swarm fabric capable of 100 petabits per second; not a familiar phrase even for HPC machines. The SLAC architecture they use is focused on AI and DEEP Learning, so measuring floating point operations is a little misleading, for instance Cerbras WSE doesn’t believe in processing zeros.

Learn more via the links at Slashdot, as well as from the erudite comments below.