TurboQuant May Be An Enthusiast’s New Best Friend

As A Bonus, It’s Existence Is Causing Memory Manufacturers Stocks To Plummet

Wouldn’t it be nice to see memory prices fall, supply to once again exist and also see memory manufacturers hoisted by their own petard because of their greed?  That is assuming you didn’t use the money you were going to spend on RAM and SSDs to invest in Micron, Western Digital and SanDisk.  Google announced something called TurboQuant last week, and it seems to be more than just a pipe dream considering the effect it has had on the market.

TurboQuant is described as “a compression algorithm that optimally addresses the challenge of memory overhead in vector quantization”, and can supposedly reduce the amount of memory required for some LLM tasks by 6x, at the minimum.  That would have a massive impact on the memory market if it is easily implemented and compatible with all the various models on the market at the moment.  We’ve seen the stock value of the aforementioned companies drop by a significant amount because of this.

It is looking hopeful, but don’t bet on being able to upgrade your system any time soon.  The stock market is far more volatile than flash memory, so stocks will fall far more quickly than product prices.  There is also the warning from TrendForce, who watches the memory market very closely, who suggests that a drop in the investment price of expanding the server farms feeding LLMs will simply mean a growth in the number of new LLM server farms being built.  After all, the money is already there, so why would a company reduce their budget instead of just buying more products at the lower price?

Then again, while TurboQuant may reduce the amount of memory needed to build a new server farm, that is only one piece of the puzzle.  There are certain geopolitical actions going on right now which have had a huge impact on the delivery of critical ingredients in the manufacturing of memory which means these companies can’t make as many DIMMs and SSDs as they had previously thought.   There is also a huge bottleneck in the production of components needed to increase current power distribution infrastructure which will impact where you can build new server farms and how many the grid can power.  If that slows growth enough it could ruin the current business model vulture capitalists are using to drive the growth of AI and LLMs.