Community involvement, possible causes, temporary fixes, and Samsung’s statement
It began with this now beast of a post on Overclock.net. Users sharing info about seeing slow downs, etc. A couple of forum users went as far as to create tools for analyzing the speeds of in-place files (Techie007's SSDReadSpeedTester), as well as a lot of excellent experimentation, analysis, and additional tools by BrainSplatter (gotta love forum nicknames!). The thread is approaching a thousand posts, but I spotted this gem, which I believe best demonstrates the effect of read speed reduction over time:
The above test reads all files from the drive and sorts their respective read speeds by date. As you can see in the above example, speeds steadily tapered off as weeks passed, eventually falling to a steady ~100 MB/sec at the 36 week old point. This is by no means an absolute value, as others who ran the same test on their own SSDs saw reduction over differing (shorter or longer) periods of time.
By this point you may be asking what the heck can be causing this? Here's my hypothesis:
- Over time, flash cell voltages slightly drift.
- Flash circuitry is designed to compensate for this by varying the read voltage thresholds and using varying levels of error detection and correction mechanisms.
- Some unforeseen coordination issue between the flash and the controller, in the face of #2 above, is resulting in slower than normal read speeds for flash that has been storing data, undisturbed, for weeks or months.
Now I know there are many folks from many threads, that having read the above, are saying 'no way, it has to be worse than that!'. No, it doesn't. Why? Because out of all of the dozens or hundreds of folks running these tools and tests, not one has reported a single unreadable sector. Think about it, dozens of people are intentionally reading all data from these drives, which is way more than a user would be doing in typical use, and despite the slowdowns – no errors. If the flash cells or stored voltages were actually degrading so badly over time, the oldest stored data would likely be unreadable. Instead we see a steady speed taper down to a seemingly fixed (~50-100MB/sec, varying with drive capacity / die count) read speed. That indicates to me that the issue is one that can be corrected – maybe even in a manner that would snap speeds back to near full simply by updating the firmware. This is certainly possible, as we've seen it happen before.
How do you get the speed back? Overwrite the files. Yup, it's that simple. Re-writing the flash resets the cell voltages and resets the timer effect on the slow down. I do ask that instead of forcing these rewrites by running defrag on an SSD (I shudder at the thought), just copy the really slow / old files to another drive (or directory on the same drive), and then move those files back to the original location, overwriting the originals. This is much nicer to the flash than a defrag operation, which tends to be much more fragmenting of an operation than a sequential file copy.
Or you can just wait for a fix. Yup, there's one coming, here's a quote straight from Samsung:
Samsung recognizes the seriousness of any potential degradation of read performance on old data. We are testing and validating the circumstances that potentially cause this performance drop and will work diligently to resolve the issue.
Realize this is going to take them some time regardless of how quickly they can reproduce the issue and crank out a firmware. The firmware has to pass QC, which can take a few weeks time even for a simple fix. Also, before those angry at the seeming non-response from Samsung over the past week grab their pitchforks and board a flight to Korea, consider that last week was Chuseok (basically Korean Thanksgiving), so all of the big wigs needed to get the ball rolling on this were home with their families. Now that they are all back in the office, I can assure you that meetings are taking place and the ball is most certainly rolling on a fix. The vibe I get is that Samsung takes these things very seriously, and I suspect we will see a fix shortly.
Stay tuned. More to follow, folks. Your game loads shouldn't take those few extra seconds for much longer!
So Prior to Samsung jumping
So Prior to Samsung jumping through the hoops of the QC process, and corporate BS. Where do I get free software to rewrite my 840 Evo occasionally, and get my speed back? I’ve noticed that my system boot time decreased dramatically over the last few weeks. I realize that the drive will actually die earlier in doing a full rewrites because of this.
Depending on the
Depending on the warranty/implied warranty, rules and regulations Samsung may be forced to offer a longer period on the affected model/s, if the firmware solution results in such increased re-writing/extra ware to maintain data integrity. TLC drives should come with enough over provisioning to allow the initial warranty to be extended, should there be a need, or requirement/judgment for a warranty extension. Any TLC based SSDs in the future may need a little more extra over provisioning.
If the firmware fix is not soon, Samsung should have some free software made available as a temporary solution for speed degradation issues, and some extra warranty time, if the firmware fix results in excess ware and tear on the affected SKU/s.
Maybe tiered storage software will make it into to PC/Laptop market from the server/HPC market, Hierarchical storage management, systems. Where old stagnant data is moved from SSD to hard drive, automatically based on use algorithms. The user should have the ability to have the SSD’s software/firmware rotate the older data off of the SSD, before it becomes so stagnant that the SSD’s controller becomes overtaxed with error correction loads, leading to the speed issue. Maybe even some software/driver software, that can create a mirrored partition on a hard drive that acts as a mirrored store for all/some SDD writes, and any files that are beyond a certain age will have the mirrored file on the hard drive overwrite the stagnant file’s content on the SSD in the background, to top of the TLC’s charge/state. With this type of mirroring, the firmware/software could keep track of the amount of Error correction the SSD is using, and if it reaches a certain threshold value in an old SSD file, it can redirect the reads to the hard drive mirrored copy, on a block by block basis, to at least keep the transfer from degrading below even a normal hard drive’s transfer speed.
Gaming PCs with large amounts of game libraries would benefit from a tiered Hierarchical storage management system, keeping the most recently used fresh data on the SSD, and automatically managing the file stores to prevent SSD speed degradation, and excess SSD ware and tear.
All I can say is that I am a
All I can say is that I am a happy user of many Samsung products. The TV may be old, but works just fine. The monitors likewise. I have had optical drives that just did the job as you would expect.
I even have an old Samsung netbook which gets lugged around everywhere. It just works without fuss
In my HTPC I have a Samsung 840 SSD and have not noticed this issue, maybe none of my data is old.
Really, so let’s go cherry
Really, so let’s go cherry pick, that Samsung 840 SSD is it fabricated on a larger process node, what year was it manufactured? An Old TV, is it CRT or LCD/other! Your testimonial, does it even include the product in question. User satisfaction on any outdated products, that may no longer be made, should not factor in as much, if at all, as with the more current products that are still in production.
Samsung’s TLC 840 EVO on the firmware/engineering side, did not receive very much TLC(of a different kind) from Samsung, so I’ll go with SLC for the 100,000+ P/E cycles, And stick with spinning rust for the longer term storage.
Tape is still an option for backups Too, when losing the data means losing your A$$.
Samsung better make good use of more of the third dimension, with that chip stacking NAND, and forgo some of the process node shrinks, to keep its flash based storage from becoming very slow DRAM.
So to confirm I understand
So to confirm I understand this news update correctly, to fix this issue all one has to do is keep his 840 SSD cool? So placing it right behind working ventilator could do the trick?
First, thank you for the
First, thank you for the article. I have not seen any other review site go into detail about this issue. Other sites have just skimmed the surface mostly.
Second, Is it possible the Samsung enterprise drives that also use TLC nand suffer from this issue? The models are 845DC EVO, the PM853T(oem version of 845DC EVO) and the PM843.
These drives haven’t been examined for the issue yet. I’d be very worried if the tlc enterprise drives didn’t have this bug discovered during their development.
I’m evaluating that now, but
I'm evaluating that now, but our samples haven't been here long enough to have stale data on them.
Nice. Very nice. Good to know
Nice. Very nice. Good to know you are trying to cover as many angles as you can.
In the quote in edit two,
In the quote in edit two, they mention the 840 EVO twice, but not the 840. That has to be an oversight right? I have a 250GB 840 and I have noticed it being slow, but didn’t know what to do about it until now. (Thanks for the disk fresh tip btw.)
FYI: a couple of years ago I
FYI: a couple of years ago I put Win7 Pro, 32bit on one of the then new Kingston 64 GB SSD’s and, after all proved installed & running well, I used an Aluratek MACHINE to clone it to a WD 320GB 7200rpm 2.5in. HDD. After booting the PC with the HDD & confirming it all worked, I used a defragger to look at it before defragging. As expected, almost 100% fragmented. Proving what someone else wisely said, that SSD Controllers “lie” to their systems (a good thing)!
After a long defrag session, the HDD ran much more quickly. And worked well as a “dated” backup.
Incidentally, I have built about 6 or 7 PCs with these WD 320GB HDD’s which have been in continuous use for over 5 years. No signs of failure. I’ll bet that’s because since 1995 I have never installed a HDD without a fan bolted to its bottom, usually running thru a resistor to slow it down to reduce noise. Always at considerable effort modding to get them mounted. But proved worth the effort.
BTW on the “antique” Kingston 64GB SSD after noticing a slowdown, I ran my SpinRite at Level gasp! 4. New speed regained (it was about 8 Mo’s o!d). Since then have run Level 1 once a month, & level 2 a couple of times and been running well. Also do SpinRite on my stock Dell laptop HDD every few weeks because I don’t trust its Seagate 750.
** Edit 2 ** (26 Sep)
** Edit 2 ** (26 Sep)
New quote from Samsung:
“We acknowledge the recent issue associated with the Samsung 840 EVO SSDs and are qualifying a firmware update to address the issue. While this issue only affects a small subset of all 840 EVO users, we regret any inconvenience experienced by our customers. A firmware update that resolves the issue will be available on the Samsung SSD website soon. We appreciate our customer’s support and patience as we work diligently to resolve this issue.”
** End edit 2 **
Ha ha ha ha ha “small subset”!
With the new firmware will
With the new firmware will the data be lost? IE will we have to format the drive to install the new firmware?
I think it has been stated
I think it has been stated that no data has ever been lost. Thus just the /relatively/ “slow” transfer or access times. So if the new F/W requires reformatting, it may take considerably longer to back up and reinstall. But so far not one “bit” of evidence or a claim of lost data.
(Ya see what I did there)?
What’s the strangest thing
What’s the strangest thing here is that a “severe” issue like this only gets discovered after about 1 year?!
The conclusion is that degradation start after just a couple of weeks. -Has anyone that run Diskfresh(or similar) some time ago already starting to see the read speed go down again?
So I have an 840 Pro, and I
So I have an 840 Pro, and I know the consensus seems to be that they are unaffected, but I just ran a defrag and my ultra slow bootup seems to have disappeared…
It may do us well to keep an
It may do us well to keep an eye on the 850 Pro. It still uses TLC but maybe their claim that the 3D Vertical stacking “locks in” the bits, fixes the problem? I think they also use a thicker process that would help the bits to remain at their original levels and need less error correction. Maybe better R/W longevity too? ??
So today there is a “Samsung
So today there is a “Samsung SSD 840 EVO Performance Restoration Software” available, but NOTHING for the 840 Basic..maybe next year Samsung? Or 2017? Take your time, it’s not like it’s important or anything. I give it another week, then this 840 comes out of the laptop and gets a spectacular Viking funeral, because the way it is now it’s unusable. Lesson learned, never a Samsung product again. Already got a little revenge when my daughter wanted a Galaxy S5 for her birthday, but I bought her the LG G3 instead. Smells like VICTORY!
My slow 840 non-EVO sits in a
My slow 840 non-EVO sits in a acer netbook from 2012, and you can be sure that IF Samsung releases that performance restoration software for my drive it will not support the netbook resolution =) Same thing happened with Magician software, I was stuck with old firmware because I had to use an old Magician version which failed to update the firmware all the time. Samsung spent over a year correcting the resolution problem, and still every time I start magician it throws a warning about the resolution..works though, but come on Samsung, really? So yeah, that’s my two cents. A fix is MAYBE underway for non-EVO and most likely will not work for netbooks until 2018. Thank you. Crucial, here I come.
So this issue only affects
So this issue only affects read speed correct? What if your write speeds are greatly reduced but your read speed is fine?
Addressing now the issue but
Addressing now the issue but i already knew it;
The read speed of one of the partition, that has been unused after creating and filling it, dropped to 2.5mb/s;
i’m searching for scans on image search but i never saw a speed so slow like the one i achieved.
In this moment i’m restoring the data using the samsung tool, it’s near the end so i think is fixing the problem.
restored; from 2.5 ~ 4 ,
restored; from 2.5 ~ 4 , using hddscan (a diagnostic tool, not a speed test) i get a fixed 480 ~ 490.
Is this an issue with windows
Is this an issue with windows only and not with linux? I am a linux user and formatted this 840 evo with ext4.
Well, as we all know, the
Well, as we all know, the issue persists after the “Performance Restoration Tool” fix and firmware update. Over time, stale data is losing performance, and we’re supposed to see another release of a restoration tool and a new firmware sometime soon.
Meantime, I just wanted to mention that the recommendation of using MyDefrag with the ‘data disk monthly’ script still works like a charm. I just ran it on my 840 Evo 1Tb and it restored performance – for now.