Background, Read Ignoring ECC, Writing = BAD!
Background
Ok, there’s a lot of background you all need to be filled in on as to just why a device like this is important. I won’t be spoon feeding you the gospel of data recovery places, this is all information that I personally agree with and is only confirmed by what many others have said. The short version is that when a drive of questionable condition contains vital information that is imperative to be recovered, software tools alone are just not sufficient for a hardware data recovery. Allow me to explain:
Most of the time, when a drive has some issues that prevent a system from booting, those issues are bad sectors. Unreadable sectors lead to timeouts at stages where the system is not the best at handling them. Since Operating Systems are very poor, or even harmful, at dealing with read timeouts in critical file system areas, using a tool like SpinRite to force rewrites can potentially get you back up and running, but there is a chance of losing some data or corrupting some file system pointers in the process. SpinRite (and other similar tools – not trying to single it out here, but it's arguably the most popular) can only rewrite their 'best guess' of the bad sector contents back to the drive, and since they are not file system aware, they have no idea what type of data they are rewriting. File pointers corrupted by this process may be silently disregarded by the OS, meaning you’d likely not realize some files or directories went missing until it was too late (contents were later overwritten). As someone with a formal background in this stuff (hard drive forensic analysis for the Navy), I can say that when you really care about the contents of a drive of questionable condition, writing *anything* back to the drive is the worst thing you can do! It is just not something you do when you have precious data on a potentially failing drive. If the data is not that important and you’re just trying to get your kids laptop back up and running, go for it, but if the data is absolutely critical to you, ALL STOP!
SpinRite is a great tool for maintenance and forcing a drive to map out a few bad sectors, but I don't recommend it for the recovery of vital data from potentially failing drives because it relies on that very same failing drive to store anything it has recovered.
…and by no writing, I mean no mounting either, especially under Windows or OSX. Resist all temptation to just toss that failing drive into a USB dock and cross your fingers. Both operating systems handle mounting file systems in different ways, but both do so in a manner that can further damage a drive that is already on its way out. Windows flips a few ‘dirty bits’ during the mounting process and it has to write to the drive to do so. Further, each file read from some partition types results in an update to the ‘Last Accessed’ tag associated with that file. There goes a bunch of other potential writes. Even if you were successfully reading files, suspected bad heads could be corrupting file table update writes in the process.
Read Ignoring ECC
Before moving on, a quick note about software tools attempting to read from drives in a manner that ignores ECC. Some of these tools can repeatedly attempt to read a bad sector while instructing the drive to ignore its own error correction data (ECC). The idea is that if you can get the 'raw' data from the drive, you can statistically analyze repeated attempts and deduce the most likely content of that sector. While that worked well for most drives back in 2004 (when SpinRite 6.0 was released), many modern drives respond to these requests with data that has nothing to do with the contents of the sector being read:
As you can see in the video above, there are plenty of cases that may throw off SpinRite's DynaStat engine by feeding it false information, and since SpinRite is not an imager, it is forced to write that new data back to the same sector, potentially corrupting that sector with irrelevant information it mistakenly thought was good data. *EDIT* I was able to circle back with Steve Gibson (creator of SpinRite), and confirm that pre-run checks are performed so that DynaStat only kicks in for drives that can safely do so.
While DeepSpar's more advanced / older tools can ignore ECC and perform statistical analysis, the newer RapidSpar *does not* support that function. This is because most modern drives no longer handle that command in a manner that yields raw sector data, so it remains an advanced function left up to the user discretion of their more advanced recovery tools.
Writing = BAD!
What’s so bad about writing? If the drive had a fault that resulted in bad sectors, it stands to reason that additional writes (or any additional activity) can lead to more bad sectors. Hard drives have the capability to ‘map out’ unreadable (unstable) sectors, but they typically hold off on doing so until you try to write to a sector previously marked as suspect. You can find a count of suspect sectors the drive has identified by checking the SMART ‘Current Pending Sector’ (C5) value. Writing over a sector marked in such a way causes the drive to shift that address to point to a spare sector elsewhere on the drive. The drive knows which sectors have been swapped out by the use of a ‘Growth’ defect list (G-list for short). This list is stored in a special service area of the platters, next to the other firmware modules of the drive. If the list is on the platters, how do you update it? You guessed it, another write that can potentially be corrupted, but if you mess up a one of these particular writes, the checksum of the associated firmware block (which handles drive IO) fails and you potentially end up with a drive unable to respond to commands the next time it is powered up.
SMART data revealing a sector that was previously unreadable. This sector will be remapped on its next (unlikely) successful read. If this sector is overwritten, the drive will map it out and begin using a spare in its place. Once that has happened, the original sector/data can only be read using very special tools and techniques.
Now that we've driven home the need to prevent writes to flaky drives, I should point out that there are does exist software tools that limit their functionality to only reading from a source drive. One such tool is Ddrescue, which runs under Linux and can avoid the Windows/OSX mounting issues mentioned above. Ddrescue is handy in that it can skip over areas containing bad sectors in favor of good ones, performing multiple passes in an attempt to get a more complete image in the least amount of time possible. There are still a couple issues with this approach. First, Ddrescue is not file system aware, meaning it would spend a lot of time attempting to image unused / irrelevant areas of the source drive. Second, and perhaps the most important factor, is that all software tools must all rely on the drives' own timeout for bad or slow read attempts.
A graphical representation of Ddrescue skipping around bad areas of a disk.
Since mechanical devices tend to degrade further after the first few signs of trouble, realize that with many data recovery efforts, you may be operating on borrowed time (one such case here). Software-based imaging tools are unable to perform a critical function for speeding up the dealing with those bad or slow sectors, as they cannot issue the hardware-based Reset command. Only dedicated recovery hardware can do this, which means all software tools must rely on the drives’ own timeout to occur for every single read attempt, a process that can take longer than 20 seconds *per sector*. Multiply that out and some drives would take weeks or months to image. One of my previous software image attempts took a week to reach 1%, and that was only a 400GB drive! That same drive later failed completely. If I had access to a better tool at that time, I would have recovered far more of that drive before it failed, easing my recovery efforts.
I kind of wonder how this is
I kind of wonder how this is different from a software solution if it’s just interfacing with the drive through ATA commands. For all I know, it could just be running a standard PC OS with some software on top of it.
I could see the benefit of something like this out in the field where you might not have a proper PC with you, but since it’s just ATA, I doubt there’s anything this can do that software couldn’t. If anything it’s more of a hardware dongle to make it so you can’t pirate their software. The one thing that one could maybe argue is that its SATA controller and drivers may be more consistent so you can maybe rule out any poor behavior there. If you’re that worried, though, you’re going to be using a service.
So overall, a useful tool to have if you’re out a lot and you need something in your toolbox to allow you to recover hard drives, but I wouldn’t treat it as something magically better.
I’m not familiar with hard
I’m not familiar with hard drive interfacing, but this part of the article suggests that a software solution won’t do what a hardware solution can do.
“Since mechanical devices tend to degrade further after the first few signs of trouble, realize that with many data recovery efforts, you may be operating on borrowed time (one such case here). Software-based imaging tools are unable to perform a critical function for speeding up the dealing with those bad or slow sectors, as they cannot issue the hardware-based Reset command. Only dedicated recovery hardware can do this, which means all software tools must rely on the drives’ own timeout to occur for every single read attempt, a process that can take longer than 20 seconds *per sector*. Multiply that out and some drives would take weeks or months to image. One of my previous software image attempts took a week to reach 1%, and that was only a 400GB drive! That same drive later failed completely. If I had access to a better tool at that time, I would have recovered far more of that drive before it failed, easing my recovery efforts.”
(The anon OP here apparently
(The anon OP here apparently didn't read the article).
That said, I have yet to see any software recovery app that can instruct the SATA controller to issue a hardware reset to the drive. In fact, SATA controllers will typically hang until they get a response from the drive, meaning that even if a piece of software was able to direct a reset if the drive was taking too long to respond, that command would be ignored until the controller hit an internal timeout or received a timeout-related (read error) reply from the failing drive.
If you've tried working with unreadable sectors you've likely seen this in action. All other drive activity halts and the system hardware drive access light remains lit solid. With some controllers, even activity to *other* drives on other SATA channels halts until the outstanding IO has been serviced in some way (successful read or read error reply received). This is mainly because PC hardware is simply not purpose built for data recovery. It just gives the drive as long as it needs to provide an answer to the request, since it assumes that it needs *all* data to be successfully read. In that respect, data recovery is the art of quickly working around the bad parts within a reasonable amount of time.
The Rapidspar has a highly
The Rapidspar has a highly specialized and advanced ATA controller that has many features and abilities a standard ATA controller does not have. It can rewrite firmware on the drive, selectively turn off and on heads, reset the drive on the fly, and a host of other critical commands that are far beyond the ATA controller in a PC.
Fantastic article and review
Fantastic article and review Allyn. Data recovery is a complex operation, and having the proper tools is critical. This tool you reviewed is amazing. I am partners with an data recovery firm for my consulting business, but this is an intriguing option for shops and medium to large IT departments within companies. I often wonder what drive recovery percentages involve actually needing a clean room environment, compared to this level you talk about here.
Thanks. I don’t know the
Thanks. I don't know the percentages, but this sort of device certainly helps pull files or images from drives that wouldn't cooperate even with a standard write-blocking hardware imager. That has to increase your odds, perhaps by more than the larger data recovery places would be happy about, as it might steal away some of their easier work.
It's certainly not going to make the big operations disappear. Clean rooms aside, there are a lot of logical-only recoveries out there that need the 'big guns' brainpower and experience of the large recovery firms. There are plenty of ways to corrupt the contents of a mechanically sound drive in a way that makes recovering the files extremely tedious and time-consuming, and not all logical recovery software can handle all situations. As an example, I recently recovered an SD card with a corrupted partition table. It imaged just fine, but it could not be mounted and no logical recovery apps were able to lock onto the correct partition offset. The answer ended up being to quick format (!!!) the SD card and re-running a logical recovery on *that* image. All files were recovered, but it was knowledge of that particular trick that made the magic happen. Knowledge and experience typically trump the tools when things get squirrely.
I won't even get into RAID recoveries (saving that for the next article), but that takes an even higher level of experience and know-how. I've done such recoveries myself, and it was an extremely complex operation that required the coding of my own tools to automate some of the work.
I couldn’t agree more. There
I couldn’t agree more. There is no substitute for experience. Also there is the business side, and many small IT service firms just won’t have the capacity that a larger outfit does that does nothing but data recovery.
Formatting is data destructive. There is NEVER any reason to do this on a drive that you need to recover data from. In any case recovering a “corrupt” partition table is trivial. I do it almost every week in one or more storage forums. I recommend DMDE for this purpose.
how does this differ from an
how does this differ from an $80 copy of spinrite? I don’t see how this would work any different except for being more expensive and coming with adapters.
SpinRite can’t issue hardware
may possibly be writing back incorrect data to modern drives that do not correctly handle read-ignoring-ECC commands. I have a query in with Steve Gibson on this one. *EDIT* I've confirmed with Steve that SpinRite does some pre-run checks to only use DynaStat when it is safe to do so.All points brought up in this article. It may be helpful to read it prior to commenting.
Further, if an $80 item was the solution to all problems, data recovery would not be such a large business. I realize that it works for a lot of folks, but people should be aware that there are risks involved, especially if the drive is about to fail completely. A drive that I was repeatedly imaging (at high speed, with the RapidSpar), completely failed after about 20 hours of work. That very drive would not have made it through a single SpinRite level 4 pass and would have died long before any data could be recovered. With the RapidSpar, I had a complete copy of that 6TB drive (minus 1024 bytes) overnight.
I whould be happy if we could
I whould be happy if we could target a drive in SR! 🙂
For what it is, is a good tool.
its*
its*
SpinRite is NOT a data
SpinRite is NOT a data recovery program. At no point does the program prompt to copy a single sector from the failing hard drive to a healthy drive. It doesn’t even take the time, that I have ever been able to see, to test and confirm that each read/write head is actually properly reading and writing. I’ve seen thousands of drives that are falsely showing bad sectors because of weak PCBs, weak heads and even firmware issues which, when corrected, read 100%. If you don’t fix those issues first and then try to remap the sectors, you are just going to make things worse, without a way to undo the changes.
If you insist running SpinRite on a drive, at least follow the advice given in the manual and make sure that all the data on the drive is first backed up.
Spinrite is potentially data destructive. You must NOT use it on a failing drive.
In any case most of the claims made in respect of Spinrite haven’t been relevant for the past 30 years.
Deconstructing Spinrite:
http://www.hddoracle.com/viewtopic.php?f=181&t=2929
Starting the article I was
Starting the article I was expecting “it’s just a little Linux PC in a box running dd with a write-blocker on one end”, but pleasantly surprised to see this is a remarkably well thought-out device with actual advantages over a roll-your-own software solution.
Dat price tho.
Dat price tho.
We purchased one of these in
We purchased one of these in March of 2016. We’ve been able to recover many hard drives that no other process would touch. Yes, the price is steep, but we were able to recoup our investment in less than 60 days!
We purchased Rapid spar tool
We purchased Rapid spar tool in 2 months back. Presently I am using. I am in this field since 10 years. This tool is not worth to purchase for the price.My suggestion is not to purchase this tool. Reviews available in the site also fake.
It’s very hard to find
It’s very hard to find reliable reviews in the DR industry.
Thx for sharing