marlyse.comme, myself and my life
Wednesday, November 15, 2006
Recovery
Unlikely things do happen. This led me to having lost 140 GB of crucial data in the first place. But now, after a week of futile recovery attempts the other, incredible - and at this point also unlikely - next event occurred : I was able to recover ALL of my data. Yes. All of it.
How did I get here, and what was actually the series of unfortunate events which have me - a backup crazy one - loose all the data?
Yes, I am somebody who daily backs up 2 machines (laptop and desktop) to external firewire drives, the systems separate from user data (thus every night a total of 4 backups to 2 different firewire drives). Next to these 2 machines I have 3 other firewire drives hooked up to my desktop (one just filled with music, the other with photos and the other is reserved for client stuff) and also these backup every night to a 3rd firewire drive. Plus our iMac in the kitchen which writes also to that 3rd drive. So every night I have 8 backups going, like clockwork, due to scheduling in SuperDuper! And from time to time I copy these archive files (they are sparse disk image files) to separate firewire drives and these mirror drives then are ready to go to the bank safety box - just in case the house burns down. When all your work is digital in one place, you begin to value the data and take (hopefully) measurements - because insurance will pay nothing of it, and this is the reason we have this whole firewire system going with bays for the cases and removable cases which slide easily in and out and then get locked down to go live.
That’s the setup and the background. And I still managed to loose all data.
It started off that suddenly a couple nightly backups where erring out and it seemed that one or the other disc image had gone corrupt - which was weird, because the next time writing to it all seemed again fine - or the drive bay was failing which seemed more likely as Bear said he had been having troubles with some hardware piece of it. Also, my powerbook needed a thorough cleanup because free disc space on it had shrunk to 4 GB and I was still carrying obsolete data around from my Pismo (which is 3 or 4 laptops earlier) and had OS 9 stuff on it which I had not touched in at least a year or two. And then my backup disc reported to be full and thus I began cleaning up my laptop by burning archives of old data which I no longer needed but didn’t want to trash while saying bye-bye to other, truly obsolete system software. While I was doing that, suddenly my Wacom tablet on the desktop stopped responding and re-installing the software did not handle this weird problem - and it was the computer having the problem as hooking up the Wacom to the laptop worked just fine. The dual G5 - my desktop machine - had been the third machine into which I had immigrated my old computer data during install and the Library folder and Preference folder where bloating considerably.
So there was a definite moment of : it is time, I need to do full cleanup on both machines and the desktop needs a total clean install - never had it before. Even though I dreaded the day or two spent going through freshly installing all of the software I use and to adjust it to my workflow, I knew it needed to be done.
First, I made a fresh mirror of my backups, i.e. copied each user and system sparse disk image to its mirror drive, for both, my powerbook and my desktop. The desktop file at this point was 140 GB big and it took almost 4 hours for it to copy. But I saw it copying, I watched it from time to time and so I felt safe, with 2 identical backups, to fully erase my desktop and freshly install the system when I did.
Once the system was installed on the G5 - now it was just around midnight, Monday a week ago - I was going to mount the mirror copy of the user folder and bit by bit move the important files over while leaving the bloated accumulation of unnecessary data behind. I had the mirror backup hooked up directly to my G5 so this part of the process would go faster.
Before doing that though I went downstairs to put the original firewire drive back into the backup system so as to enable SuperDuper! writing to it during the coming night. While I was downstairs I realized that I’m not interested in having the new backup writing into this bloated old user folder backup but that I rather wanted to quickly create a fresh sparse disk image so it could write a clean and fresh backup. And that is the moment that I trashed my original backup of the user folder - bathed in the knowledge that I had the mirror copy upstairs, I felt it was totally okay. Trashed it and emptied the trash to free up the space on the firewire drive.
Went upstairs and clicked onto the sparse disk image on the mirror drive to mount it to begin the copy process of the user folder. And while I double-clicked the file, I saw the 140 GB file size suddenly shrink to 4 GB and then an error popped up, that it was not a mountable disk image. And THAT was the exact moment I knew that I had lost all my data.
I do not know what corrupted the mirroring processes - it did copy for 4 hours and it showed the correct 140 GB size after copying in the Finder. So here I am at a loss of logic, the only explanation I have is that the physical universe played a trick on me. MY mistake in this whole process though was not to mount the mirror file BEFORE trashing the original file. Or even mount and copy all needed data back to the desktop before trashing the original backup file. THAT was my side of dumbness.
And this is where my week of futile recovery attempts come in. ALL applications I found (on the Mac) only recover bits and pieces and in the process dump all folder structure plus all file naming and in the end you are left with numbered files which don’t relate to each other and you have to open each and every one to find out a) is it corrupt and b) what is its content. Tedious to impossible, this process, when it comes down to 100,000 files. It was so frustrating. I did not do anything after emptying the trash, did not write to the drive, so my hopes were there that it should be recoverable, even though all sites found via Google told me otherwise. But worse, none of these applications seemed to be able to find and recover sparse disc images!
And then I found one application which beat all others : Stellar Phoenix Macintosh. This was the ONLY software which found and displayed the disk folder structure AND quite quickly found my huge file in the emptied trash. Seeing this, I went ahead and payed the ~$130 for it. But then new troubles began. It was sloooow and worst of all, it seemed to hit a 2 GB limit and after that continue READING the file but no longer write and recover. Trying then to recover bits and pieces of files I found displayed all resulted in corrupt files and nothing that I could use. It made me mad. Not only did every scan take 5 hours+ but selecting the files to recover resulted in just as slow actions of the application showing me that it had so-and-so-many files selected ready to recover and after these additional hours of waiting it recovered only a few before hanging. So trying the file by file approach did not seem to work plus I was never interested in that as the firewire drive always only had sparse disk images on it and I knew the other files was just some visible content of these disc images. But mostly mad it made me because the 140 GB file was sitting there and grinning at me.
In my frustration I had several email exchange with technical support at Stellar. I was so frustrated to only recover a few corrupt files and not the big one, at one point I said I want a refund. But then, after trying to work with other, in my case useless recovery software and realizing that on the PC there never were 2 GB file size limitations and knowing that there used to be a PC version of Stellar Phoenix which could read HFS+ drives (Mac drives are formated as such), I wrote again to Stellar Phoenix, telling them exactly what I was trying to recover and asked them if the PC version is able to do that and that I am much more interested in finding a solution than a refund.
Within a few hours I had a response and a download link to the PC version and step-by-step guidance on how to create a specific registration text file (the software gets coded to a single specific PC). There we ran into troubles, it didn’t do as outlined in the email (later I found that it seemed to have missed something minor but crucial in the outline) but after several back and forth I received a phone call from their support executive (the company is in India!… not sure about where their support is) and I was walked through some specific process to generate the needed file. And then within minutes I received the registration code.
Then it only took minutes to locate the needed file and to begin recovery. After just a few seconds the same seemed to occur as on the Macintosh, the software did not give ANY feedback if it was progressing or not, and the activity monitor on the PC showed only that the application is not responding, but I saw again that the firewire drive was being accessed and after a while I could see that the disc space on the selected recovery drive was getting smaller, obviously SOMETHING was being written to it. But other than that, screen redraw did not work on the PC, I could not even close the Finder window. But after seeing that obviously something was happening, Bear shut off the monitors and we let it run - it took about 5 hours or more and then suddenly the activity to/from the firewire drive had stopped.
All excited I turned on the PC monitors and located the recovered file - and halted in dismay. It showed to be only 47 GB - where did the other 90 GB go to? But then I remembered that sparse disk images only write to the file while archiving but never reduces file size, thus begins to bloat over time. IF it had bloated to this extent and IF I could mount this file, all would be good. Thus came the slow, slow copy from PC to Mac via Timbuktu.
We went for dinner meanwhile but it took even longer than that. But at 8 PM last night, the file was there and the important moment was here. Would it open and mount the virtual drive or not?
Yes, it did. Even hours after the fact I can continue feeling these waves of relief washing over me.
Now I am curious as to why only one software company has gotten it right. This piece of software is the ONLY one which I found which could read and display the directory tree on OS X correctly. And it’s the ONLY one which I found which read sparse disk images and displayed them as such. All of this NEXT to the fact of service way and beyond what I usually expect to get from support (to call and walk me through and help UNTIL a working solution is found PLUS hooking me up with the PC version when the Mac version failed the task). Big kudos to them!
When I had searched for this huge file, I had found all the older incarnations of the same file, growing slowly over time - Stellar Phoenix Macintosh was able to track down all of them - and I had them all at my finger tip, ready to recover (now, if the OLDER files would have come back mountable, I do not know). How come everybody else says, nah, once deleted under OS X it’s gone, you can only get bits and pieces back and all in one big mess with no original file names attached?
I know the upcoming Mac operating system Leopard will feature something called TIME MACHINE which allows you to recover deleted data, and somehow it reminds me of the same technology, only that Apple of course puts a futuristic and sugar candy user interface around it all… but, next to this musing, I am just plain relieved and happy to have all my data back.
While it is gray, windy and stormy outside - this might be the perfect day for our first fire this winter - I will be copying diligently folder for folder from the mounted disk image back to my desktop, savoring each moment of this incredible recovery.
Technorati Tags: performance, challenge, code, good to know, tips, Stellar Phoenix Macintosh, data recovery, trashed data, kudos, Stellar Phoenix
created just before lunchtime - it was 11:15 am to be exact | trackback |
2 Comments »
Leave a comment
Well, it was more than 47 GB - once the sparse disk image was mounted it was 72 GB of data. And of that I was able to copy and recover 71 GB - the main things which failed were some types of aliases and then 1 specific folder was only fluff and disappeared when I clicked on to it - and it’s this site, yes, marlyse.com with all it’s sub-stuff and that could have easily mounted to 1 GB of data. Not sure exactly what I lost, the latest version is here and live, safe in Dallas and online - but I know I was going to fix my photo gallery and possibly, but hopefully not, the backup of that broken gallery was in this specific folder. But even if so, the pain is small compared to the remaining 71 GB which have been saved.
Comment by marlyse — Thursday, November 16, 2006 #
[…] The application I found which saved me from rolling up into a ball and to give up was from Stellar Phoenix. It was the ONLY application that was able to find the deleted disk image! And not only to find it but to be able to figure out the folder structure and to retaining this when recovering. As my disk image when expanded was larger than 2 gigabyte I ran into some additional (Mac) problems but I received exceptional support from that company, including a long distance phone call and in the end I was probably able to recover 4/5 of the original data. You can read more on the original stupidity and following recovery on my personal blog - more than a year later it does seem all pretty funny. […]
Pingback by mStudiosTALK | Fresh out of Recovery. — Thursday, February 28, 2008 #