Filed under: Odds and ends
Backups save the day after data loss
About 13GB of data was accidentally deleted from my account on my Mac yesterday. Fortunately, due to an aggressive backup system, I was able to recover all of the files.
I was trying to test out a tip sent to the TUAW team, which led me to log into (and out of) the "Guest" account several times. One of the logins seemed to hang, which triggered a memory about a data loss bug in Snow Leopard which I had read about (and which Apple is now acknowledging in "extremely rare" cases, per Cnet). I immediately reached around the back of my iMac and shut it down.
At first I thought everything was OK, but then Dropbox started asking me to configure settings, as if it was the first time the app was run. Fortunately, due to the new "LAN sync" feature in recent Dropbox betas, the iMac copied my Dropbox files from my MacBook Pro on the same network in a matter of seconds, rather than having to download almost 2GB of data from Dropbox's servers.
I thought that was the end of the problems, until I saw that my Terminal customizations were missing. Then I noticed that a few other applications were not functioning correctly. Suddenly I realized that I had most likely suffered more data loss than was immediately obvious.
Read on to see what had to be done to recover after the data loss occurred.
SuperDuper is my main backup tool, and it runs every day at 9:30 a.m. on my computer to clone the entire hard drive, so I mounted the drive that it targets and compared the size of my home directory. The current version was 13GB smaller than the backed up version, even though I had not purposefully deleted any files.
Being a command-line nerd, I opened Terminal.app and issued these commands:
cd /Users
find luomat -print | sort > /tmp/luomat.1
cd /Volumes/iMac_Backup/Users
find luomat -print | sort > /tmp/luomat.2
bbdiff /tmp/luomat.1 /tmp/luomat.2
(Note that "luomat" is my username on the iMac.) What did these commands do? They printed out a complete list of all of the files found in my Home directory on my iMac, and then it printed out a complete list of all the files found in the Home directory on my backup drive (/Volumes/iMac_Backup/Users). The "bbdiff" command then opened the two files and compared the differences using BBEdit, which made it easy to see what was gone.
Originally I had planned to manually copy the files from the backup drive, but there were just too many of them. So I fired up ChronoSync 4.0 (and made sure that I had updated to the latest version). I selected the two "luomat" folders in ChronoSync, chose "Synchronize Bidirectional" and made sure that I did NOT check "Synchronize deletions" (since I wanted to make sure that the files on the backup drive were not deleted). Then I ran a "Trial Sync" which showed me what would be done if I chose to do a complete sync.
After the "Trial Sync" ran, I opted to just copy files from the backup drive to the iMac drive. I did this for two reasons: 1. there was nothing on the iMac drive I wanted to copy to the backup drive, which I wanted to keep as-is. When SuperDuper runs again the next time it will take care of updating the backup drive. 2. Dropbox had already updated the files on the iMac drive to the latest version, so I did not want to mess that up either.
ChronoSync ran fairly quickly, and copied the missing data as fast as it could be transferred over USB. I re-ran the Terminal commands to see if anything else was missing, and determined that I had recovered everything that I needed.
Whew.
I can't stress enough how important it is to have a regular, automatic, complete backup of your entire hard drive. Yes, Time Machine (which I do not use, for reasons I will explain below) can also do this, but I was looking at an estimated 14,000 files. Time Machine would have been completely impractical. I didn't even know which files were missing or from where!
You've heard it said many times, but I'll say it again: backup, backup, backup. I have several layers of backups which run every day. Here is how I protect my data:
1. Apple's Backup.app runs on both the iMac and MacBook Pro every day. I use it to backup to my .Mac (sorry, "MobileMe") account, but you can use it to backup elsewhere. This backs up my iCal and Address Book information. This information is also synced via MobileMe, but keeping backups is important in case you realize that corruption has snuck in.
2. SuperDuper (or CarbonCopyCloner) runs every day to backup the entire hard drive.
3. Dropbox is where I keep my "daily" files that I am working with on a regular basis. With 2GB available for free (up to 5GB with referrals), this gives me plenty of space for documents, spreadsheets, the occasional picture, etc. It's not going to hold your entire iPhoto or iTunes library, but it gives you two very important things (even if you don't need to syncing across computers): online backups, and the ability to "undo" changes made in the past 30 days. Also, it works like a regular folder, unlike iDisk. You can upgrade your storage with Dropbox for $9.99/month, which gives you a 50 GB allocation.
4. Gmail. "What? Gmail isn't a backup system." You're right, and what I really mean more than Gmail specifically is IMAP, which maintains your email on the remote server instead of having the only copy of it residing on your hard drive. Gmail is a good choice (recent outages notwithstanding) but most decent ISPs will give you IMAP support. The nice thing about Gmail is the excessive amount of space they give you so you don't have to worry about deleting email if you don't want to. The big take-away here is: if email is important to you, don't leave the only copy of it on your hard drive. Functionally, IMAP is an off-site backup system for email.
If I was to add another part to the system, it would be Time Machine (although maybe not a Time Capsule!) to give an even more well-rounded backup coverage. And someone is bound to mention off-site backups to protect against theft/fire/etc, which is another area that could be improved upon.
That said, what I have is probably more than 75% of the computing world, and yesterday was a reminder for me of why backups are so essential.
![TUAW [Cafepress]](http://www.blogsmithmedia.com/www.tuaw.com/media/tuaw-cafepress-promo.png)
Reader Comments (Page 1 of 1)
Joe S. said 10:30AM on 10-13-2009
Really informative piece. Question though--what's the need to have both ChronoSync and SuperDuper? It seems like overkill.
Reply
Joshua Ochs said 10:31AM on 10-13-2009
Why not Time Machine? You can restore an entire directory with ease - you certainly don't have to go file by file. In the specific case you had - restoring a *partially* deleted home directory, yes, it would have been inefficient, as it would have copied more data than strictly necessary.
It's free and by far the easiest backup system to not only set up, but *keep using*. Most people are not going to set up SuperDuper on a repeating schedule, or remember to manually back up. Time Machine is completely invisible - and is its genius. Set it and forget it at its best.
And despite all of the woes with the Time Capsule, I've had mine running for over 19 months constantly, and (knock on wood) it's never had a problem. I always did worry a bit about the lack of a fan, but in extended operation it never gets more than slightly warm. Meanwhile, it backs up four Macs, my work Thinkpad, and a couple SuperDuper backups as well (for large datasets that I don't need versioning on - like video).
Reply
djfred said 11:17AM on 10-13-2009
"Most people are not going to set up SuperDuper on a repeating schedule.." Why not? It's really easy to do. Two clicks in the latest version in plain English.
I'm not sure why Time Machine didn't get the nod either as it's an extra layer and as you pointed out free and easy to use. But it it shouldn't be anyone's only solution. Sometimes you need a clone.
Pradador said 11:35AM on 10-13-2009
@djfred: It's true, you can never go wrong with a clone.
I used to rely on RAID-1 + Time Machine for backups but since RAID-1 is a mirror of your original drive, in cases like this the data would be gone on both drives. I accidentally deleted some of pictures from my iPhoto library once. With Time Machine no data was lost as it was updated constantly, but it was kind of a pain to recover the files manually. It would've been nice to have a clone from the morning that I could just swap and forget
In the end, it's good to use the right tool for the right job:
RAID to protect from down time due to hardware failure.
Clone to protect from accidental data loss.
Time Machine to archive all your data.
DrWho said 11:40AM on 10-13-2009
I used to clone to another drive with super duper now that drive won't mount any more. So much for that idea!
cydeweyz said 10:46AM on 10-13-2009
Get a Readynas and run Time Machine. That way you have dual backup drives, plus some- http://www.readynas.com/?p=1097
Reply
Mike said 12:37PM on 10-13-2009
CrashPlan. It is never mentioned by TUAW (surprise). It's free and will backup your data to another computer, an external disk etc. Pay for the pro version and it uses a file alteration monitor to immediately backup files when they change. It's a lot easier and reliable than what was discussed in this article.
Reply
Matthew Dornquast said 1:52PM on 10-13-2009
To be fair, they did mention us in 2007 - but for some reason we're old news now - even though we've added tons of new features over last two years and continue to win in the space. Macworld picked us as the preferred online backup solution this month.
Brad Owens said 11:22AM on 10-13-2009
I had a very similar thing happen to me that scared me into re-evaluating my backup plan. I completely overhauled it and now keep backups of backups.
I detail my backup system here: http://bit.ly/10KpXT
Also, for a review of how I use DropBox: check here: http://bit.ly/97VDJ
Reply
Jordan said 11:22AM on 10-13-2009
C'mon, this isn't an extremely rare problem. If you were able to reproduce it by only logging in and out "several" times, then it can happen at any moment to anybody. Stop trying to downplay this severe error by Apple and major flaw in Snow Leopard
Reply
shayes said 12:21PM on 10-13-2009
"(and which Apple is now acknowledging in "extremely rare" cases, per Cnet). "
I would say the author was merely sharing from another source that the issue is "real" - what gave you the impression that he was downplaying the issue at all???
Poor blogger's can never catch a break:)
As an aside, just to check, I tried to (re)create the issue on one of my macs last night and couldn't - perhaps I didn't log in and out a sufficient number of times ....but I then disabled the guest account just to be on the "safe" side....
jon_thomp said 12:08PM on 10-13-2009
I use both Time machine and SuperDuper! together for a complete backup solution on my MacBook Pro - Time Machine at home for the ability to reach back to old versions, and a USB hard drive SuperDuper! backup at work (manually updated every day or so) to give me an off-site backup. I also carry a USB Key (encrypted with TrueCrypt) around with me with rapidly changing and critical current files that I might need on other machines. That one is kept up to date with an rsync script, so it's quick and easy to maintain.
Reply
Trane said 12:23PM on 10-13-2009
Could you say why you don't use Time Machine? I only see your statement that you will explain why below, immediately followed by your assertion that it wouldn't have been practical in this case anyway, and later your statement that it would be a reasonable next-level of backup. I'm confused.
I'm just getting ready to setup Time Machine, and I'm curious why you don't think it is a good idea.
Reply
PSM said 12:51PM on 10-13-2009
I just go with Time Machine right now. I back it up at least every two days, or sometimes several times a day if I'm creating a lot of important new files. TM has saved me on a number of occasions when I've overwritten important files, thrown things in the trash by mistake, or hosed my entire drive.
I used to clone my drive, which I liked because my external drive is compatible with my Macbook Pro, and the plan was that if anything ever happened hardware- or software-wise I could physically swap them out and move on like nothing happened. Also, it was bootable. With TM I lose that convenience, but the ability to have many layers of backups is worth much more. Sometimes I don't realize that I screwed up a file until several backups have gone by, and with TM I can always get back to the version I want.
Someday I'm going to buy a larger HD for TM and then maybe I'll use my current one for a clone, but I'm not sure I'm enough of a pessimist to lug two external drives all over the country. Might not be a bad idea to keep the clone in my suitcase though, in case I ever lost my computer bag.
Reply
jeffh said 2:38PM on 10-13-2009
I keep seeing/hearing blogs down on Time Machine / Time Capsule...
I have to at least offer up this. Not once or twice - but three times I have actually had Time Machine / Time Capsule save my butt!
I went with the Time Capsule because there is more than one laptop in my house that doesn't have a 'home' to connect to a fixed drive.
I've been very happy with it since I purchase it.
I let them do their first backup while connected to ethernet - but since then it's been strictly wireless - and it has been running well - knock wood!
Reply
Eric Oltersdorf said 11:04PM on 10-13-2009
Any article about backing up your computer I read never addresses the problems of those of us who regularly create monster files. Two-three gig files will quickly fill up any back up. So what do we do?
Reply
Nate True said 11:04PM on 10-14-2009
Remember, the Time Machine backup volume contains perfectly accessible file backups of every file sorted by time in a very easy-to-use directory structure. You could easily have done exactly the same thing you did with all the ChronoSync stuff from a Time Machine backup.
Reply
alex serg said 8:39PM on 10-20-2009
Some mandatory invitations.
https://www.getdropbox.com/referrals/NTIxNDI1ODc5
Reply