LS: Why is Smart Update so large?
 

Just curious, but if I run SD! (2.6.1), surf, e-mail, and then run SD! again, some 2.5GB are updated. In 10.5, less than 1GB would have been updated. Seems odd, at least to me (but, what do I know!).

We've had some reports of this, and we're investigating to see why some files under Snow Leopard files are comparing different when users indicate they haven't really changed anything.

Better copying too much than too little, though. :)

After Snow Leopard update it does take longer to Smart Update
 

I upgraded to Snow Leopard Friday evening. I ran disk permissions before and after, of course. I also did a complete erase & backup with SuperDuper ... which took about the usual time for me ... 60 gb of files ... a little over an hour on USB (newer MacBook but without FireWire).

I just now ran Smart Update (which I do every 24-36 hours BTW); and Smart Update took almost 30 minutes. It normally takes about 5 minutes ... no more than 10 when I've changed a lot of files during the day.

No complaints. I still love SuperDuper ... couldn't live without it ... but it does take longer now.

Let's see if this changes over time. We're investigating what might be going on, but it's not clear at present, certainly not something that would change 5 minutes to 30...

Smart update taking a long time to complete...
 

I'm also hoping that the cause of slow smart updates with a Mac OS X 10.6 / SD 2.6.1 configuration gets attention. My 250+GB boot drive clone has gone from about 10 minutes to almost 20, but worse, my nightly 800+GB Time Machine clone has jumped from under 4 hours to over 9... and this is on an 8 core 2.8GHz MacPro talking exclusively to internal SATA II hard drives.

After discussing this with Dave, there's nothing different in what SD 2.6.1 is doing, so hopefully there will be some "a ha" moment and a 2.6.2 will appear that returns smart update's efficiency.

-- Marc

I literally ran SmartUpdate with SD! 2.6.1, rebooted, and ran SmartUpdate again. Amazingly, over 2 GB of my 35 GB drive were updated. Hmmm

I think this MIGHT be a side effect of prebinding. We're looking into it.

Smart Updates taking forever!
 

I'm another who is having the problem with very L-O-N-G backups.
What used to take 3 - 4 minutes is now taking 20-22.
I thought that if I started all over again and erased the drive and did a complete new backup, it might help.
It didn't.
I sure hope that a fix will be found!

I believe this is a side effect of prebinding, as I said above. Investigation has shown that most of the time is spent in /Applications /System and /Library, which would help to confirm that. We're looking into it.

SDCopy CPU load under 10.5.8 and 10.6?
 

Dave, after playing with the unbelievably slow backups on my MacPro, I don't get why the prebinding is an issue for overall speed, as that is in the final phase and not performed on non-boot drives.

Maybe this will be helpful:

Today my now 9+ hour Time Machine drive clone was still percolating at 10 this morning, so I popped open Activity Monitor and saw that SDCopy was eating 87-100% of a single core with 2 threads. I'm running SD 2.6.1 right now to clone my MacBook Pro drive under 10.5.8 before upgrading it to 10.6 later today, and SDCopy is only eating at max 15% of one core with an average of around 1-3%. That puppy really beats itself up under 10.6.

-- Marc

That's not at all normal. For one thing, we're not threaded. For another, SDCopy is really not significantly different under SL. I think something else is going on there (assuming that OSX itself isn't eating the core when we call it).

That doesn't have much to do with the other issue here, which is that Smart Update is copying things over, which I believe does have to do with prebinding.

With reference to the prebinding possibility... I noted that when I was making my final SD backups before updating to 10.6 (today), that an inordinate amount of time was spent in the final step of adjusting prebinding on the backup. Normally this step would have required virtually no time. This was true both for full backup and sandbox.

Don't know if this provides any hint or not...

Opposite experience for me ... time lag at the beginning of process
 

The huge amount of extra time seems to come at the beginning. Once SD gets beyond a certain point, it zips along like it used to. I do agree that the pre-binding at the end seems to run a little longer but for me that's not the lion's share of it.

Just to update, what I used to do in about 5 minutes is now taking 20-30 minutes. I know you are working on it; but I thought I'd let you know that time is not clearing up the issue ... although I suppose there really hasn't been a huge amount of time since the weekend. :-)

Well... adjusting prebinding is new to 2.6, so I don't think it would have taken no time... but it can take longer under SL. We're trying to find a different way.

And I think people are onto something with the whole 'compressed files are expanding' thing, even though it shouldn't be. We are looking into it. No sense chiming in with a "me too" if you're thinking about it, unless you have more to add: we're trying like hell to get to the bottom of what's up.

I had failed to notice this added step. So, yes, all I had noted with earlier SD backups is that the last step was accomplished very quickly.

I trust we'll all wait patiently for this to be resolved.

Yes, it's important to note (again) that if it's taking a little extra time in some situations in this version, and if the backup takes up more space at a low level, the copy is still reflective of the source and is valid... continued improvements will be made as we determine the cause.

Usually, this kind of thing is investigated in testing, and although we've been in test for a long time, the 'seems a bit slower/added copying' issue came up relatively late, in the "gold bits" timeframe, and then Snow Leopard shipped about 4 weeks earlier than we expected. It was unexpected, and we didn't consider it wise to make risky changes late-cycle, since the copy itself was OK, and there just wasn't enough time to get into the low-level details.

We could have delayed weeks to try to figure it out... but considering how people reacted last time we needed to delay, and since it was a performance issue and not a copy problem, we decided to go with what we had.

Now that we have a bit more time to investigate more completely, we're working on it, and if the fix is riskier, we have the luxury -- necessity, really -- of being able to test it without the same deadline breathing down our neck.

Smart update size difference
 

Adding something :)

Two identical internal MacPro Seagate 1TB SATA II hard drives, with identical GUID single volume partitioning, used as the Time Machine repository and a clone of same. Just completed (yet another nightly 800+ GB smart update):

Source: capacity 999.86 GB, available 184.17 GB, used 815.69 GB
Target: capacity 999.86 GB, available 178.64 GB, used 821.22 GB

Time Machine is turned off during the clone. Not bootable, so no post prebinding operation performed.

-- Marc

Perhaps a difference in the spotlight indexes in the two drives, Marc (one's inside the TM archive, too)? Could also be that the compression was preserved in the source but not when copied.

It may not be just SD that is bloating its output with OS X 10.6. Over the past two days I've seen similar inexplicable bloating of Time Machine files.

http://discussions.apple.com/thread....42173&tstart=0

I have always had Spotlight indexing disabled on the clones. I just checked the exclusion list again, and the TM copy is still listed as excluded. Then I did a du on both the source and target Spotlight dirs:

So this does explain part of the 5.42 GB difference, but it's not the whole story.

So why would a copy result in expansion of compressed data?

-- Marc

Examining individual TM session differences
 

I've been using BackupLoupe to show the detailed differences in TM snapshots:

http://soma-zone.com/BackupLoupe/

-- Marc

I'm sorry -- I can't take the time to go into detail on this here, since we're still investigating. But I'm going to write a blog post about it when I get a few minutes of free time to discuss the issue and what we did (or are in process of doing) to fix it.

As I recall (and mentioned above), there's a spotlight index inside the TM backups.backupdb too.

Only one .Spotlight set on TM Volumes
 

There's only one Spotlight location on Time Machine volumes - and none were found within the Backups.backupdb directory...

A more general search yields only spotlight-named things that should be in the backup...

Perhaps you're confusing this with a .Spotlight-V100 directory within each TM sparse bundle for networked backups. But there is ONLY one.

By the way - lest you think that I'm being critical or pedantic about this - I'm just VERY HAPPY that SuperDuper handles cloning TM disks at all... NOTHING else out there can even touch them except a block based cloner... and for that support I am EXTREMELY thankful...

-- Marc

Quite possibly... I don't remember the details exactly, since I did this research almost two years ago... :)

The explanation (at least, partial) for the bloating of TM is that I found that WD Drive Management is incompatible with OS X 10.6. It was adding a spurious message to Console every 10 seconds whether drives were connected or not. It appears that each entry is unique and is seen by TM as a "change"! This caused TM to bloat by about 10GB daily.

Needless to say I've disabled the WD program without any "visible" effect other than TM is no longer bloating hourly.

did 2.6.2 (87) fix this problem?
 

so - did 2.6.2 resolve the "binding" issue?



side note:
i tried to upgrade from 2.6 to 2.6.2 under auto update this evening, and after 5 minutes of watching the download spin, i aborted. i am moving to o.s. 10.6 shortly so i guess i need 2.6.2

but the long download time was troubling. was this just a one time quirk?

note: i tried download on tue eve at 10 pm est.

i have updated superduper many times in past - past as lightning.

2.6.2 resolves the re-copying issue completely, yes.

Downloads might have been a bit slow last night because a lot of people hit the site at around 9pm.