Clone larger than source, after erase/copy.

[mikebore]

MacBook Pro, 10.6, SD 2.6.1

Source Volume is a Utilities partition in the MBP, sized 21.47 Gb of which 6.23 Gb is used.

I used Superduper! to make a clone to a FW external partition sized 21.89 Gb, using the erase and copy option. On completion the amount used on the copy is 9.73 Gb.

First thought was connected to sleepimages, but thats the wrong way round, and there is no sleepimage on the copy.

Any thoughts?

[dnanian]

Please send in your logs using the "Send to shirt pocket" button, Mike. Could be Spotlight or many other things.

[mikebore]

Hello Dave,

I have run Superduper! again since the one I posted about and the log only shows the latest. Will it be somewhere else?

I can do it again to generate a log if necessary.

[dnanian]

No, just use the send to shirt pocket button and the last 10 logs will be sent.

[mikebore]

Ah! I have already sent the log to you under case number 101098...which was a different problem, which happened just after the copy which started this thread.

[dnanian]

Didn't know that was you.

It's not entirely clear what the additional data is. The right amount of data is copied (about 5.3GB). The volume ends up larger because of Spotlight and prebinding, I think.

[mikebore]

I just repeated this erase/copy run and immediately on completion used Whatsize to see where the main space is being used.

The top seven items on the source are:

1.98 Gb System
1.37 Gb Library
718 Mb Applications
547 Mb private
500 Mb usr
396 Mb Users
17.8 Mb mach_kernel

and on the copy are:

3.69 Gb System
2.16 Gb Library
1.11 Gb usr
1.06 Gb Applications
434 Mb private
396 Mb Users
75.4 Mb .Spotlight-V100

Total used on the source is 5.82 Gb, and on the copy is 9.13 Gb

All these are in Whatsize Gb not SL Gb.

Does this make any sense to you?

I have sent the log from this run from the application.

EDIT I meant to add that the spotlight file on the source is zero. It is not excluded from Spotlight by the privacy option.

[mikebore]

I have set two copies of Whatsize running so I can drill down on the source and copy and compare.

There is no single answer to where the size has gone. Thousands of files are different. Just as a random example the Russian Quicktime help index is 56kb on the source and 168 on the copy!

[dnanian]

That's really weird. Again, looking into it. Maybe the API we're using to copy files is incorrectly expanding the compressed data (which wouldn't make any sense, but I guess it's possible). It's a single call into the OS... weird.

Anyway, again, still investigating. Even if the data might be larger, it's still "correct"...

[mikebore]

For the benefit of others, your later reply to me on this in the tech support case was "Yes. I think it's because the backup has been fully prebound, whereas the source hasn't".

Thanks. I guess I need to read up on prebinding, I didn't realise it had that kind of effect!

[dnanian]

Right: this is a theory we're investigating. We don't know for sure yet, because we're still looking into it... which is why the data in this thread is helpful.

[mikebore]

FWIW a Chronosync bootable copy results in the identical increased size as Superduper!

Unless Chronosync is also prebinding (which I haven't heard) the cause is perhaps something in the OS.

I did a CCC copy too, but it was block level, so unsurprisingly it was identical to source.

[mikebore]

Two extra bits of info:

1.A copy of my main boot volume also has the 3-4 Gb "discrepancy".....less noticeable on a 150Gb source than a 5 Gb source.

2.A Superduper! (2.6.1) copy of a Tiger volume does NOT have the discrepancy.

My laymans conclusion is that it is something to do with the OS not prebinding.

[dnanian]

We've done quite a bit of initial investigation into this, and the file size increase is, indeed, due to compressed files expanding during the copy.

This is not necessarily the same thing as the 'recopying', although the two issues are combined in this thread.

Spot checks of Finder, cp, tar and others shows that they're also expanding the files on copy. Since we use the base-level copy calls that Apple provides, and supplement that with additional metadata copies/checks, and the "external" size looks the same, we both assumed the base call was handling this properly and didn't catch it when looking at the file sizes themselves (since they're reported at their uncompressed size).

We're investigating how to avoid this expansion, and continuing to determine why things are being copied that you'd think wouldn't; more as I've got it (although I might move to blogging about this so it's more useful to others and other developers running into the same problem).

[mikebore]

Thanks for the update.

Main questions are:

1. Is it happening to everyone using Superduper, CCC and Chronosync? or only some people. I would have expected it to be more widely reported.

2. How much of a concern is it? Everything seems to work. Only problem is loss a few Gb of space. Everything seems to work.

BTW ... can't see any mention of "recopying" in this thread...what is that about?