question about "bit rot"

[Dan Lester]

I've been thinking about hard disks as long-term storage, and I've realized that magnetic domains on hard disk actually aren't permanent. The field strength decreases by more than 1%/year. So even for a hard disk that is never used, your data on it will gradually fade away. Errors will increase on time scales of years. This is well understood.

As far as I can tell, the only way to prevent this is to completely rewrite your hard disk data once in a while. Every year or two? Very little cogent info about mitigation strategy that I can find.

That makes me wonder about SuperDuper smart updates, which just write things that have changed. Things that haven't changed don't get rewritten. So after a while, much of your archive disk data can get stale. Is this a concern? As in, is it smart to do a full up backup every once in a while instead of smart update? Some expert advice would be handy.

[dnanian]

I take a major OS update as an opportunity to do a full backup, Dan. On top of that, I use a number of backup volumes, some of which are RAID and get scrubbed, to ensure that things are generally safe. Plus, I use online backup... your best approach is thorough, diverse coverage.

[Dan Lester]

Originally Posted by dnanian

I take a major OS update as an opportunity to do a full backup, Dan. On top of that, I use a number of backup volumes, some of which are RAID and get scrubbed, to ensure that things are generally safe. Plus, I use online backup... your best approach is thorough, diverse coverage.

I certainly agree that diverse coverage is good. But if you have a few archive disks that you only do incremental SuperDuper backups on, there is a good chance that after a decade none of them will work.

Are you aware of any Mac utilities that do disk refreshing? I think there used to be one, called DiskRrefresher, but that's no longer available. There are PC utilities that do this. There was, many years ago, a post about doing disk refreshes in a Unix-based system ...

https://larryjordan.com/articles/tec...-disk-storage/

but I frankly think those techniques don't really do what he says they do.

It would be nice to see some well-thought-out strategy for long term archiving on hard disks. I realize that's not necessarily the purpose of SuperDuper, but your insights would be welcome.

[dnanian]

Well, again, the scrubbing does refresh the data (as, obviously, does the yearly rewriting). So, if you consider your NAS-based backups "archival" and have them scrubbed, they seem reasonably safe to me.

As far as there being a good chance that after a decade none will work - I've been surprised that most drives *do* work just fine after a decade, retaining all their data. Which, of course, is no guarantee.

[Dan Lester]

Originally Posted by dnanian

Well, again, the scrubbing does refresh the data (as, obviously, does the yearly rewriting). So, if you consider your NAS-based backups "archival" and have them scrubbed, they seem reasonably safe to me.

As far as there being a good chance that after a decade none will work - I've been surprised that most drives *do* work just fine after a decade, retaining all their data. Which, of course, is no guarantee.

Thank you. But I've always looked at "scrubbing" as simple hard erasure of a disk for security. What kind of scrubbing does data refresh? Is there an App that does that?

And yes, while MTBF of hard disks is supposed to be about five years, they've always lasted much longer than that for me, at least for mechanical performance.

[dnanian]

No, that's not what scrubbing does: your NAS usually will offer data integrity scrubbing.

And I'm talking about more than mechanical performance. There's a lot of "fuzziness" in magnetic drop-off, hence the ability to recover data from master tapes that are 50 years old.