View Single Post
Old 12-12-2005, 09:19 PM
xochi xochi is offline
Registered User
Join Date: Dec 2005
Posts: 24
Terrible performance with folders with 5000+ files?

I'm cloning a boot drive to an external firewire drive using "Smart Copy". Here is what happens:

1. SD churns along happily, reaching effective copy speeds of nearly 1 gig/second (processing about 3000 files/second). 99% of my files are not updated, so this speed makes sense. Everything is good.

2. After processing about 150,000 files, suddenly SD hits some mailing list mail in my mail folder (verified by doing 'sudo lsof | grep SDCopy'). These files are not up to date and do need to be copied. At this point, SD drops to about 10 files per second. I can watch the file counter slowly count up. These are small files (e.g. 8k each), but since they are mailing list folders, they may have 5000 or 10,000 messages in each folder.

Something about this seems to choke SD.

I'm guessing that some of your code does not deal gracefully with folders of 5000+ items?
Reply With Quote