This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Rsync over SSH dies before it is able to send with lots of files


On 9 Aug 2005 at 21:41, Eric Blake wrote:

> For starters, since you are using rsync recursion, get rid of the * to
> give rsync just a single filename at startup, rather than making bash
> try to expand 45000 names all at once on the command prompt:

Bingo!  Of course now that you mention it, it's as clear as day!  I omitted 
the * and the rsync works perfectly.

> Hmm, you're making bash expand a wildcard into 45000 files.  This
> shouldn't crash bash, but I'm guessing there is some of memory
> corruption or malloc failure when consuming so much memory all before
> forking(). I'll have to see if I can try and reproduce the failure.

Do you think the win32 path length will be included for every file?  If this is 
the case you can probably safely assume 30-40 bytes per filename.  If this 
were the case it would quickly add up to about 1.7MB in just passed 
filenames alone.  Considering that this will ultimately process a directory 
with 500,000 files in it it's not hard to work out that things are going to go 
pear shaped in a hurry.

Thanks for the quick spotting!

Cheers,

- Bob -

--
Bob Brown, bob@smsolutions.co.nz
SmartMedia Solutions, Dunedin NZ


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]