This is the mail archive of the
cygwin
mailing list for the Cygwin project.
Re: Rsync over SSH dies before it is able to send with lots of files
- From: "Bob Brown" <bob at smsolutions dot co dot nz>
- To: Eric Blake <ebb9 at byu dot net>, cygwin at cygwin dot com
- Date: Wed, 10 Aug 2005 16:25:14 +1200
- Subject: Re: Rsync over SSH dies before it is able to send with lots of files
- References: <42FA081C.25957.24F4EADF@localhost>
On 9 Aug 2005 at 21:41, Eric Blake wrote:
> For starters, since you are using rsync recursion, get rid of the * to
> give rsync just a single filename at startup, rather than making bash
> try to expand 45000 names all at once on the command prompt:
Bingo! Of course now that you mention it, it's as clear as day! I omitted
the * and the rsync works perfectly.
> Hmm, you're making bash expand a wildcard into 45000 files. This
> shouldn't crash bash, but I'm guessing there is some of memory
> corruption or malloc failure when consuming so much memory all before
> forking(). I'll have to see if I can try and reproduce the failure.
Do you think the win32 path length will be included for every file? If this is
the case you can probably safely assume 30-40 bytes per filename. If this
were the case it would quickly add up to about 1.7MB in just passed
filenames alone. Considering that this will ultimately process a directory
with 500,000 files in it it's not hard to work out that things are going to go
pear shaped in a hurry.
Thanks for the quick spotting!
Cheers,
- Bob -
--
Bob Brown, bob@smsolutions.co.nz
SmartMedia Solutions, Dunedin NZ
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/