This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

RE: Cygwin Rsync/SSH performance issue


Hi,

Today I did another interesting test in an attempt to resolve this issue.
With Microsoft Process Explorer (downloadable from the Microsoft website) it
is possible to 'pause' and 'resume' processes.

When I start a rsync/ssh session to my server and pause the SSH process. The
rsync process consumes 100% CPU, while the IO is 0. In the case I pause
rsync and leave SSH running, there is no CPU load and no IO.

In the first case rsync cannot send any data into the SSH tunnel. I think
rsync goes into a CPU consuming loop while it waits for access to the SSH
tunnel. This is somehow the same as when rsync tries to send data with SSH
and it has to wait because it is exceeding the bwlimit variable.

The next step for me would be to dive into the rsync code, which is
something I really try to avoid at this moment.

Any volunteers ?

Regards,

Jan


-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar@dotcolour.com] 
Verzonden: woensdag 17 februari 2010 23:18
Aan: 'cygwin@cygwin.com'
Onderwerp: RE: Cygwin Rsync/SSH performance issue

Hi again,

Still working on this one. When I rsync/ssh a big file to my server over the
local network (100Mbit/s) with the --bwlimit=1000 I only have a small CPU
load. SSH takes about 10% and rsync approximately 0-5%. In the end the
transfer speed usually is around the value I set in bwlimit.

When I transfer the same file to a remote machine, where the bandwidth is
limited to roughly 140KB/s and the bandwidth limit is set to something below
this value I get around the same values which I have on the local network,
SSH 10% and rsync somewhere between 0-5%.

If I change the bwlimit value to 150KB/s or higher the SSH load goes up to
70% and rsync to 30% fully consuming my CPU.

I think the problem is pretty clear now. Does anybody have an idea, what the
next step could be to the resolution of this ?

Regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar@dotcolour.com] 
Verzonden: woensdag 17 februari 2010 12:27
Aan: 'cygwin@cygwin.com'
Onderwerp: FW: Cygwin Rsync/SSH performance issue

Hi

At first this email was sent to the developers list, but I understood this
list is more appropriate.

Regards,

Jan


-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar@dotcolour.com] 
Verzonden: woensdag 17 februari 2010 12:25
Aan: 'cygwin-developers@cygwin.com'
Onderwerp: RE: Cygwin Rsync/SSH performance issue

Hi,

In an attempt to narrow this down a bit I installed a RedHat machine with
openssh-5.3p1 and rsync-3.0.7 (thus without cygwin) and did the same test as
below.

Transferring data, both up and downstream, are not giving me any CPU load.

Although there is a small version difference in the rsync versions, I would
conclude that this is somehow caused by the cygwin layer.

If anybody can have a look at it, I would really appreciate it. If there is
something I can do, please let me know.

Warm regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar@dotcolour.com] 
Verzonden: dinsdag 16 februari 2010 16:46
Aan: 'cygwin-developers@cygwin.com'
Onderwerp: RE: Cygwin Rsync/SSH performance issue

Hi again,

I did some more testing on this and figured out that rsync/ssh is only
taking the full cpu if the bandwidth allocated to the session exceeds the
true bandwidth available on the link.

For example, if the uplink is 128KB/s and I pass in the --bwlimit=24
parameter the cpu is around 0%. Even --bw-limit=126 does not take 100% cpu,
but --bwlimit=132 does.

Would this be because of packet loss on the link and packets have to be
retransmitted ? I will do a quick test tonight with rsync/ssh between two
linux boxes, see what that gives me.

Regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar@dotcolour.com] 
Verzonden: dinsdag 16 februari 2010 11:34
Aan: 'cygwin-developers@cygwin.com'
Onderwerp: Cygwin Rsync/SSH performance issue

Hi everybody,
 
I am using rsync/ssh to copy files from my computer to my server and
everything seems to function as expected, except performance.

During a file transfer on my home computer (Intel I7) there is not a
noticeable CPU load. But on other (older) machines the rsync/ssh transfer
consumes all CPU resources on the sending side. I noticed there are other
questions regarding this subject in the archives, but I could not find a
solution for this. Also throttling the bandwidth to a lower value is not
helping to reduce the CPU load.

I am using quite a basic command to sync my files:

        rsync -e ssh * <user>@<server>:/data

The versions of the software I am using are:

Cygwin: 1.7.1-1
ssh: OpenSSH_5.3p1
rsync: 3.0.6
OS: WinXP

Was anybody able to resolve this ?

Any help is greatly appreciated.

Warm regards,

Jan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]