git on cygwin 1.7.2

[Mark Levedahl <mlevedahl at gmail dot com>]

I've spent some time trying to diagnose the previously reported 
git-fetch failures running on cygwin 1.7-x (up to and including 1.7.2). 
The failure manifests itself as an incomplete transfer of data using 
git's native transfer protocol, and happens either talking directly to a 
git-daemon via sockets or using a pipe tunneled through an ssh connection.

git started life as a large number of shell scripts wrapping a few 
compiled programs, and retains the architecture of multiple processes 
communicating via pipes even though most of the shell scripts were 
converted to C. For the given problem, the situation is as follows:

The main git process either opens ssh as a subprocess to execute 
'git-upload-pack' on the server, or opens a socket to talk to a 
git-daemon on the server (running under inetd). Either way, the parent 
git process now has two file handles for communicating to the server, 
and redirects these to subprocesses. The data sent from server back to 
the client is multiplexed as packets of up to 64 KBytes, each packet 
having a header giving a channel number and the number of bytes to 
follow. Channel 1 is for the data, channel 2 is typically stderr and is 
used as such in this case.

Next, git starts a sideband demux process, connecting its stdin to 
stdout from the server process above. This demux process prints the 
information channel (channel 2) to stderr, and the data channel is 
written back to the demux process's stdout.

Next, git starts an index-pack process with stdin connected to the demux 
process's stdout.

All of the above connections and i/o processing are done using 
bog-standard fork, pipe, open, read, write, and associated file 
commands. Git has no trouble opening all of the processes and pipes 
under Cygwin 1.7.2. However, in general before the data transfer is 
complete the demux process issues an error upon getting an incomplete 
packet (fewer bytes transmitted than declared in the header), and 
debugging shows the error code of the read() is ENOENT, indicating the 
pipe or socket has been terminated early. The index-pack process also 
issues an error due to receipt of an incomplete pack.

I am able to get this error very reliably trying to clone a repository 
about 50 MBytes in size across my office network (100-base T). The error 
shows up randomly but in the range 2% to 10% of the total repository 
having transferred. The same repository clones just fine when connecting 
to the same server using VPN over a relatively slow internet connection. 
Also, the same git executable on the same computer works flawlessly a 
Cygwin 1.5 installation.

I'm guessing the error is in the Cygwin dll, and a further guess is that 
it is triggered by hitting buffer limits on pipes between the several 
processes running inside of git as the error does not occur using when 
the transfer rate is slow.

I'm not sure where to continue investigating, but it is quite obvious 
the problem is not in git itself (works fine on cygwin 1.5 not to 
mention Linux) or in ssh (the failure occurs using the direct protocol 
without ssh). I'm not familiar with the internals of the dll, but I'm 
willing to help dig if someone can give me some hints on where to look.

Mark Levedahl

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple