This is the mail archive of the
cygwin-developers
mailing list for the Cygwin project.
Re: 1.7.8: write fails with EAGAIN
- From: Christopher Faylor <cgf-use-the-mailinglist-please at cygwin dot com>
- To: cygwin-developers at cygwin dot com
- Date: Mon, 7 Mar 2011 12:22:53 -0500
- Subject: Re: 1.7.8: write fails with EAGAIN
- References: <4D72992D.4090007@tweerlei.de> <20110307103951.GL6393@calimero.vinschen.de> <20110307153708.GA2538@ednor.casa.cgf.cx> <20110307163849.GA26824@ednor.casa.cgf.cx> <20110307171045.GA26028@ednor.casa.cgf.cx>
- Reply-to: cygwin-developers at cygwin dot com
On Mon, Mar 07, 2011 at 12:10:45PM -0500, Christopher Faylor wrote:
>On Mon, Mar 07, 2011 at 11:38:49AM -0500, Christopher Faylor wrote:
>>On Mon, Mar 07, 2011 at 10:37:08AM -0500, Christopher Faylor wrote:
>>>On Mon, Mar 07, 2011 at 11:39:51AM +0100, Corinna Vinschen wrote:
>>>>On Mar 5 21:12, Robert Wruck wrote:
>>>>> Hi,
>>>>>
>>>>> recently, I found that cygwin-git was not able to 'cat-file' files
>>>>> that exceeded some size (in my case about 80MB).
>>>>> I tracked this down to the cygwin implementation of write() that
>>>>> behaves quite odd in some cases.
>>>>>
>>>>> I wrote a small program (source attached) that mmaps a given file
>>>>> and tries to write it to another file or stdout.
>>>>>
>>>>> The results vary:
>>>>>
>>>>> If the destination is a file (`writetest infile outfile` or
>>>>> `writetest infile > outfile`), the write succeeds in a single call.
>>>>>
>>>>> If the destination is a pipe (`writetest infile | cat > outfile`),
>>>>> the write succeeds in most cases. BUT:
>>>>>
>>>>> Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and
>>>>> errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my
>>>>> case 4096 byte for each call).
>>>>> This breaks git since it does an infinite loop while errno=EAGAIN.
>>>>
>>>>Hang on, you are saying that a *blocking* write(2) to a pipe returns
>>>>with EAGAIN? Are you sure? It would be quite a surprise if git would
>>>>actually do that. EAGAIN is only an expected error for non-blocking
>>>>I/O, so applications which use blocking I/O usually only test for EINTR.
>>>
>>>I can barely convince myself that there's a pathological case where an
>>>EAGAIN could leak out. I'm investigating now.
>>
>>Actually, in this case, it looks like the problem is that Windows
>>doesn't like sending a huge buffer to a pipe. The errno in this case
>>should probably be something like EFBIG rather than EAGAIN.
>>
>>Does git deal with this type of errno gracefully or does it just abort
>>if the write() fails for any reason? I'd rather just fail and let the
>>caller deal with it than complicate Cygwin's code by trying to loop
>>writing smaller amount of data to the pipe so I'd prefer just changing
>>the errno if that solves the problem.
>
>Answering my own question: No, git doesn't make any useful decisions based
>on the errno. So, just returning a different errno is not going to make
>this work.
>
>So, my options are to:
>
>1) Limit the buffer size to some value like 64 mib and expect the caller to
>deal with that.
>
>2) Limit the buffer size to some value like 64 mib and loop in
>fhandler_overlapped::write_overlapped until everything has been written.
>
>3) Attempt the WriteFile(), notice the error condition and then do 1) or 2).
>
>2) or 3) would be the most like Linux. 1) is by far the easiest and would
>probably solve the current git error case.
>
>Just thinking out loud here. I think I've convinced myself that I should
>go with 3 + 2.
Except that it's tricky to get this right in the non-blocking case. Sigh.
cgf