Re: Clearing O_NONBLOCK from a pipe may lose data

[Lasse Collin <lasse dot collin at tukaani dot org>]

On 2015-02-19 Corinna Vinschen wrote:
> I can imagine.  A few years back, when changing the pipe code to
> using overlapped IO, we stumbled over a problem in Windows.  When
> closing an overlapped pipe while I/O is still ongoing, Windows
> simply destroys the pipe buffers without flushing the data to the
> reader.  This is not much of a problem for blocking IO, but it
> obviously is for non-blocking.
> 
> The workaround for this behaviour is this:  If the pipe is closed, and
> this is the writing side of a nonblocking pipe, a background thread
> gets started which keeps the overlapped structure open and continues
> to wait for IO completion (i.e. the data has been sent to the reader).
> 
> However, if you switch back to blocking before closing the pipe, the
> aforementioned mechanism does not kick in.
> 
> I can look into improving that at one point, but not soon.

OK. I think xz needs a workaround then. xz 5.2.0 uses non-blocking I/O
primarily to avoid a race condition in signal handling. This includes
writing to stdout. xz restores the original file status flags at the
end of the file because leaving stdout to non-blocking mode can break
other programs that use the same file description after xz has exited.

For example, if xz is modified to leave O_NONBLOCK enabled on stdout,

    ( xz -dc smallfile.xz ; cat bigfile ) | ( sleep 1 ; wc -c )

will make cat print

    cat: write error: Resource temporarily unavailable

on GNU/Linux and most of bigfile won't be seen by wc. However, on
Cygwin the above works because the modifications to the file status
flags in xz aren't seen by cat. That is, stdout in cat is in blocking
mode even though xz left it in non-blocking mode.

The above Cygwin behavior would make it very easy to add a workaround
to xz for the pipe-data-loss problem: simply don't clear O_NONBLOCK on
stdout. However, I wonder if it is a bug in Cygwin that the changes to
file status flags aren't seen via other file descriptors that refer to
the same file description. If it is a bug, then the workaround idea will
cause trouble in the future when the bug in Cygwin gets fixed.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple