This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Clearing O_NONBLOCK from a pipe may lose data


(Please Cc me when replying, I'm not subscribed to the list.)

Hi!

I suspect that there is a bug in Cygwin:

1. Create a pipe with both ends in blocking mode (O_NONBLOCK
   is not set).
2. The writer sets its end to non-blocking mode.
3. The writer writes to the pipe.
4. The writer restores its end of the pipe to blocking mode
   before the reader has read anything from the pipe.
5. The writer closes its end of the pipe.
6. The reader reads from the pipe in blocking mode. The last
   bytes written by the writer never appear at the reader,
   thus data is silently lost.

Omitting the step 4 above makes the problem go away.

A small test case is attached. I have tested it under Cygwin64 that was
installed a few days ago on Windows 7. Usage:

    gcc -std=gnu99 -Wall -Wextra writer.c -o writer
    gcc -std=gnu99 -Wall -Wextra reader.c -o reader
    ./writer | ./reader

The reader only gets the first 65536 bytes. The remaining 5555 bytes
are never seen by the reader even though all write() calls succeed at
the writer.

I found this problem because I had heard a rumor that there is something
wrong with xz 5.2.0 on Cygwin. Since the file I/O code had been changed
in 5.2.0, I investigated that first. There also seems to be something
wrong with xz 5.2.0 in threaded mode under Cygwin which needs more
investigation (random pthread_join() failures cause assertion failures
if xz was built with --enable-debug).

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode
// gcc -std=gnu99 -Wall -Wextra reader.c -o reader

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>

static void
fail(const char *str)
{
	fprintf(stderr, "R: %s\n", str);
	exit(EXIT_FAILURE);
}

int
main(void)
{
	// It is assumed that stdin is a pipe open for reading.
	//
	// Read 8 KiB at a time.
	static unsigned char buf[8 << 10];

	// Total amount read
	unsigned total = 0;

	for (unsigned i = 0; i < 10; ++i) {
		fprintf(stderr, "R: Sleeping\n");
		sleep(1);

		int amount = read(STDIN_FILENO, buf, sizeof(buf));
		fprintf(stderr, "R: read() returned %d\n", amount);

		if (amount >= 0) {
			total += amount;
			fprintf(stderr, "R: Total amount read is %u\n",
					total);
		}
	}

	if (close(STDIN_FILENO))
		fail("Error closing stdin");

	return EXIT_SUCCESS;
}
// gcc -std=gnu99 -Wall -Wextra writer.c -o writer

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>

static void
fail(const char *str)
{
	fprintf(stderr, "W: %s\n", str);
	exit(EXIT_FAILURE);
}

int
main(void)
{
	// It is assumed that stdout is a pipe open for writing.
	//
	// Set stdout to non-blocking mode.
	int flags = fcntl(STDOUT_FILENO, F_GETFL);
	if (flags == -1)
		fail("Cannot get stdout file status flags");

	if (fcntl(STDOUT_FILENO, F_SETFL, flags | O_NONBLOCK) == -1)
		fail("Setting stdout to non-blocking mode failed");

	// Writing 64 KiB works as expected.
	static unsigned char buf[64 << 10];
	if (write(STDOUT_FILENO, buf, sizeof(buf)) != sizeof(buf))
		fail("The first write() failed");

	fprintf(stderr, "W: Wrote %zu bytes\n", sizeof(buf));

	// Writing more returns successfully but the reader won't see
	// these bytes. The reason appears to be the fcntl() below.
	const unsigned small_size = 5555;
	if (write(STDOUT_FILENO, buf, small_size) != (ssize_t)small_size)
		fail("The second write() failed");

	fprintf(stderr, "W: Wrote %u bytes\n", small_size);

	// Restore the original file status flags to stdout.
	// For some reason this makes the bytes from the last
	// write() disappear. If this is commented out, all
	// data is visible at the reader.
	if (fcntl(STDOUT_FILENO, F_SETFL, flags) == -1)
		fail("Restoring stdout file status flags failed");

	if (close(STDOUT_FILENO))
		fail("Error closing stdout");

	return EXIT_SUCCESS;
}

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]