Re: checking in >= 256k file fatally corrupts rcs file

[Ryan Johnson <ryan dot johnson at cs dot utoronto dot ca>]

On 08/10/2013 7:48 PM, Warren Young wrote:
> On 10/8/2013 04:22, Don Hatch wrote:
> > Checking in a text file of size >= 256k
> > corrupts the rcs file, irretrievably losing most of the contents
>
> It's documented in the rcs NEWS file:
>
>     - Env var RCS_MEM_LIMIT controls stdio threshold.
>
>       For speed, RCS uses memory-based routines for files up to
>       256 kilobytes, and stream-based (stdio) routines otherwise.
>       You can change this threshold value by setting the environment
>       variable ‘RCS_MEM_LIMIT’ to a non-negative integer, measured in
>       kilobytes.  An empty ‘RCS_MEM_LIMIT’ value is silently ignored.
>
> So, use the new environment variable, or build up your huge diffs a 
> few steps at a time, so as to avoid spamming this buffer.
So in other words, a misguided performance optimization [1] that almost 
certainly has little measurable impact on performance [2] has introduced 
a silent data corruption bug (or tickled a latent one somewhere else). 
Lovely.

The gcc devs have the right philosophy: features that break things badly 
get reverted immediately regardless of whose fault the bug is, and will 
be considered for re-inclusion once the bug has been fixed on the side. 
In this case, though, the I'm not sure re-inclusion is even warranted.

[1] Modern filesystems and filesystem caching are pretty darn good at 
handling temporary files these days. Further, if you really care about 
using RAM to improve performance, 256kB is an absurdly low limit for a 
buffer size, and has been for most of the last decade.

[2] I'd be shocked if even 0.1% of checkins were large enough to have a 
noticeable latency in a modern system, and even more shocked if the 0.1% 
that are large enough to be slow were still small enough that 256kB of 
buffering made any difference in their runtime. Unless the code is 
calling fsync() after every newline or something, in which case that's 
what needs to be fixed.

$0.02
Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple