This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: cvs over ssh with tcsh (testcase and workaround)


Sent: Wednesday, April 05, 2006 3:18 AM
Subject: Re: cvs over ssh with tcsh

> On Tue, 4 Apr 2006, Jay Abel wrote:
>
> > This is just a ping to see if anyone has noticed problems running
> > cvs in command line server mode over ssh with tcsh installed as the
> > login shell (in /etc/passwd).  After a lot of testing (thanks Rene
> > Berber) is seems that the problem only occurs when tcsh is my login
> > shell.  If I change my login shell to /bin/bash, the problem goes
> > away.
> >
> > versions are as follows:
> > [snip]
> > I googled 'ssh tcsh' and found that some programs (sftp) don't like
> > noisy rc scripts, but
>
> cvs also doesn't like noisy scripts.
>
> > tcsh -C /bin/true
> >
> > produces no output.
>
> I take it you meant "tcsh -c /bin/true" (lowercase "c").  Your test
> above only tests tcsh in non-login mode.  Try 'tcsh -l' or '(exec -l
> tcsh)' from bash instead.
>
> > The symptoms of failure that I obsererve fall into two categories:
> >
> > 1. cvs client reports 'unrecognized command' along with a piece of
> > one of the files uploaded displayed as the offending command, or
> >
> > 2. cvs trace stops dead, with both client and server processes in an
> > O state by ps.
>
> Both of these seem to indicate extra output from somewhere.
>
> > [snip]
> > At this point I'm mostly interested in whether anyone else has ever
> > seen this behavior.  If not, I'll continue to try to acertain what
> > it is about my tcsh configuration which is causing the problem.
>
> I'd say you have a noisy .login (or /etc/csh.login)...  If you rule
> that out, we can look for other causes.
>
> FWIW, simply "ssh user@host cvs server" should show you all of the
> output your cvs client sees (and complains about)...

Test Number 1: Is tcsh login noisy?

[jabel@jabelxp][home/jabel] % tcsh -l < /dev/null | od -ab
tcsh -l < /dev/null | od -ab
0000000
[jabel@jabelxp][home/jabel] %

Ok, fair enough. FWIW, I was actually wrong -- "ssh u@h command" doesn't use a login shell. In fact, even though cvs doesn't *like* the noise, it doesn't complain about it in the same way (i.e., putting an "echo Hi" into .tcshrc, I get back "cvs update: warning: unrecognized response `Hi' from cvs server"). Also, I get this as essentially the first thing after "Starting server:", whereas you get this in the middle of updating...

Test Number 2: does problem exist with CVS?

[jabel@jabelxp][jabel/cvstemp] % cvs -t up
cvs -t up
-> main loop with CVSROOT=:ext:jayabel.com:/home/spring2006
-> Starting server: /bin/ssh jayabel.com cvs server
-> Sending file `report.pdf' to server
-> Sending file `report.tex' to server
unrecognized request `OF'
-> Lock_Cleanup()
-> Lock_Cleanup()
[jabel@jabelxp][jabel/cvstemp] %

Am I missing something here?  If cvs uses a plain old login shell, isn't
it subject to the vagaries of things like CTRL-z suspending the job,
CTRL-s suspending output, and certain other control and escape sequences
sending back all kinds of chatter?

It doesn't -- that was a red herring. However, you didn't try the last test I proposed, i.e., "ssh jayabel.com cvs server".

Is there something obvious I'm supposed to have in my startup scripts
that turns all that stuff off if the connection is supposed to be
binary, and how does CVS tell ssh it wants a binary shell?

I'm currently using /bin/ssh as my CVS_RSH variable, but at least
according to the trace output cvs isn't sending any special options.
Do I need a wrapper script to create a silent connection?

Nah, I think we're barking up the wrong tree here.


One other observation is that, when I put something in the .rc script, my
cvs complained about "unrecognized response", whereas yours complains
about "unrecognized request".  This makes me think that it's not the
server sending something bad, it's the client.  You could write a cvs
wrapper script that does something like

tee CVS_OUT | /bin/cvs "$@"

stick it in the PATH before /bin, and examine ~/CVS_OUT for commands sent
from the client.
HTH,
Igor

THE PROBLEM


Okay, I have made some progress. Highlights are, there is a way to reproduce the error, and I've attached a possible workaround.

When bash is selected as the login shell, the cvs server process is passed stdin open in binary mode. When tcsh is the login shell, the cvs server process is passed stdin open in some kind of text mode where CRLF gets mapped to LF. For protocol commands, the cvs server doesn't care. When receiving files, as when performing import, add, update and commit operations, the server does indeed care. When the client wants to send a file, it first sends an 'Entry' command followed by the number of bytes to be sent, then it sends those bytes. The server receives exactly that many bytes then starts parsing the next command.

If CRLF's are present during the file upload, the number of bytes actually received is reduced by one for each CRLF in the stream. If file compression is enabled, on the command line or in or in the client .cvsrc file, then the occurrence of CRLF's is quasi-random, but repeatable. If compression is turned off, the contents of the file can be manipulated to cause the error.

Steps to reproduce the problem:

A few necessary preliminaries:

0A. ssh must be installed and properly configured.
0B. tcsh must be installed
0C. all directories used must be binmode, particularly (as used in this example):


/var/repo/
~/testcase/
/tmp/

THE TESTCASE

1. Turn off file compression (if present in .cvsrc look for a line starting with cvs, if a -z switch is present change it to -z0.

2. Change user's shell to tcsh by modifying /etc/passwd:

from:
user::......:/bin/bash
to:
user::.....:/bin/tcsh

3. Create an empty repository, e.g.

mkdir /var/repo/
cvs -d ":local:/var/repo" init

4. Create an import directory, copy testcase files, and import them:

% cd
% mkdir testcase
% cd testcase
% cp ??/abnormal.txt .
% cp ??/normal.txt .
% setenv CVS_CLIENT_LOG /home/usr/cvsclient
% cvs -td ":ext:user@localhost:/var/repo" import -m 'broken import' broke LOCAL IMPORT
unrecognized request `ified normal.txt'


if you don't see the error, verify that your mailer hasn't 'fixed' these files:

abnormal.txt has a few lines, three of which end in CRLF instead of newline.
normal.txt has a few lines of ordinary unix-like text

and recheck the preconditions at the start of this procedure.

If you do get the error message, take a look at ~/cvsclient.in to see the commands the client `thinks' it is sending to the server. While there is no way I know of to intercept the stream to the server, you should be able to see that the number of characters missing from the command following the bad file is equal to the number of CRLF line endings.

THE WORKAROUND:

Fortunately, the server code is pretty straightforward. A simple way to get a clean input stream is just to have cvs reopen its input in binary mode when it is started as a server. The attached patch modifies server.c to reopen stdin as "rb". There might be a gentler way to do this, and it may be that tcsh should do whatever bash is doing, but when cvs is recompiled with the attached patch, the problem described is fixed. What, if anything else, is broken, is left as an exercise for the reader. Considering, though, that pserver and kserver both use sockets (which are binary) and that they both run the same server code as the command line server once authentication is complete, I doubt there will be any issues.

To apply the patch:

1. Download unzip, untar cvs-1.11.21
2. ./configure
3. make
(don't try to make check, it's broken)
4. cd /src/
5. patch <??/patch  (wherever you put the attached patch file)
6. make
7. cp /bin/cvs /bin/cvs-original
8. cp cvs /bin/cvs

Verify that cvs now behaves as expected. You should now be able to complete the import, check out the added repository `broke', fiddle with the files therein, update, and commit.

My point here isn't to say how cvs / cygwin / tcsh should work, only to point out that they currently aren't playing by the same rulebook. The workaround is only provided to show that the problem is correctly diagnosed, I don't have any stake in how this is resolved.

THANKS TO

Rene' Berber for allowing me to test against his server, for help in figuring out what was wrong, and for guidance in the (hopefully) proper way to post a problem report.

Igor Peshansky for insight in how to diagnose the problem.

Jay Abel

Attachment: patch
Description: Binary data

Attachment: abnormal.txt
Description: Text document

Attachment: normal.txt
Description: Text document

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]