Re: Bug: Cygwin Shell hangs, waiting, when win32-GVIM forks into background

[Brian Dessent <brian at dessent dot net>]

Linda Walsh wrote:

>         Perhaps -- you don't, but my first symptom of a problem was a simple cygwin
> program refusing to run from cmd.exe.  At the same time, I'm having
> problems with 'bash' (or 'ash') as launched from windows as well -- in both
> cases, the programs "used to work", and now I see a cmd-window come up,

I agree that it is potentially a bug if fork() spews errors, unless of
course it is being molested by some kind of BLODA type of interference. 
However that is a different issue than the one that is the subject of
this message -- I still think we're talking about two totally unrelated
phenomina.

>         So I'll go back to my first posted problem.  Why can't I fork when
> running from cmd.exe if the binary I am running is in "\windows" or
> "\windows\system32"  I'm not aware of there being special cases for \windows
> or \windows\system32 in Posix either.  The Cygwin "bin" dir is first in my
> path, so cygwin libraries should be accessible via the path.

I'm still not clear on the details here.  You have a Cygwin wrapper
program that forks and execs a native Win32 binary, then itself exits. 
This fails with fork errors if the wrapper is located in %windir% but
works otherwise?  Does it work if you use a native (MinGW) wrapper that
consists of just _execv("c:/path/to/vim.exe", argv)?  What about a
simple shell wrapper that consists of "#!/bin/sh\ncygstart
/path/to/vim"?

> > Just because the Win32 version prints that in its help doesn't mean it
> > does anything.  It's certainly not forking, since it's a native Win32
> > program.
> ---
>         That is a "non-sequitur".

Okay, let me rephrase: Often software that has a native Win32 port will
contain remnants of Unix terminology or features that do not work.

> Are you saying there is no way for
> a process to create another process out of it's own process space and
> allow it to continue on "in background", while the first process terminates?

Process creation on Windows is centered around the notion of creating a
new process with new PID and completely separate VM address space.  I'll
call this primitive *spawning* from here on to avoid confusion.

"in the background" is not a very precise technical term.  In Win32,
when spawning a child the parent gets to choose:

- whether the child starts in a suspended or running state
- whether the child is to be attached to the parent's console (if one
exists), or to create a new console for the child, or to create the
child detached from any console
- whether the child is part of the parent's process group or should be
the root of a new process group
- whether to wait and block on the child terminating or to continue
executing independently

Note the last point is I think the crutial one here: it's the parent
that decides whether to wait or continue.  The child has no control over
the behavior of the parent.

But as to "forking", what specifically does not exist anywhere in Win32,
and what Cygwin has to go through enormous hoops to emulate, is the
notion of *replacing* one process by another (exec) or *duplicating* one
process into two (fork).  These have very different semantics than
spawning a new process, so that's why I find it very wrong to say that a
native Win32 program does anything resembling a "fork".  

The exec() that Windows does have is just an elaboration of *spawning*
followed by the parent *exiting*.  On the surface this may seem to be
the same as the POSIX exec but it's not -- the main difference being
that the child has a new and distinct PID as well as potentially other
per-process attributes[1].  In other words, the child was created as a
separate and new process, it did not take over in the shoes of the
parent.

> I'm pretty sure the "&" addendum in cmd.exe does something similar to this
> within cmd.  The functionality seems to be there despite that the function
> may not be called "fork" nor have the exact same semantics of "fork".

I think you're conflating what the parent does after the new process has
been created (i.e. whether to wait and block on the child to terminate
or not) with how the process was created. 

>         It is not immediately evident where the splitting of the GUI
> process off into the background is done: whether in in cmd.exe or
> every windows program that can interactively launch processes,
> or in the processes themselves.  Usually, a "hint" is that if a program
> has an option to "-daemonize" or not, is a hint of a dual nature, though I
> certainly can't think of any on Windows, where things seem predominantly
> limited to CLI or GUI interfaces (seemingly not a "great" design
> decision, but it's a bastard schizophrenic OS, so what can one expect.

I'm not sure I follow.  Say you are a Win32 GUI executable.  You start
up, you create your windows, you interact with the user.  At no point do
you control anything about the process that created you.  Whether it
wants to wait or not for you to terminate is not something you (vim in
this case) have any control over.

>         It's a typing stressor...not always a bother, but enough so that it isn't
> automatic.  On Linux, "not a problem", but due to a cygwin incompatibility with
> Linux (a design "deficit" (bug)), I can't just
> hit Control-Z to put it in background after the fact.  I have to
> terminate the program (presuming it can be terminated without botching
> up some work or losing work) and restart it.

Well, two things.  First, the reason you can't just ^Z is because you're
mixing native and Cygwin apps.  If you used a Cygwin gvim (X11) then it
should work just as on *nix.

Second, if typing is the issue then: alias gvim='gvim &'

>         I frequently do not start commands in background, because I
> want to make sure they start correctly. On Linux, I can then suspend and
> through them in background.  Not possible on Cygwin.  Hitting the & is
> one of my "problem keys"...besides cross-hand coordination, with left
> hand in lower left holding shift (and sometimes missing and hitting
> control or capslock), the right hand has to stretch up off home-row and
> aim for 7, but often hitting 6 or 8 due to the reach taking my hand off
> home row.  I can't quite do it with 1 hand.  Most guys I know can -- larger
> hands, and some women are lucky enough to have long fingers (though I'm sure
> they suffer when trying to find gloves that fit, that expect shorter fingers
> on women with narrower hands or fingers too long when hands are wide enough
> *plegh*.

I too would hate to have to type & for every editor invocation...
Luckily there are shell aliases, shell functions, and wrapper scripts.

> Perhaps true, bit I also had a "foolish" expectation that some effort would have
> been made to make the application behave the same across multiple platforms,
> especially when it is documented to do so.

I agree that it would be nice if the MAY_FORK code had a Win32
equivalent.  It is certainly possible to do natively without requiring
any fork sematics: the process simply spawns another copy of itself,
communicates its filename arguments through a pipe or other IPC, then
terminates.  This also has the advantage of making the app into a
single-instance application, e.g. like most web browsers, where if
firefox is already running it simply opens a new tab or window, instead
of having more than one instance.

> > int main (int argc, char **argv) { _execvp ("gvim.exe", argv); }
> ---
>         Need to include the path -- that's one of the reasons for
> the redirector.

Okay, then use _execv (without the p) give an absolute Win32 path.

>         Right -- I want the graphical program to launch and return control
> to the shell no matter what shell I am in  without having to type &.  "&" is
> embedded in my brain as "launch a long running process in background while
> I go do something in foreground.  Editing doesn't fit into that paradigm.

Then a wrapper script/wrapper program seems like the correct solution if
shell aliases are out.

>         I hesitate to even suggest this, but maybe bash and other shells
> should have an option to detect non-cygwin GUI programs and launch them
> in background?  It would give a consistent shell presentation on windows
> as well as between linux-and-cygwin.  As it stand now, givm launched on
> linux launches in background.  gvim launched via cmd.exe or the windows
> shell launches in background.  The only place gvim doesn't launch
> automatically into the background is under a cygwin shell.

Oh god no.  That is going in the opposite direction of consistency with
linux.  As it is now, everything is perfectly consistent: the shell
waits for the job to complete unless it was invoked with &.  The part
that is missing is the part that exists on *nix but not on win32, the
MAY_FORK stuff in vim.  And I dare say that the reason that is not
consistent is that you're mixing native Win32 with Cygwin here.  If this
was vim or gvim built as a Cygwin application then all that MAY_FORK
stuff would work exactly as on *nix and the behavior would be identical
-- without having to stoop to the level of looking at the PE header.

But if you really want that, here's an idea that uses wrapper scripts:

$ mkdir ~/wrappers    # or whereever
$ cd ~/wrappers
$ cat <<EOF >wrapper.sh
#!/bin/bash

export PATH=$(perl -e 'print join(":", grep(!/\Q$ARGV[0]\E/, 
                       split(":", $ENV{PATH})))' "$(dirname "$0")")

arg0=$(basename "$0")

if file "$(which "$arg0")" | grep -q '(console)'; then
  exec "$arg0" "$@"
fi
setsid "$arg0" "$@" &
EOF
$ chmod +x wrapper.sh
$ ln -s wrapper.sh notepad
$ export PATH="$HOME/wrappers:$PATH"   # and add to .profile
$ notepad

Now, when you type "notepad", it finds the wrapper script, which removes
the wrappers dir from the front of the path, checks the type of the
target, and executes it in the background if it's a GUI-mode app.  You
can then just place any many of these symlinks in the wrappers dir as
you want, each with the name of the command to wrap.

>         If cygwin is intended to follow the linux in behavior, I assert,
> in this case, it is not and would be more compatible if it new a launched
> binary was a Windows-GUI program that would normally be launched in
> on windows (or, with its equivalent posix-GUI, on linux).

You're trying to fix the wrong thing.  Cygwin is acting exactly as *nix,
it's that native gvim that's out of line.  On *nix if gvim had the
MAY_FORK stuff disabled you'd be in the same exact situation, except
that there is no shell on *nix that somehow sniffs binaries to see if
they are linked with X11 client libs.

>         Sorry if my calling the wrong behavior a "bug" irked you.  Doesn't

No offense taken or intended.

Brian

[1] Yes, on Win32 there is the ability to inherit some things, like open
handles.  But this is at the discretion of the parent, which can mark
them non-inheritable.  If you look at CreateProcess you will see that it
is essentially a giant list of things for the parent to specify of the
new process: executable, arguments, environment, working directory,
window station, process and thread security descriptors, handle
inheritance, console (inherit parent's, create new, detached), error
mode, process group, process state (i.e. suspended/running, being
debugged/not debugged), full screen or windowed (for console apps) or
maximized/minimized/hidden/normal (for windowed apps), shared or
separate VDM, process priority, etc.

Contrast this to the POSIX exec, where the parent can specify the name
of the new binary to run, the command line arguments, and the
environment.  The parent can also mark open fds as close-on-exec.  But
everything else in the child is inherited, and remains unchanged: PID,
PPID, GID, access groups, working directory, root directory, controlling
terminal, resource usages, interval timers, resource limits, umask, and
signal masks.

That's not to say that the parent cannot control any of those aspects
when creating child processes - that's very much not the case.  It's
just that it does it first by forking, then adjusting whatever aspects
are necessary (e.g. close some fds, chroot, set umask) and then execing
the new image.

In other words, the Win32 way is to create from scratch specifying
everything at creation time; the POSIX way is to first duplicate an
existing process, adjust as necessary, then replace the executing code
with the new.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/