Bug 214611 - UM: stdout output ceases under certain conditions

[Glenn Washburn]

On Wed, 6 Oct 2021 17:44:14 +0100
Anton Ivanov <anton.ivanov at kot-begemot.co.uk> wrote:

> 
> 
> On 06/10/2021 16:57, Anton Ivanov wrote:
> > 
> > 
> > On 04/10/2021 17:54, Glenn Washburn wrote:
> >> On Mon, 04 Oct 2021 14:48:34 +0200
> >> Johannes Berg <johannes at sipsolutions.net> wrote:
> >>
> >>> On Sat, 2021-10-02 at 21:00 -0500, Glenn Washburn wrote:
> >>>> Hi list,
> >>>>
> >>>> I'm notifying the list of a bug report[1] I created in the kernel
> >>>> bugzilla. I'm not subscribed to this list, so please add this my email
> >>>> in any replies to this email.
> >>>
> >>>
> >>>> [1] https://bugzilla.kernel.org/show_bug.cgi?id=214611
> >>>
> >>> This really has nothing to do with UBD or something. What's happening is
> >>> that you're using the command line badly.
> >>>
> >>> What do you expect this:
> >>>
> >>>    ... < <(cat /dev/null)
> >>>
> >>> to do?
> >>
> >> This was just a way to trigger the issue I was seeing. I have a bash
> >> script which was doing something like the following:
> >>
> >> grep "search" /path/to/file |
> >> while read VAR; do
> >>    run_some_script_which_eventually_runs_uml $VAR;
> >> done
> >>
> >> I was confused why running this script caused UML to lose output always
> >> when mounting the ubd in the UML mount script. And it didn't happen
> >> when I ran "run_some_script_which_eventually_runs_uml" alone. Since the
> >> amount of data returned by the grep was small, this issue was triggered
> >> all the time. If the output were a lot of data, I might have noticed
> >> that early runs of run_some_script_which_eventually_runs_uml would not
> >> have output disappear after mounting. Thanks for debugging this.
> >>
> >>> What happens is that the shell creates a pipe. This pipe is connected on
> >>> the one side to fd:1 in UML (stdin) and on the other to stdout of 'cat'.
> >>>
> >>> Now this is all fine, but 'cat' will *quit immediately* since it cannot
> >>> read anything from /dev/null (it's write-only!).
> >>>
> >>> Therefore, the fd:1 in UML will be invalidated pretty much immediately,
> >>> receiving EPOLLHUP.
> >>>
> >>> This is detected by the epoll code, raising an interrupt into the line
> >>> level code, and the line code then closes the stdio console channel
> >>> entirely, including stdout.
> >>
> >> This seems like it could be a bug. Couldn't the console not be closed,
> >> but the console handling code internally mark stdin as closed? Perhaps
> >> there could even be logic to detect if stdin and stdout are from the
> >> same fd, then close the console, otherwise don't. From a user
> >> perspective, thinking of UML as a normal process, it doesn't make sense
> >> that closing stdin would close stdout as well.
> > 
> > There is an even more convoluted case where the stdin is a socket (which
> > is possible - you pass it to UML as a fd:N). That can be half-closed.
> > 
> > Looking at it at the moment, but to be honest, separating the logic for in
> > and out if the fd is the same is going to be quite difficult (if at all
> > possible). It all ends as EPOLL events at the bottom. Even if you handle IN
> > and OUT separately in the upper layers, the kernel will handle them as the
> > same fd and any event (f.e. closure) will show up on both.
> 
> Further to this, the same holds even if we start playing games with multiple
> EPOLL descriptors, dup-ing fds, etc, the event will still show up on all of
> them.

Thanks for looking into this. If I'm understanding correctly, you're
looking at the case where the UML process has STDIN and STDOUT to the
same file descriptor. However, the situation is when STDIN is to a pipe
that gets closed and STDOUT is to something else (pty, tty, file,
different pipe, etc..). Does your logic still hold true in this case?

Glenn

> 
> I frankly do not have any ideas at present to solve this. This is just
> the way it is - a file close event for one fd will show up on all instances
> where it is mentioned.
> 
> A.
> 
> > 
> > A.
> > 
> >>
> >>> If anything, the bug is that when you're not causing enough interrupts
> >>> by using ubd, somehow this situation doesn't get detected, and the
> >>> console remains open, so you still see the output... I think this might
> >>> be if closing the FD didn't generate a SIGIO?
> >>
> >> This leads to strange behavior. But for this issue, I think the
> >> suggestion above would obviate the need to do anything about this.
> >>
> >>> In fact, if you generate SIGIO in *any* other way, including pressing
> >>> enter while the script is running even if stdin is redirected from your
> >>> dead cat [1], you still get the same behaviour of the channel getting
> >>> closed.
> >>
> >> Ok, I've confirmed that. I think that's another reason to fix this in
> >> a manner that doesn't tie stdin to stdout, just hitting enter can make
> >> outut disappear (non-intuitive). In my case, stdout was going to a file.
> >> So my intuition would say that stdin (from a pipe) and stdout (to a
> >> file) shouldn't be connected (yes a program can create any kind of
> >> connection it wants, but it's not intuitive). It would be like rsync
> >> -av dir1 dir2 < <(cat /dev/null) having its output disappear if you
> >> pressed enter in the middle of the run.
> >>
> >>>
> >>>
> >>> johannes
> >>>
> >>> [1] did I really just write that? heh.
> >>
> >> Thanks, got a chuckle out of that.
> >>
> >> _______________________________________________
> >> linux-um mailing list
> >> linux-um at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/linux-um
> >>
> > 
>