Can't run x11docker within script.

[hoshsadiq]

Trying to run x11docker from within a script, however, xorg refuses to run. Error is as follows:

x11docker ERROR: Error during startup of X server --xorg.
  Last lines of xinit log:

/usr/lib/xorg/Xorg.wrap: Only console users are allowed to run the X server



  Type 'x11docker --help' for usage information
  Debug options: '--verbose' (full log) or '--debug' (log excerpt).
  Logfile will be: /home/hosh/.cache/x11docker/x11docker.log
  Please report issues at https://github.com/mviereck/x11docker

sed: -e expression #1, char 16: Invalid collation character
Error: Can't open display: :100

Trying to run it simply using:

#!/usr/bin/env bash

set -exo pipefail
x11docker --gpu --homedir "/tmp/kodi/home" -- --volume /tmp/kodi/media:/media:ro -- erichough/kodi &>/tmp/kodi.log &
sleep 20
echo "it's running!"

Have had this issue previously while doing something else, but was unable to fix it. Any ideas how I can get this running?

[mviereck]

Does the same command work if you run it directly?
Where/how do you run the script? It seems to me on tty, am I right?

I just did a test run on tty with command x11docker --xorg x11docker/lxde one time directly and one time in a script. It works so far.

Can you show me cat /etc/X11/Xwrapper.config?
Here it is:

allowed_users=anybody
needs_root_rights=yes

However, this configuration should only be needed if running --xorg within a running X. Compare: https://github.com/mviereck/x11docker/wiki/Setup-for-option---xorg

[hoshsadiq]

It runs fine if I run it directly. Correct, it's on tty.

$ cat /etc/X11/Xwrapper.config
allowed_users=console

The file includes some comments that I've removed.

I've been able to overcome it by using xpra, but surely I shouldn't have to?

[mviereck]

I've been able to overcome it by using xpra, but surely I shouldn't have to?

xpra would not run directly on tty and x11docker uses --weston-xwayland as a fallback.
That seems to work on your system.
However, it should work with no option / --xorg nonetheless.

Can you please run without --xpra in the script and afterwards show me ~/.cache/x11docker/x11docker.log at www.pastebin.com?

Can you show me the output of command tty and echo $XDG_VTNR?

[hoshsadiq]

$ echo $XDG_VTNR
7
$ tty
/dev/pts/2
$ ./test
+ sleep 20
+ x11docker --gpu --debug --homedir /tmp/kodi/home -- --volume /tmp/kodi/media:/media:ro -- erichough/kodi
+ echo 'it'\''s running!'
it's running!
$ cat /tmp/kodi.log
x11docker note: Option --homedir is deprecated.
  Please use --home=DIR instead.

DEBUGNOTE[18:00:35,512]: ps can watch root processes: yes
DEBUGNOTE[18:00:35,556]: Command at Line 5649 returned with error code 1:
  logname 2> /dev/null
  7365 - ::check_hostuser::main::main
DEBUGNOTE[18:00:35,578]: Command at Line 5649 returned with error code 1:
  Lognameuser="$(logname 2>/dev/null)"
  7365 - ::check_hostuser::main::main
x11docker WARNING: User hosh is member of group docker.
  That allows unprivileged processes on host to gain root privileges.

DEBUGNOTE[18:00:36,255]: 
x11docker version: 6.1.1
docker version:    Docker version 18.09.7, build 2d0083d
Host system:       Linux Mint 19.1
Command:           '/usr/bin/x11docker' '--gpu' '--debug' '--homedir' '/tmp/kodi/home' '--' '--volume' '/tmp/kodi/media:/media:ro' '--' 'erichough/kodi' 
Parsed options:     --gpu --debug --homedir '/tmp/kodi/home' -- '--volume' '/tmp/kodi/media:/media:ro' '--' 'erichough/kodi'
DEBUGNOTE[18:00:36,270]: Dependency check for --xorg: 0
DEBUGNOTE[18:00:36,282]: Dependencies of --xorg already checked: 0 
DEBUGNOTE[18:00:36,296]: Dependencies of --xorg already checked: 0 
x11docker note: Using X server option --xorg

x11docker note: Option --xorg: x11docker assumes that you need
  a window manager. If you don't want this, run with option --desktop. 
  Enabling option --wm to provide a window manager.

x11docker WARNING: Option --gpu degrades container isolation.
  Container gains access to GPU hardware.
  This allows reading host window content (palinopsia leak)
  and GPU rootkits (compare proof of concept: jellyfish).

x11docker note: Could not find Xephyr, Xnest, nxagent,
  xpra, weston+Xwayland or kwin_wayland+Xwayland to run a nested X server.
  Consider to install one of them.

x11docker WARNING: Found custom DOCKER_RUN_OPTIONS.
  x11docker will add them to 'docker run' command without
  a serious check for validity or security. Found options:
   '--volume' '/tmp/kodi/media:/media:ro'

DEBUGNOTE[18:00:36,397]: Stored background pid 20016 of tailstdout
DEBUGNOTE[18:00:36,403]: Waiting for file content in /home/hosh/.cache/x11docker/erichough-kodi-1083504/container.pid1pid
DEBUGNOTE[18:00:36,415]: Stored background pid 20029 of tailstderr
DEBUGNOTE[18:00:36,456]: Stored background pid 20048 of watchpidlist
DEBUGNOTE[18:00:36,472]: Stored background pid 20055 of watchmessagefifo
x11docker note: Current X server :0.0 runs on tty 7.
  Access it with [CTRL][ALT][F7].

/usr/bin/x11docker: line 2373: [: not a tty: integer expression expected
x11docker note: New Xorg server  will run on tty not a tty.
  Access it with [CTRL][ALT][Fnot a tty].

x11docker WARNING: On debian 9, switching often between multiple X servers can
  cause a crash of one X server. This bug may be debian specific and is probably
  some sort of race condition. If you know more about this or it occurs on
  other systems, too, please report at https://github.com/mviereck/x11docker.

  You can avoid this issue with switching to a black tty before switching to X.

DEBUGNOTE[18:00:36,664]: New X environment:
  DISPLAY=:100 XAUTHORITY=/home/hosh/.cache/x11docker/erichough-kodi-1083504/share/Xclientcookie XSOCKET=/tmp/.X11-unix/X100  X11DOCKER_CACHE=/home/hosh/.cache/x11docker/erichough-kodi-1083504
DEBUGNOTE[18:00:36,710]: X server command:
  /usr/bin/Xorg :100 vtnot a tty \
  -dpms -s off -retro \
  +extension RANDR \
  +extension RENDER \
  +extension GLX \
  +extension XVideo \
  +extension DOUBLE-BUFFER \
  +extension SECURITY \
  +extension DAMAGE \
  -extension X-Resource \
  -extension XINERAMA -xinerama \
  -extension MIT-SHM \
  -auth /home/hosh/.cache/x11docker/erichough-kodi-1083504/Xservercookie \
  -nolisten tcp \
  +extension Composite +extension COMPOSITE \
  -extension XTEST -tst \
  -dpi 96 \
  -verbose -keeptty
DEBUGNOTE[18:00:36,930]: Users and terminal:
  x11docker was started by:                       hosh
  As host user serves (running X, storing cache): hosh
  Container user will be:                         hosh
  Container user password:                        x11docker
  Getting permission to run docker with:          bash -c 
  Running X and other user commands with:         bash -c
  Terminal for password frontend:                 bash -c
  Running on console:                             yes
  Running over SSH:                               no
DEBUGNOTE[18:00:36,953]: Found tini binary: /usr/bin/docker-init
x11docker WARNING: Sharing device file: /dev/dri

x11docker WARNING: Sharing device file: /dev/vga_arbiter

DEBUGNOTE[18:00:37,486]: Generated docker command:
  docker run --tty --rm --detach \
  --name x11docker_X100_1083504_erichough-kodi \
  --user 1000:1000 \
  --env USER=hosh \
  --userns host \
  --group-add 44 \
  --group-add 29 \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --security-opt label=type:container_runtime_t \
  --volume '/usr/bin/docker-init':'/x11docker/tini':ro \
  --tmpfs /run --tmpfs /run/lock \
  --volume '/home/hosh/.cache/x11docker/erichough-kodi-1083504/share':'/x11docker':rw \
  --volume '/tmp/kodi/home':'/home/hosh':rw \
  --device '/dev/dri':'/dev/dri':rw \
  --device '/dev/vga_arbiter':'/dev/vga_arbiter':rw \
  --volume '/tmp/.X11-unix/X100':'/X100':rw \
  --workdir '/tmp' \
  --entrypoint env \
  --env 'container=docker' \
  --env 'XAUTHORITY=/x11docker/Xclientcookie' \
  --env 'DISPLAY=:100' \
   '--volume' '/tmp/kodi/media:/media:ro' \
  -- erichough/kodi /bin/sh - /x11docker/container.CMD.sh
DEBUGNOTE[18:00:37,514]: Command at Line 4041 returned with error code 1:
  grep -e '^DOCKER_'
  7455 - ::create_dockerrc::main::main
DEBUGNOTE[18:00:37,724]: Stored background pid 20772 of containershell
DEBUGNOTE[18:00:37,731]: Waiting for X server --xorg to be ready.

x11docker ERROR: Error during startup of X server --xorg.
  Last lines of xinit log:

/usr/lib/xorg/Xorg.wrap: Only console users are allowed to run the X server

  

  Type 'x11docker --help' for usage information
  Debug options: '--verbose' (full log) or '--debug' (log excerpt).
  Logfile will be: /home/hosh/.cache/x11docker/x11docker.log
  Please report issues at https://github.com/mviereck/x11docker

sed: -e expression #1, char 16: Invalid collation character
DEBUGNOTE[18:00:37,911]: Terminating x11docker.
Error: Can't open display: :100
DEBUGNOTE[18:00:37,937]: List of stored background processes:
20016 tailstdout
20029 tailstderr
20048 watchpidlist
20055 watchmessagefifo
20772 containershell
DEBUGNOTE[18:00:37,952]: time to say goodbye (finish-subshell)
DEBUGNOTE[18:00:38,028]: Checking: 20772 (containershell): 20772 pts/2    00:00:00 bash
DEBUGNOTE[18:00:38,033]: Stopped waiting for file content of /home/hosh/.cache/x11docker/erichough-kodi-1083504/container.pid1pid due to terminating signal.
DEBUGNOTE[18:00:38,055]: Command at Line 468 returned with error code 1:
  return 1
  7397 - ::setup_verbosity::main::main
DEBUGNOTE[18:00:38,087]: Terminating 20772 (containershell) x11docker: 20772 pts/2    00:00:00 bash
DEBUGNOTE[18:00:38,456]: time to say goodbye (watchpidlist)
DEBUGNOTE[18:00:52,759]: Command at Line 5526 returned with error code 1:
  bash -c "exec                                   xinit $Xinitrc -- $Xcommand             >> $Xinitlogfile  2>&1 "
  7501 - ::start_xserver::main::main
DEBUGNOTE[18:00:52,772]: time to say goodbye (main)
DEBUGNOTE[18:00:52,784]: Terminating x11docker.
DEBUGNOTE[18:00:52,800]: List of stored background processes:
20016 tailstdout
20029 tailstderr
20048 watchpidlist
20055 watchmessagefifo
20772 containershell
DEBUGNOTE[18:00:52,866]: Checking: 20772 (containershell): 
DEBUGNOTE[18:00:52,926]: Checking: 20055 (watchmessagefifo): 20055 pts/2    00:00:00 bash
DEBUGNOTE[18:00:52,969]: Terminating 20055 (watchmessagefifo) x11docker: 20055 pts/2    00:00:00 bash
DEBUGNOTE[18:00:53,153]: Checking: 20048 (watchpidlist): 
DEBUGNOTE[18:00:53,236]: Checking: 20029 (tailstderr): 20029 pts/2    00:00:00 bash
DEBUGNOTE[18:00:53,307]: Checking: 20016 (tailstdout): 20016 pts/2    00:00:00 bash
DEBUGNOTE[18:00:53,319]: time to say goodbye (finish)
DEBUGNOTE[18:00:53,331]: Exitcode 1

x11docker.log

[mviereck]

It seems that command tty does not give a valid output if running within your script ("not a tty").
I did a quick workaround in x11docker to catch a failing tty check, I will have to check further implications later.

It seems this time you run in X, not from tty, right?

Please update to latest master version of x11docker and try again.

[hoshsadiq]

This has introduced a bug, option -S doesn't exist for /usr/bin/env, seems stable coreutils currently is at 8.28, just shy of -S support.

$ x11docker --version
/usr/bin/env: invalid option -- 'S'
Try '/usr/bin/env --help' for more information.
$ /usr/bin/env --help
Usage: /usr/bin/env [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]
Set each NAME to VALUE in the environment and run COMMAND.

Mandatory arguments to long options are mandatory for short options too.
  -i, --ignore-environment  start with an empty environment
  -0, --null           end each output line with NUL, not newline
  -u, --unset=NAME     remove variable from the environment
  -C, --chdir=DIR      change working directory to DIR
      --help     display this help and exit
      --version  output version information and exit

A mere - implies -i.  If no COMMAND, print the resulting environment.

GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Full documentation at: <http://www.gnu.org/software/coreutils/env>
or available locally via: info '(coreutils) env invocation'

[mviereck]

This has introduced a bug, option -S doesn't exist for /usr/bin/env, seems stable coreutils currently is at 8.28, just shy of -S support.

Thanks for reporting this. I have removed this and switched back to #! /usr/bin/env bash. Here I have coreutils 8.30.

Parsing the shebang seems to be quite different across the *nixes, and there is no general solution: https://stackoverflow.com/a/4304187

So x11docker cannot set bash options -i +H itself.

In your script you can replace x11docker with bash -i +H -- $(which x11docker), that should help.
However, this is an ugly workaround, and I hope I'll find a better solution.

[hoshsadiq]

Works!

Re -S, I guess you could simply check $- to see if the relevant options are there, and rerun bash using correct options if they're not. Something like:

if [[ "$-" != *"i"* || "$-" != *"H"* ]]; then
	/usr/bin/env bash -i +H "$0" "$@"
	exit $?
fi

I'm not sure if i and H are the correct options since I can't test against /usr/bin/env -S.

[mviereck]

Works!

Great! But:

Re -S, I guess you could simply check $- to see if the relevant options are there, and rerun bash using correct options if they're not.

A good idea, but it seems there is a bug in bash. Sometimes it works, and sometimes I get:

free(): invalid next size (fast)

This occured in my first tests with bash -i +H -- $(which x11docker), too, but I could not reproduce. Now it occures several times. So the workaround above might fail sometimes, too.

---

For the record, my implementation:

# Enforce running in interactive mode to allow commands tty and weston-launch. Not supported in general by shebang.
case $- in
  *i*) ;;
  *) 
    bash -i -- "$0" "$@" 
    exit $?
  ;;
esac
set +H

[mviereck]

I have added a new experimental option --enforce-i to enforce bash interactive mode.

grep -q -- "--enforce-i" <<< "$*" && case $- in
  *i*) set +H ;;
  *)   exec bash --noprofile --norc --noediting -i -- "$0" "$@" ;;
esac

With additional bash options --noprofile --norc --noediting I could not reproduce the bash memory bug.
However, I still do not trust this entirely.
Could you please use --enforce-i in your script and report back if you encounter issues some day?

[hoshsadiq]

Seems to work. however, what's strange is that I removed all these additional options, and ran it, and this time it works fine. Reason I did that is because I saw the --interactive flag while trying this, so wanted to test if that would work.

Logs

[mviereck]

Ok, I see.
I've included a fallback check if tty fails (and it did in the log): If DISPLAY or WAYLAND_DISPLAY are set, x11docker assumes that it does not run on console.

Probably your script will fail without --enforce-i if you run it on console.

Good that you mention --interactive. I just tested it, it fails along with --enforce-i. sigh.

[hoshsadiq]

Probably your script will fail without --enforce-i if you run it on console.

Sorry, I'm not sure I understand. I still don't really fully understand tty and pts and the like. How can I test this?

[mviereck]

How can I test this?

You could switch to a console with CTRL+ALT+F2 and run your script there.

I still don't really fully understand tty and pts and the like.

For the differences:
Your system has a set of ttys. You can access 12 of them with CTRL+ALT+F1..F12.

• The ttys with a black screen and a cursor are called 'console'. (Output of tty: /dev/ttyN)
• The tty with X is not a console; there you can run a 'terminal window' / 'terminal emulator'. (Output of tty: /dev/ptsN)
• The ttys with a black screen and no cursor are 'unused ttys' / 'unused consoles'.

x11docker makes some decisions depending on the output of tty.

• On console it can run Xorg directly on the same tty.
• In X x11docker can run Xorg on a different tty.
• In some circumstances tty gives result not a tty. This can happen if it runs e.g. nested in a script or from a desktop starter.

x11docker did not catch not a tty previously, that caused the bug you have reported. I've seen that in your log with the buggy line /usr/bin/Xorg :100 vtnot a tty \. (The same bug was reported later in #177.)
x11docker believed to run on console and did not recognize that it runs in X. It errately tried to run Xorg with wrong parameters.

Now x11docker catches the not a tty case.
In some cases it can help to use option --enforce-i to get a useful tty output and to allow commands like e.g. Xorg and weston-launch. However, I am not entirely happy with this option; hopefully it will be needed rarely.

[hoshsadiq]

Ah, cool.

Okay, so, I tried it on multiple console, and it seems to be working without --interactive or --enforce-i (or --xpra for that matter). Logs

[mviereck]

Thank you for your tests!
The latest release v6.2.0 includes the fixes for this ticket.
--enforce-i is part of this release. However, it seems it will be needed for rare edge cases like in #166 now.

[hoshsadiq]

Thanks for the fix @mviereck