docker-for-win: Use runx to provide X on MS Windows

[mviereck]

Current master version of x11docker drops option --vcxsrv on MS Windows.

x11docker can be started with runx.
Example: runx -- x11docker x11docker/check.
Alternatively, it is possible to source runx in ~/.bashrc.

• WSL:
  • EDIT: runx supports XWin and VcXsrv. It prefers XWin if available.
    • For XWin install Cygwin with packages xinit xauth and xwininfo.
    • For VcXsrv install VcXsrv.
  • Dependencies in WSL: xauth and xwininfo/x11-utils.
  • If nested X servers like Xephyr, nxagent or xpra are installed in WSL, x11docker will use them additionally.
• Cygwin
  • x11docker still supports --xwin in Cygwin/X without needing runx.
  • Dependencies: xinit, xauth, xwininfo.
  • Alternatively, runx can be used.
• MSYS2
  • VcXsrv must be installed on MS Windows.

EDIT:
runx can run VcXsrv or XWin in WSL and Cygwin.
In MSYS2 it can only run VcXsrv.

This thread aims to announce this change and to ask for feedback / test results.
@1138-4eb would you please have a look?

[mviereck]

@1138-4eb could you please run some tests?

So far I tested runx with VcXsrv and XWin in WSL, Cygwin and MSYS2.
I've also installed Docker/MobyVM. It failed to start here because my Windows VM does not support nested virtualization, but the created Docker command looks right.

So far I assume runx -- x11docker [...] will work in all of MSYS2, Cygwin and WSL with --vcxsrv and --xwin. (However, falling back to --vcxsrv in MSYS2).
In Cygwin you would not need runx, x11docker can natively use XWin.

It would be nice if you check some of the possible setups:

• If they work at all.
• If shown messages make sense.

	WSL	Cygwin	MSYS2
runx -- x11docker [...]
runx --xwin -- x11docker [...]			-
runx --vcxsrv -- x11docker [...]
x11docker [...]	-		-

Setups with entry - should fail, but give a useful message how to get a working setup.

[eine]

Sure. I've been quite busy lately, but I can give it a try.

First off, thanks a lot for the example commands and the table. I was just going to ask you to create some test list. This table will make it much easier. Let's go for it:

NOTE: I'm trying [...] = x11docker/check.

	WSL	Cygwin	MSYS2
runx -- x11docker [...]	fails, *4	two servers, *3	ok *1 *2
runx --xwin -- x11docker [...]	fails, *5	two servers, *3	falls back to --vcxsrv, ok *1 *2
runx --vcxsrv -- x11docker [...]	fails, *4	two servers, *3	ok *1 *2
x11docker [...]	fails, note about using runx	ok *2	fails, note about using runx

*1 runx WARNING: Missing dependency: xwininfo is shown, but C:\Program Files\VcXsrv\xwininfo.exe exists.

*2 x11docker note: Your docker version does not support option --group-add. Possible sound or GPU setup may fail. Please update your docker installation. is shown. I don't think I can do much about that on Win10. So, the suggestion to update the installation might not be valid here.

*3 Compared to running x11docker alone, runx show an additional message runx note: If you get application error messages like 'Cannot open display' or 'Invalid MIT-MAGIC-COOKIE', the X cookie might be broken. You can remove old cookies and stop running X servers with: runx --cleanup. Moreover x11docker seems not to detect the display started by runx. I see DISPLAY=10.0.75.1:100 XAUTHORITY=/home/eine/runx_Xauthority generated by runx just before the output of x11docker starts. But x11docker starts an additional xwin server anyway. Sometimes, it fails to connect to it in lees than 60s. When this happens, I cannot terminate the process with Ctrl+C, I need to close cygwin. Otherwise, the window is properly shown; however, I cannot tell which of the two running xwin instances is it running on.

*4:

eine@DESKTOP-E1ER43H:~$ /mnt/d/data-dev/github/runx//runx -- /mnt/d/data-dev/github/x11docker/x11docker x11docker/check
runx note: MSYS2 does not provide xauth to create a cookie.
  Fallback: Enabling discouraged option --no-auth.

runx ERROR: No X server found for MSYS2.
  Please install X server VcXsrv:
      https://sourceforge.net/projects/vcxsrv

*5:

eine@DESKTOP-E1ER43H:~$ /mnt/d/data-dev/github/runx//runx --xwin -- /mnt/d/data-dev/github/x11docker/x11docker x11docker
/check
runx note: MSYS2 does not provide xauth to create a cookie.
  Fallback: Enabling discouraged option --no-auth.

runx note: runx in MSYS2 does not support --xwin.
  Fallback: Enabling option --vcxsrv.

runx ERROR: No X server found for MSYS2.
  Please install X server VcXsrv:
      https://sourceforge.net/projects/vcxsrv

[mviereck]

Much thanks for the test runs!

WSL seems to be misdetected as MSYS2. That might be fixed now, compare #157.

Moreover x11docker seems not to detect the display started by runx. I see DISPLAY=10.0.75.1:100 XAUTHORITY=/home/eine/runx_Xauthority generated by runx just before the output of x11docker starts. But x11docker starts an additional xwin server anyway.
however, I cannot tell which of the two running xwin instances is it running on.

ok, that is not obvious. x11docker runs its own instance of xwin and does not use the one provided by runx. You could use the runx-xwin with option --hostdisplay.

Sometimes, it fails to connect to it in lees than 60s. When this happens, I cannot terminate the process with Ctrl+C, I need to close cygwin.

I had some trouble with xwininfo, too. I hoped it would be fixed. Unfortunately, if xwininfo on Windows connects a running X server, it waits infinitely for a response instead of terminating with an error if it does not get a response.
Maybe I should drop the check on Windows at all and use a random display number by default. The xwininfo check is dropped at all now, in x11docker as well as in runx. By default a random display number is used.

So, the suggestion to update the installation might not be valid here.

It is invalid, indeed. What does docker --version show on your system?

[eine]

WSL seems to be misdetected as MSYS2. That might be fixed now, compare #157.

As commented in #157, -e xterm seems to work, but x11docker/check does not. I can see a note shown in the top right corner of the screen telling that xpra will start soon, but the scripts just exit.

$ runx -- x11docker x11docker/check
runx note: Using random display number :2984.
  If this display number is already in use, start of runx will fail.
  You can specify a display number N with option '--display N'.

runx note: Windows firewall settings can forbid application access
  to the X server. If no application window appears, but no obvious error
  is shown, please check your firewall settings.
  Compare:   https://github.com/mviereck/x11docker/issues/108

runx note: If you get application error messages like 'Cannot open display'
  or 'Invalid MIT-MAGIC-COOKIE', the X authentication cookie might be broken.
  You can remove old cookies and stop running X servers with: runx --cleanup

DISPLAY=10.0.75.1:2984 XAUTHORITY=/home/eine/runx_Xauthority
x11docker note: Using X server option --xpra

x11docker note: Your xpra version 'xpra v0.17.6' is out of date. It is
  recommended to install at least xpra v1.0.

x11docker note: Xpra startup is rather slow. For faster startup
  with seamless applications,   try --nxagent.
  If security is not a concern, try --hostdisplay.

mkdir: cannot create directory ‘/run/user/1000’: Permission denied
chown: cannot access '/run/user/1000': No such file or directory
chmod: cannot access '/run/user/1000': No such file or directory
x11docker note: Stay tuned, xpra will start soon.

ok, that is not obvious. x11docker runs its own instance of xwin and does not use the one provided by runx. You could use the runx-xwin with option --hostdisplay.

With --hostdisplay it works: runx -- x11docker x11docker/check.

In both cases, an Xwin server is started but not closed.

Maybe I should drop the check on Windows at all and use a random display number by default.

As long as it does not conflict with a another running server, I think it is ok for it to be random.

It is invalid, indeed. What does docker --version show on your system?

# docker --version
Docker version 18.09.2, build 6247962

EDIT

runx and x11docker in cygwin still create two X servers.

[mviereck]

I can see a note shown in the top right corner of the screen telling that xpra will start soon, but the scripts just exit.

Your xpra version is quite old:

x11docker note: Your xpra version 'xpra v0.17.6' is out of date. It is
recommended to install at least xpra v1.0.

Probably x11docker sets an xpra option that is not recognized.
Can you please give me an x11docker.log with --xpra in WSL? E.g.

runx -- x11docker --xpra --verbose x11docker/check

In both cases, an Xwin server is started but not closed.

Should be fixed now. Old WSL versions did not terminate Windows processes after a kill in Linux. The current one does.

As long as it does not conflict with a another running server, I think it is ok for it to be random.

It is not a nice solution, but will fail less often than xwininfo would block the script. The random number is just the smaller devil.

runx and x11docker in cygwin still create two X servers.

That is right. x11docker does not use the X server from runx if it can run XWin itself.

x11docker note: Your docker version does not support option --group-add. Possible sound or GPU setup may fail. Please update your docker installation.

Should be fixed now.

[eine]

Your xpra version is quite old:

x11docker note: Your xpra version 'xpra v0.17.6' is out of date. It is
recommended to install at least xpra v1.0.

Yes. The WSL I am using is Debian Stretch: https://packages.debian.org/search?keywords=xpra&searchon=names&suite=stretch&section=all

Probably x11docker sets an xpra option that is not recognized.
Can you please give me an x11docker.log with --xpra in WSL? E.g.

runx -- x11docker --xpra --verbose x11docker/check

x11docker.log

In both cases, an Xwin server is started but not closed.

Should be fixed now. Old WSL versions did not terminate Windows processes after a kill in Linux. The current one does.

Yes, it seems to be fixed now.

As long as it does not conflict with a another running server, I think it is ok for it to be random.

It is not a nice solution, but will fail less often than xwininfo would block the script. The random number is just the smaller devil.

I think that the random number is not a devil at all. There are some other docker features that are mapped randomly, so it is coherent. As said, I'm just worried if there is some possibility of conflict.

runx and x11docker in cygwin still create two X servers.

That is right. x11docker does not use the X server from runx if it can run XWin itself.

I would expect to be the other way: x11docker should create another X server if it cannot use the one provided by runx. Actually, it can be avoided with runx [...] -- x11docker --hostdisplay [...]. It might be worth adding a note about this somewhere.

x11docker note: Your docker version does not support option --group-add. Possible sound or GPU setup may fail. Please update your docker installation.

Should be fixed now.

It is.

---

	WSL	Cygwin	MSYS2
runx -- x11docker [...]	*1	*2	ok, *3
runx --xwin -- x11docker [...]	*1	*2	ok, *3, *4
runx --vcxsrv -- x11docker [...]	*1	*2	ok, *3
x11docker [...]	fails, note about using runx	ok	fails, note about using runx

*1 It fails because of xpra, as commented above. But runx [...] -- x11docker --hostdisplay [...] is correct.

*2 It works, but two X servers are started, as commented above. However, it can be avoided with runx -- x11docker --hostdisplay [...], which works ok too and uses a single X server.

*3 runx shows the following note:

runx WARNING: Option --no-auth: Cookie authentication is disabled!
  Your X server vcxsrv listens on TCP connections without any protection!
  This is a security leak. Please use option --no-auth for debugging only.

and then x11docker shows:

x11docker WARNING: Command 'xauth' not found.
  Please install 'xauth' to allow X cookie authentication.
  Securing X access with cookie authentication is not possible.
  Fallback: Disabling X authentication protocol. (option --no-auth)
  You can look for the package name of this command at:
 https://github.com/mviereck/x11docker/wiki/dependencies#table-of-all-packages

x11docker note: To allow protection against X security leaks,
  please install one or more of:
    xpra, Xephyr, nxagent, weston+Xwayland, kwin_wayland+Xwayland or Xnest,
  or run a second Xorg server with option --xorg.

x11docker WARNING: Option --hostdisplay allows less security hardening.
  It is recommended to use another X server option like --xpra or --nxagent.

I think that some of these should not be shown. E.g. 'xauth' does exist, I believe, but it is not used because runx sets --no-auth. Also, the note about using xpra, Xephyr, etc. does apply to MSYS + vcxsrv?

*4 runx in MSYS2 does not support --xwin. Fallback: Enabling option --vcxsrv.

[totaam]

Yes. The WSL I am using is Debian Stretch..

We have a dedicated wiki page explaining why using the Debian (Stretch) package(s) is a bad idea: Broken Distribution Packages

[eine]

Yes. The WSL I am using is Debian Stretch..

We have a dedicated wiki page explaining why using the Debian (Stretch) package(s) is a bad idea: Broken Distribution Packages

Thanks. I am aware of that, and I do not use it as a regular tool. However, in this issue we are testing some common setups. If any user installs WSL Debian, this is the behaviour they will get. I do not remember installing xpra explicitly, and I didn't select it through a CLI option when running runx/x11docker. Hence, a safe solution would be runx/x11docker not to default to xpra unless the version is a above a minimum. Actually, on Cygwin xpra is not installed, and the default works.

[mviereck]

The WSL I am using is Debian Stretch:

*1 It fails because of xpra, as commented above.

I've installed debian stretch with same xpra version in WSL.
I've improved the check for valid xpra options. It works here now, e.g.

runx -- x11docker --xpra --exe xterm

I think that the random number is not a devil at all. There are some other docker features that are mapped randomly, so it is coherent. As said, I'm just worried if there is some possibility of conflict.

It will conflict is the random display number is already in use. If you run to instances of runx, the second one has a chance of about 0.3% to fail. A third instance would have a change of about 0.6% to fail.

I would expect to be the other way: x11docker should create another X server if it cannot use the one provided by runx. Actually, it can be avoided with runx [...] -- x11docker --hostdisplay [...]. It might be worth adding a note about this somewhere.

*2 It works, but two X servers are started, as commented above. However, it can be avoided with runx -- x11docker --hostdisplay [...], which works ok too and uses a single X server.

A core concept of x11docker is to run additional X servers. --hostdisplay is just a fallback. x11docker is not aware of runx at all.
The wiki will document the differences. I'll update it after publishing the current master as a release.

E.g. 'xauth' does exist, I believe, but it is not used because runx sets --no-auth

MSYS2 does not provide an xauth package. runx automatically sets --no-auth to be comfortable; however, instead it could just request the user to set --no-auth.

Also, the note about using xpra, Xephyr, etc. does apply to MSYS + vcxsrv?

MSYS2 does not provide any of xpra, nxagent etc. So it does not apply to MSYS2, but to Cygwin and WSL. I would not like to suppress the messages with an MSYS2 specific check. I want to keep x11docker as general as possible and to reduce system specific checks.

---

I'd say, if xpra works now in Debian Stretch-WSL, everything is fixed now, and runx and x11docker behave as expected.

[eine]

I've installed debian stretch with same xpra version in WSL.
I've improved the check for valid xpra options. It works here now, e.g.

runx -- x11docker --xpra --exe xterm

I will try.

It will conflict is the random display number is already in use. If you run to instances of runx, the second one has a chance of about 0.3% to fail. A third instance would have a change of about 0.6% to fail.

Well, it's low enough. Is there any comment about this in the docs?

A core concept of x11docker is to run additional X servers. --hostdisplay is just a fallback. x11docker is not aware of runx at all.
The wiki will document the differences. I'll update it after publishing the current master as a release.

The point is: what do the additional servers created by x11docker provide to a context created by runx? Using --hostdiplay seems sensible, as a user would expect runx to take care of some issues (related to the X server), and x11docker of others (setting up and starting the docker container). However, at the moment it seems pointless to use runx & x11docker in cygwin, if the same result can be achieved by running x11docker only (and we avoid the apparently 'dummy' server created by runx). This specially so, because the behaviour is completely different from WSL or MSYS, where x11docker does not start an additional X server.

MSYS2 does not provide an xauth package. runx automatically sets --no-auth to be comfortable; however, instead it could just request the user to set --no-auth.

My bad. xauth is provided by vcxsrv, not by MSYS2.

Nonetheless, my comment was related to x11docker checking the authentication at all, if runx was started without authentication. Does the user need to provide --no-auth twice? I.e. runx --no-auth -- x11docker --no-auth [...]?

Also, the note about using xpra, Xephyr, etc. does apply to MSYS + vcxsrv?

MSYS2 does not provide any of xpra, nxagent etc. So it does not apply to MSYS2, but to Cygwin and WSL. I would not like to suppress the messages with an MSYS2 specific check. I want to keep x11docker as general as possible and to reduce system specific checks.

I think this can be 'fixed' without any additional logic. Just add some word to the comment. E.g., instead of To allow protection against X security leaks, please install one or more of: To allow protection against X security leaks, please check if one or more of the following tools is available on your platform and install it:. You can also add some link to a page in the wiki where users can see which tools are expected to be available on each platform.

[mviereck]

Well, it's low enough. Is there any comment about this in the docs?

This runx note:

runx note: Using random X display number :784.
  If this display number is already in use, X server startup  will fail.
  You can specify a display number N with option '--display N'.

The point is: what do the additional servers created by x11docker provide to a context created by runx?

In general, running additional X servers avoids some X security leaks. This is much more important on native Linux systems than on MS Windows. runx -- x11docker --hostdisplay would not allow container applications to access the host as x11docker --hostdisplay would on native Linux.
However, x11docker does not know how the host X server accessed with --hostdisplay is set up. The X server can be from runx, from a custom user script or from a possible future WSL-X. x11docker cannot assess the amount of a security risks by using the host X server.

Example: One could run a desktop with runx --desktop -- mate-session. Inside this desktop one runs x11docker --hostdisplay. The container application can access the mate desktop and through it the entire Windows file system.

This specially so, because the behaviour is completely different from WSL or MSYS, where x11docker does not start an additional X server.

It is only MSYS2 where x11docker cannot run an additional X server. If available, x11docker will run XWin, xpra, nxagent or Xephyr in WSL or Cygwin.

Nonetheless, my comment was related to x11docker checking the authentication at all, if runx was started without authentication. Does the user need to provide --no-auth twice? I.e. runx --no-auth -- x11docker --no-auth [...]?

x11docker automatically falls back to --no-auth and shows a warning.
runx only falls back to --no-auth in MSYS2. In WSL and Cygwin it terminates with an error message.
That being said, x11docker should not fall back, if I think about it. On a native Linux system it is ok: The X server is connected over a unix socket, and attackers would access the container at most if they take over the X server, and would need to have access to the machine already.
But on Windows the X servers are accessed over an open TCP connection, and a network attack is possible. That makes it important to have authentication cookies.

I think this can be 'fixed' without any additional logic. Just add some word to the comment. E.g., instead of To allow protection against X security leaks, please install one or more of: To allow protection against X security leaks, please check if one or more of the following tools is available on your platform and install it:. You can also add some link to a page in the wiki where users can see which tools are expected to be available on each platform.

Good idea, I'll think about it.

[eine]

I now tried runx -- x11docker --xpra --exe xterm in WSL, and it works ok. However, neither runx -- x11docker --xpra x11docker/check nor runx -- x11docker x11docker/check work.

The X server can be from runx, from a custom user script or from a possible future WSL-X. x11docker cannot assess the amount of a security risks by using the host X server.

IMHO, runx should set some specific environment variable or write some config file, so that x11docker can be aware if the X server was started with runx.

Example: One could run a desktop with runx --desktop -- mate-session. Inside this desktop one runs x11docker --hostdisplay. The container application can access the mate desktop and through it the entire Windows file system.

In this context (Cygwin), what's the difference between doing so or running x11docker --desktop directly? Better said, why would a user want to use runx?

It is only MSYS2 where x11docker cannot run an additional X server. If available, x11docker will run XWin, xpra, nxagent or Xephyr in WSL or Cygwin.

XWin is not supported in WSL, nxagent or Xephyr are not installed, and the version of xpra fails. That's probably why I assumed that no additional X server is created in WSL, i.e., because the only working solution ATM is --hostdisplay.

[mviereck]

IMHO, runx should set some specific environment variable or write some config file, so that x11docker can be aware if the X server was started with runx.

x11docker still would not know if other applications are running on the runx X server, e.g. a desktop or a file manager, that could be accessed from container applications.
x11docker cannot rely on any X setup it did not set up itself. Not even on the setup of runx because the user could use it for other purposes at the same time.

In this context (Cygwin), what's the difference between doing so or running x11docker --desktop directly? Better said, why would a user want to use runx?

There is no point to use runx in Cygwin for x11docker (except XWin is not installed but VcXsrv is).
Other/real use cases for runx in Cygwin are native Cygwin X applications / desktops without using or having x11docker at all.

However, neither runx -- x11docker --xpra x11docker/check nor runx -- x11docker x11docker/check work.

I found a bug, the unix socket hasn't been shared. Is fixed now.
However, I am not entirely sure if sharing the unix socket from WSL will work, I just hope so. It would be more performant and more secure than a TCP connection.

If any user installs WSL Debian, this is the behaviour they will get. I do not remember installing xpra explicitly

xpra is not part of Debian by default. You've probably installed it for one of #125 #123 #122 #121 #118 #117 where we looked how to provide Xvfb for xpra on Windows.

[mviereck]

@1138-4eb Could you please run a final check if xpra works on WSL?

runx -- x11docker --xpra x11docker/check

[eine]

@1138-4eb Could you please run a final check if xpra works on WSL?

runx -- x11docker --xpra x11docker/check

It does not:

runx note: Using random X display number :748.
  If this display number is already in use, X server startup  will fail.
  You can specify a display number N with option '--display N'.

runx note: Windows firewall settings can forbid application access
  to the X server. If no application window appears, but no obvious error
  is shown, please check your firewall settings.
  Compare:   https://github.com/mviereck/x11docker/issues/108

runx note: If you get application error messages like 'Cannot open display'
  or 'Invalid MIT-MAGIC-COOKIE', the X authentication cookie might be broken.
  You can remove old cookies and stop running X servers with: runx --cleanup

DISPLAY=10.0.75.1:748 XAUTHORITY=/home/eine/runx_Xauthority
x11docker note: Your xpra version 'xpra v0.17.6' is out of date. It is
  recommended to install at least xpra v1.0. Look at:  www.xpra.org

x11docker note: Xpra startup is rather slow. For faster startup
  with seamless applications,   try --nxagent.
  If security is not a concern, try --hostdisplay.

x11docker WARNING: Request of Windows path to path within WSL:
  /tmp/.X11-unix/X100
  Write access from Windows host to WSL files can damage the WSL file system.
  Read-only access is ok.
  Option --share: You can add :ro to the path to allow read-only access.
  Example: --share='/tmp/.X11-unix/X100:ro'

x11docker WARNING: Request of Windows path to path within WSL:
  /tmp/.X11-unix/X100
  Write access from Windows host to WSL files can damage the WSL file system.
  Read-only access is ok.
  Option --share: You can add :ro to the path to allow read-only access.
  Example: --share='/tmp/.X11-unix/X100:ro'

x11docker note: Stay tuned, xpra will start soon.

SUCCESS: The process with PID 97036 has been terminated.

x11docker.log

[mviereck]

Thank you for the test!

It does not:

Surprisingly, it look like xpra starts well but the container stops immediatly for no obvious reason.
Or x11docker's container check fails.
I don't see any message from kaptain, neither success nor failure.

Does --hostdisplay really work?

runx -- x11docker --hostdisplay x11docker/check

Maybe check with another image, e.g.

runx -- x11docker --xpra x11docker/xfce xfce4-terminal

[eine]

I don't see any message from kaptain, neither success nor failure.

I'm not using kaptain. If you mean the kaptain inside the container, I think it is not even started.

Does --hostdisplay really work?

runx -- x11docker --hostdisplay x11docker/check

Yes, it works. However, an error is shown when the window is closed:

$ /mnt/d/data-dev/github/runx/runx -- /mnt/d/data-dev/github/x11docker/x11docker --hostdisplay x11docker/check
runx note: Using random X display number :2471.
  If this display number is already in use, X server startup  will fail.
  You can specify a display number N with option '--display N'.

runx note: Windows firewall settings can forbid application access
  to the X server. If no application window appears, but no obvious error
  is shown, please check your firewall settings.
  Compare:   https://github.com/mviereck/x11docker/issues/108

runx note: If you get application error messages like 'Cannot open display'
  or 'Invalid MIT-MAGIC-COOKIE', the X authentication cookie might be broken.
  You can remove old cookies and stop running X servers with: runx --cleanup

DISPLAY=10.0.75.1:2471 XAUTHORITY=/home/eine/runx_Xauthority
x11docker WARNING: Option --hostdisplay allows less security hardening.
  It is recommended to use another X server option like --xpra or --nxagent.

Error: No such object: f4462090a785fbd3c1dfe8cec2f0c5096c4a25060ddf4aff7a4e15aa94ec744f
SUCCESS: The process with PID 21432 has been terminated.

Maybe check with another image, e.g.

runx -- x11docker --xpra x11docker/xfce xfce4-terminal

$ /mnt/d/data-dev/github/runx/runx -- /mnt/d/data-dev/github/x11docker/x11docker --xpra x11docker/xfce xfce4-terminal
runx note: Using random X display number :1221.
  If this display number is already in use, X server startup  will fail.
  You can specify a display number N with option '--display N'.

runx note: Windows firewall settings can forbid application access
  to the X server. If no application window appears, but no obvious error
  is shown, please check your firewall settings.
  Compare:   https://github.com/mviereck/x11docker/issues/108

runx note: If you get application error messages like 'Cannot open display'
  or 'Invalid MIT-MAGIC-COOKIE', the X authentication cookie might be broken.
  You can remove old cookies and stop running X servers with: runx --cleanup

DISPLAY=10.0.75.1:1221 XAUTHORITY=/home/eine/runx_Xauthority
x11docker note: Your xpra version 'xpra v0.17.6' is out of date. It is
  recommended to install at least xpra v1.0. Look at:  www.xpra.org

x11docker WARNING: Request of Windows path to path within WSL:
  /tmp/.X11-unix/X100
  Write access from Windows host to WSL files can damage the WSL file system.
  Read-only access is ok.
  Option --share: You can add :ro to the path to allow read-only access.
  Example: --share='/tmp/.X11-unix/X100:ro'

x11docker WARNING: Request of Windows path to path within WSL:
  /tmp/.X11-unix/X100
  Write access from Windows host to WSL files can damage the WSL file system.
  Read-only access is ok.
  Option --share: You can add :ro to the path to allow read-only access.
  Example: --share='/tmp/.X11-unix/X100:ro'

Error: No such object: 9158d9c7786b559b63a1a8a0ad77e76ab81adc3184c6a8066fd1063c9efeffbb
SUCCESS: The process with PID 41408 has been terminated.

x11docker.log

[mviereck]

Thank you for the tests and the log.

It is confusing that it shows no obvious error. Xpra seems to run well.
The container is started, some initialization is shown, but than it immediatly stops at the point where the X application should start.

A guess: Maybe the container entirely crashes without a message when the X application tries to access the shared unix socket.

The latest commit sets up an X over IP connection in WSL like runx does for VcXsrv and XWin. Could you please try again?

I'm not using kaptain. If you mean the kaptain inside the container, I think it is not even started.

Yes, I mean kaptain in x11docker/check.

[mviereck]

The latest commit sets up an X over IP connection in WSL like runx does for VcXsrv and XWin.

@1138-4eb Could you please run a check in WSL with --xpra?

runx -- x11docker --xpra x11docker/check

I hope it works now with X over IP.

---

Edit: Meanwhile I've changed runix behaviour on MSYS2: It fails without --no-auth. I think that this behaviour is more consistent. I hope it is not too inconvenient for you.
For your use case I'd assume that an entry of runx in ~/bashrc would fit your needs best.

[eine]

I've been side-tracked, and I'm trying to test the latest updates (x11docker 0470946 and runx 8882b5b6).

runx -- pcmanfm on WSL works ok. However, it modifies the font of the WSL terminal.

Unfortunately, runx and x11docker seem not to work together:

In WSL, the server and the container are started, but no window is shown. The firewall is disabled, just in case. /mnt/t/runx/runx -- /mnt/t/x11docker/x11docker --hostdisplay x11docker/xfce xfce4-terminal: x11docker.log. Same result with x11docker/check: x11docker.log. Or using --xpra: x11docker.log.

In MSYS2, the X server is started, but the container is not. When terminating the execution with Ctrl+C, the logfile is empty:

# /t/runx/runx --no-auth -- /t/x11docker/x11docker --debug --no-auth --hostdisplay x11docker/checkrunx note: Using random X display number :2834.
  If this display number is already in use, X server startup  will fail.
  You can specify a display number N with option '--display N'.

runx note: Windows firewall settings can forbid application access
  to the X server. If no application window appears, but no obvious error
  is shown, please check your firewall settings.
  Compare:   https://github.com/mviereck/x11docker/issues/108

runx WARNING: Option --no-auth: Cookie authentication is disabled!
      SECURITY RISC!
  Your X server vcxsrv listens on TCP connections without any protection.
  Others could try to access your system through network connections.
  Please use option --no-auth for debugging only.

DISPLAY=10.0.75.1:2834
DEBUGNOTE[17:03:04,603]: check_parent_sshd(): Failed to check for sshd. ps -p not supported.
DEBUGNOTE[17:03:04,769]: check_host(): ps can watch root processes: yes
DEBUGNOTE[17:03:07,228]: storeinfo(): cache=/home/eine/.cache/x11docker/x11docker-check-51383732488
DEBUGNOTE[17:03:07,336]: storeinfo(): stdout=/home/eine/.cache/x11docker/x11docker-check-51383732488/share/stdout
DEBUGNOTE[17:03:07,444]: storeinfo(): stderr=/home/eine/.cache/x11docker/x11docker-check-51383732488/share/stderr
DEBUGNOTE[17:03:07,649]: storeinfo(): x11dockerpid=7759
DEBUGNOTE[17:03:07,913]:
x11docker version: 6.3.1-beta
docker version:    Docker version 19.03.4, build 9013bf5
Host system:
Command:           '/t/x11docker/x11docker' '--debug' '--no-auth' '--hostdisplay' 'x11docker/check'
Parsed options:     --debug --no-auth --hostdisplay -- 'x11docker/check'
DEBUGNOTE[17:03:07,976]: Dependency check for --hostdisplay: 0
DEBUGNOTE[17:03:08,040]: Dependencies of --hostdisplay already checked: 0
DEBUGNOTE[17:03:08,104]: Dependencies of --hostdisplay already checked: 0
DEBUGNOTE[17:03:08,167]: storeinfo(): xserver=--hostdisplay
x11docker WARNING: Option --hostdisplay allows less security hardening.
  It is recommended to use another X server option like --xpra or --nxagent.

DEBUGNOTE[17:05:56,971]: Received SIGINT
DEBUGNOTE[17:05:57,028]: storeinfo(): error=130
DEBUGNOTE[17:05:57,124]: Terminating x11docker.
DEBUGNOTE[17:05:57,180]: time to say goodbye (finish)
DEBUGNOTE[17:05:57,447]: x11docker exit code: 130

Fortunately, I can just source runx and pass DISPLAY to the containers manually:

# source /t/runx/runx --no-auth
runx note: Using random X display number :2043.
  If this display number is already in use, X server startup  will fail.
  You can specify a display number N with option '--display N'.

runx WARNING: Option --no-auth: Cookie authentication is disabled!
      SECURITY RISC!
  Your X server vcxsrv listens on TCP connections without any protection.
  Others could try to access your system through network connections.
  Please use option --no-auth for debugging only.

DISPLAY=10.0.75.1:2043

# echo $DISPLAY
10.0.75.1:2043

# docker run --rm -e DISPLAY=$DISPLAY x11docker/check

This is undesired, tho, because I cannot use x11docker's security features and options.

Meanwhile I've changed runix behaviour on MSYS2: It fails without --no-auth. I think that this behaviour is more consistent. I hope it is not too inconvenient for you.

I'd prefer if providing it to runx was enough. I.e., if runx set some envvar to tell x11docker that the X server open (e.g. RUNX_NO_AUTH). Currently, --no-auth needs to be provided twice. Anyway, I'm good as long as I can find a setup that works with runx and x11docker, even if it's verbose.

[mviereck]

Thank you for testing and giving new feedback!

WSL:
x11docker tried to share a folder from the subsystem. This lead to the following shared volume that does not seem to work properly:

  --volume '/c/Users/eine/AppData/Local/Packages/TheDebianProject.DebianGNULinux_76v4gfsz19hv4/LocalState/rootfs/home/eine/.cache/x11docker/x11docker-xfce-xfce4-terminal-49245618108/share':'/x11docker':rw \

The log shows this error although the file exists:

/tmp/containerrootrc: 11: /tmp/containerrootrc: cannot create /x11docker/container.log: Directory nonexistent

I've changed x11docker accordingly, it will now use the path
cmd.exe /C "echo %userprofile%" plus /x11docker/cache as base cache folder. (Can be changed with --cachebasedir.) That should fix the bug.

---

MSYS2:
The fix above does not apply to MSYS2 setups.

I'd prefer if providing it to runx was enough.

That should be enough already. x11docker will drop a warning if the X server runs without a cookie, but should continue. x11docker does not need an additional --no-auth.

In MSYS2, the X server is started, but the container is not. When terminating the execution with Ctrl+C, the logfile is empty:

That is odd. Could you please try with --verbose? I need to find the point where it exactly hangs. It might even help to make bash itself verbose with set -x to find the code line where execution stops. (set +x disables bash verbose mode.)

---

Cygwin:
Recently some issues in Cygwin/X has been fixed in #187 . It runs well using Xwin without runx at all.

[eine]

I've changed x11docker accordingly, it will now use the path
cmd.exe /C "echo %userprofile%" plus /x11docker/cache as base cache folder. (Can be changed with --cachebasedir.) That should fix the bug.

It seems to hit some other issue now: x11docker.log

That should be enough already. x11docker will drop a warning if the X server runs without a cookie, but should continue. x11docker does not need an additional --no-auth.

My bad. Since it fails just after printing the warning, I thought that it was an error. You are correct.

That is odd. Could you please try with --verbose? I need to find the point where it exactly hangs. It might even help to make bash itself verbose with set -x to find the code line where execution stops. (set +x disables bash verbose mode.)

With --verbose there is no difference at all. With set -x in x11docker: x11docker.log.

Cygwin:
Recently some issues in Cygwin/X has been fixed in #187 . It runs well using Xwin without runx at all.

I will try after ensuring that WSL and MSYS2 work as expected.

[mviereck]

In both cases the cache base folder seems to be the critical point.
You could try --cachebasedir DIR to verify this.

The WSL log shows the long path again although it should not:

  --volume '/c/Users/eine/AppData/Local/Packages/TheDebianProject.DebianGNULinux_76v4gfsz19hv4/LocalState/rootfs/home/eine/.cache/x11docker/x11docker-check-50385568108/share':'/x11docker':rw \

It looks a bit like the code was not updated.
I found that downloading code from github with curl or wget sometimes does not give the latest code. There seems to be a delay until it is updated after a commit. Maybe that has happened here.
I have changed the version number from 6.3.1-beta to 6.3.2-beta. Please have a look at the version number in your next check.

---

In the set -x log I find these lines before SIGINT:

+ case $Winsubsystem in
++ rmcr
+++ cmd.exe /C 'echo %userprofile%'
++ case "${1:-}" in
+++ printf '\r'
++ sed $'s/\r//g'
+ Cachebasefolder=/x11docker/cache
++ finish_sigint

It looks like cmd.exe /C 'echo %userprofile%' does not give a valid output. Could you please check this command directly?
However, I yet changed x11docker to use this in WSL only.

[eine]

It looks a bit like the code was not updated.
I found that downloading code from github with curl or wget sometimes does not give the latest code. There seems to be a delay until it is updated after a commit. Maybe that has happened here.
I have changed the version number from 6.3.1-beta to 6.3.2-beta. Please have a look at the version number in your next check.

I'm downloading both x11docker and runx through git. I pulled the latest commit, but it is still not working: x11docker.log.

It looks like cmd.exe /C 'echo %userprofile%' does not give a valid output. Could you please check this command directly?

Executing this in MSYS2 returns an interactive cmd. I think that /C needs to be escaped:

# cmd.exe /C 'echo %userprofile%'
Microsoft Windows [Version 10.0.18362.418]
(c) 2019 Microsoft Corporation. All rights reserved.

D:\data-dev\aptman\dbhi-hub\vunit>exit
exit

# cmd.exe //C 'echo %userprofile%'
C:\Users\eine

[mviereck]

Executing this in MSYS2 returns an interactive cmd. I think that /C needs to be escaped:

Oh, yes, I forgot about the path conversion of MSYS2. This code works now (tested in VM). MSYS2_ARG_CONV_EXCL='*' disables the path conversion:

rmcr() {                        # remove carriage return to translate DOS/Windows newlines into UNIX newlines
  # convert stdin if $1 is empty. Otherwise convert file $1.
  case "${1:-}" in
    "") sed    "s/$(printf "\r")//g" ;;
    *)  sed -i "s/$(printf "\r")//g"  "${1:-}" ;;
  esac
}
wincmd() {                      # execute a command on MS Windows with cmd.exe
  MSYS2_ARG_CONV_EXCL='*' cmd.exe /C "${@//&/^&}" 2>&1 | rmcr
}
wincmd 'echo %userprofile%'

---

I'm downloading both x11docker and runx through git. I pulled the latest commit, but it is still not working: x11docker.log.

Hm. Now the log shows the desired pathes for running in WSL. The three different pathes are right:

• /x11docker in container
• /mnt/c for xauth in WSL
• /c for docker.exe

  --volume '/c/Users/eine/x11docker/cache/x11docker-check-70521955858/share':'/x11docker':rw \

DISPLAY and XAUTHORITY look valid:

  --env 'XAUTHORITY=/x11docker/Xauthority.client' \
  --env 'DISPLAY=10.0.75.1:742' \

[...]

-rwxrwxrwx 1 eine eine 62 Oct 30 22:22 /mnt/c/Users/eine/x11docker/cache/x11docker-check-70521955858/share/Xauthority.client

[...]

DEBUGNOTE[22:22:11,645]: xinitrc: Created cookie: #ffff#4445534b544f502d45314552343348#:742  MIT-MAGIC-COOKIE-1  317059d0b3e6f72d4926bb8549cb171a

But the GUI fails with an X connection error:

kaptain: cannot connect to X server 10.0.75.1:742

It is probably not a cookie issue, that normally shows something like No protocol specified. I'd rather suspect a firewall issue, but you surely had that in mind already. Nonetheless, please try without a cookie:

runx --no-auth -- x11docker --hostdisplay x11docker/check

I currently have no real idea what is going wrong, as everything in the setup looks right now.
My own tests in WSL without containers do work:

runx -- x11docker --hostdisplay --exe xterm

Please try also runx --cleanup to remove possible old invalid cookies. Should not be needed, but who knows.

[mviereck]

Just a thought: You run runx without specifying the X server. It defaults to XWin if available.
It might be worth to try with VcXsrv instead.
Please run first:

runx --cleanup

Than:

runx --vcxsrv --no-auth -- x11docker --hostdisplay x11docker/check

And compare with:

runx --xwin --no-auth -- x11docker --hostdisplay x11docker/check

[eine]

I don't know what happened (I run --cleanup and updated x11docker), but the following four options are working ok on WSL now:

runx --no-auth -- x11docker --hostdisplay x11docker/check
runx -- x11docker --hostdisplay x11docker/check
runx --vcxsrv --no-auth -- x11docker --hostdisplay x11docker/check
runx --vcxsrv -- x11docker --hostdisplay x11docker/check

And these work on MSYS:

runx --no-auth -- x11docker --hostdisplay x11docker/check
runx --vcxsrv --no-auth -- x11docker --hostdisplay x11docker/check

I tested Cygwin too, and x11docker works ok without runx.

Hence, I guess that this issue can be closed for now.

[mviereck]

I don't know what happened (I run --cleanup and updated x11docker), but the following four options are working ok on WSL now:

Great! What a relief.
runx --cleanup might have been the key. I found that the cookie file is corrupted sometimes for unknown reasons.
For that reason runx always shows the message:

runx note: If you get application error messages like 'Cannot open display'
  or 'Invalid MIT-MAGIC-COOKIE', the X authentication cookie might be broken.
  You can remove old cookies and stop running X servers with: runx --cleanup

Thank you for testing and reporting!