-
Notifications
You must be signed in to change notification settings - Fork 380
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature Request: Porting x11docker to a programming language #353
Comments
Also I saw using x11docker, a guy made dx11. Which is just one more layer of abstraction. With programming language's extensibility, we can add multiple layers of abstraction, to a single codebase, instead of making a seperate project. |
I am open to this. I gave x11docker a structure that should accord to good practice. So it should be possible to port it to another language. Currently I am looking for a new language to learn, maybe C++. The reason behind is that I need a GUI for another project (microscopy-tools) that provides some image editing features. One possible way: If one tries to rewrite x11docker in C++, I would assist wherever I can with experience of the setup and minor but important details. While this goes on, I learn C++ looking at the new code.
The core dependencies are an X server, some X utils, Docker and linux coreutils. Other dependencies depend on desired features like pulseaudio support. Another language would not really decrease the amount of dependencies. Including the X utils or pulseaudio as libraries just hides the dependencies, but does not decrease them. x11docker itself rarely changes nowadays. I consider it to be stable. If you look at the code frequency, you'll see that only minor changes where done the last 18 month. |
And they are a lot of things, weston, Xorg, Xephyr, pulseaudio, diferrent environments(gnome, xfce, etc) + (none of them is what we find a lot of documentation for, neither good amount of discussions). And at all the time we want to add something or another to all these tools, tinkering around with that or something, it breaks one or two things manytimes, I have witnessed that myself.
Its about adding even more. Though linux is stable, did the commit race stopped? Lets take it to heights. For ex, I wanted to make multiseat setup possible with x11docker, such that u will not need docking device, which costs a lot, if u wanted to open a computer lab for African villagers. Right now, I have to make a seperate project, but If we first port it to a language, then we can just extend it to provide modularity at a level, that people like me or dx11 can just add-in as a feature.
For languages, I love cpp from my heart for its truthfulness to the kernel, but because every major application level project I see is in golang, hence i have invested much in Golang. Docker and everything new is going go. Also a heck lot of community there for go. |
If something comes up, you can just ask for it. A lot of x11docker code consists of checking possible interferences, avoiding them and giving messages about it. So yes, changes have to be done carefully and well checked.
I am not sure about this. What could be modularized?
One big part is the X server setup because several different X servers are supported where each has its own advantages and disadvantages. I thought about separating this part into another script, just did not do the work without seeing an obvious advantage. Another complex part is init support in containers; maybe I should not have included it, because it is lot of code for less needed setups. Though, some desktop environments and even some applications depend on it. x11docker codebase could be shrinked a lot if:
I am looking at C++ because it has direct support by imagemagick, a tool that I heavily use in my current project where I need a GUI in future. I had a look at Golang; there is only an unmaintained wrapper to imagemagick, not sure if I could use the imagemagick C++ libs directly without it. I also had a look at python, but the imagemagick support is unmaintained, too. |
Yeah u are right in all this points. Static linking increases size,but giving hundred percent guarantee that it will run in any environment, any distro, without saying. "some.so.3 not found".
About cpp:
So now, whatever u choose, cpp or go, we should lets GO! |
Let me add some more,
Believe me, multiseat with docker is going to be the best use of it, it will reduce the costs of computer labs all around the world. Yeah!! |
Please don't hurry me. :-)
Currently I have the impression that C++ is the best choice here.
Good to know, thanks! Though, as said above, direct support of imagemagick developers is a critical point, so I won't choose golang for me.
At this low base there is a short example in the wiki: https://github.com/mviereck/x11docker/wiki/Short-setups-to-provide-X-display-to-container#extended-xephyr-script
Providing GPU support already requires multiple X server support. None of them fulfills all desirable setups. Compare the attributes of X servers: https://github.com/mviereck/x11docker/wiki/X-server-and-Wayland-Options#attributes-of-x-server-and-wayland-options
Don't consider dx11 too much. It is just a shortcut wrapper for x11docker commands. At its current stage it looks rather unpolished. I dislike its approach to drop security features by default just for ease of use. Though, I am honored by the developer's excitement of x11docker, and his advertisement gave x11docker some more popularity.
This point sounds most interesting to me.
|
Multiseat allow one running linux, to be shared among multiple users. Each user has a set of keyboard, mouse and monitor, that is exclusively for them. This is the most cost effective solution, for setting up a computer lab. As one single tower(powerful cpu) can be used.
Currently they are just used in cybercafes or customer-care centers,because of these short comings. |
I like to support this. To encircle your needs (please extend/correct):
Proposal:
|
thanks mviereck for getting in. Our proposal:
I will take 2 months to make space for Cpp as I am currently indulged in GO, dart and python3 based projects.
Yeah!!
I just need Xephyr to run both on wayland and X11. We can think of keeping others also as an extension,not part of core codebase, don't know if community utilize them. Encircling our needs:
As mutiseat is just a layer of abstraction over x11docker, x11docker will just have both (desktop+seamless), multiseat will just hide seamless application mode.
Latest version Xephyr 20 is lacking evdev support, but version 19 had it, which I have installed currently. Currently I am experimenting Xephyr with one other way. If that doesn't work, either we keep a binary of Xephyr in code or keep updating Xephyr with the patch and make it locally, some people have done that.
It depends on you, as it depends on the core x11docker. I have little interest in gpu, as in docker computation for gpu might be just for playing games and videos.
People generally need less memory footprint for their docker based desktop environment, atleast me. But I really don't know abour all the people out their using x11docker out their with gnome and waiting for next big release.
Hey thats a big one. Currently its just like as if running an OS from live CD boot. Persistance of my work is atmost importance to me, even in general. There can be many ways for persisting state, I was thinking of giving user choice for what method he wants, depending on his need. We need a .ini based file or .json/yaml file to load config from, also could be overwritten from command line. This will give user more managibility. |
Currently persistance for user files is possible with options You can design multiseat-x11docker to reuse containers. That would nail down the setup to X over TCP. That in return reduces the choice of possible X servers. Noteable, that disallows Xwayland setups that are needed for multiple X servers with GPU access. So we have an odd choice: We can't have GPU acceleration along with persistant containers. (Possible untestet way: If each user has its own monitor plugged into its own graphic card, all containers could run with Xorg that supports both X over tcp and GPU acceleration.)
We need to get clear about the desired setups, because no X server supports all possibly desired setups.
Open questions:
This doesn't depend on me, but on your target. :-) The more choices you provide, the more work you'll have. Rather have a concept design that is slim but fits most of the targeted multiseat users.
It seems to be a bug and not an intended deprecation. Hopefully it will be fixed. In general it is no good idea to keep old versions of anything alive. Their problems grow with their age. |
GPU is not a priority, if it comes along with rest of things then ok, otherwise I wouldn't crave for it.
I am stuck with this actually. They intentionally removed support for evdev. Let me see nxagent. Atleast, for me, its not an issue till yet. But as ubuntu 22 comes, it might break. Hence, I will not hurry on a solution for this. Meanwhile closig this issue, as I will take 1-2 months to learn Cpp along with all other tasks, will continue after that. |
IMHO, using a compiled language for this project does not make much sense. There is no intense computing which would benefit from the advantages that either C++, golang or Rust might provide. However, the need to compile the tool for each target architecture and OS is a huge maintenance burden which is not justified except by your excitement to learn a new language (for other purposes). Therefore, I would strongly discourage using any of those. Do not take me wrong. I like, know and use golang; I've heard very good things about Rust, and C++ is a very used language in lots of tools. However, a relevant aspect of engineering is picking the appropriate tool for a task. x11docker and runx are scripting projects. It's sensible to consider Python, Perl, TCL or even JS/TS, as any of those would provide some better data abstraction. Yet, a compiled language is overkill. I would personally suggest Python if you really wanted to rewrite this tool. Python is a very interesting language, because it's an scripting/interpreted language with first class support for C/C++ under the hood. Therefore, it allows using scripting only, and then switching to shared libraries written in C/C++ for better performance requirements. For didactic purposes, this is a very interesting project about wrapping a golang tool in Python: https://github.com/asottile/dockerfile. |
@eine python is not statically typed language which makes debugging bad.
However I totally agree with your views and believer of same. Go can be used in interpreted mode using "go run", hence making it platform independent too. Also I do favor, it should not be the language we have learnt, but the language project needs. This has been an important factor in widespreading of projects in community.
I think u mean any interpreted language. |
Also @mviereck , about evdev, this is the commit they removed it from xserver. https://gitlab.freedesktop.org/xorg/xserver/-/commit/27819950e4158326e0f83a30f2e8968b932625ef, If I add that code back and compile with master, I believe it will work. However, I am willing to debug if any issues arise. Just created an issue also there. https://gitlab.freedesktop.org/xorg/xserver/-/issues/1174 |
That's a good point, thank you!
Thanks for the link. However, it is still not clear to me if that only affects Xephyr or the entire X11 base.
Some basic thoughts:
Conclusions:
Proposal:
The code base could be ready within a week. |
Currently, I just change Xephyr version, to make it work.
I think you mean it is too huge to transfer. If yes, then no problem there. I think I can do it in few months.
My company wants to use it for general debug environment, in place of a vm. Hence all the features I wish to port, all of them. Multiseat is just a social help from me.
Yeah let me get started, and write the https://github.com/mviereck/x11docker/wiki/Short-setups-to-provide-X-display-to-container#extended-xephyr-script . I have worked on audio, but webcam might take a bit of time. Yeah I get start in 3 days, after killing my current leftover tasks. Thanks |
@velcrine, that is a tradeoff. You decide to have static typing by default or add it afterwards. By the same token, you decide to have generic interfaces or you add them later.
Therefore, from a robustness point of view, both Python and Golang do allow very similar use cases, albeit with very different learning/effort curves. The Python learning curve is exponential and Golang's is inverse exponential.
That is not correct. Golang has a blazing fast compiler. Therefore, you can execute Conversely, in Python execution is interpreted by default. However, it has some internal heuristics and it compiles the pieces of the software that are used/executed frequently. That is done transparently to the user.
Honestly, I believe that the line between Python and Golang is very blurry in this case. If x11docker was to be written from scratch, I would suggest Python because of the faster iteration. However, since x11docker is written already (and the codebase itself is pretty well structured), it might be feasible to go for golang straightaway. Personally, I will be good with supporting and helping if you pick any of those. My concern was that I perceived you were "convinced" about C++; that's what I find a no-go. Overall, there is a very relevant topic that went unnoticed in your dialogue above: Docker is written in golang, and therefore, their libraries/modules can be used in golang projects! Any other language needs to duplicate the data structures and API, but using golang avoids all of that. @velcrine did comment it, but the attention was not focused longer. Very precisely, I do have a tool written in golang which uses the docker library for creating and running containers:
See how are flags, args and config options declared: https://github.com/eine/issue-runner/blob/master/tool/main.go. See where are the docker libs used for creating, pulling, removing containers: https://github.com/eine/issue-runner/blob/master/tool/docker.go. Find the official examples in https://docs.docker.com/engine/api/sdk/examples/. Therefore, as you might imagine, I'm absolutely ok if you/we decide to use golang. Nevertheless, you saw that official bindings and examples for Python are available too. Therefore, from the point of view of mviereck, who would need to learn Python or golang for the purpose of this project only, I cannot make an strong defense of golang. I honestly think he needs to evaluate both languages, and we need to help him decide which one he feels more comfortable with. The value of this project is the knowledge in his mind, we need to make it feasible to translate that into a different language, without the language being a burden for him. On the other hand, if x11docker itself is not going to be rewritten, the choice is irrelevant, because reusing data structures won't be possible. With regard to the side discussion, I believe that you don't want to use neither Python nor golang for serious GUI applications. None of those languages were designed for that purpose. Python is bash on steroids. A lot of projects do provide Python bindings (ScyPy, ONXX, Blender...). It's for scripting and interacting with optimised tools or shared libraries. You can build standalone GUIs with Python (e.g. https://www.youtube.com/watch?v=QWqxRchawZY or https://www.pygame.org), but it's not the best language for that purpose. On the other hand, golang is a "system language", so it's neither designed for GUI tools. Certainly there have been attempts (https://golangr.com/gui/), but it's absolutely out of the scope of what the language is good for. Any GUI framework in golang is several years behind the solutions available in Python. So, you can use Python if you want a GUI with a low frontend activity (just a viewer for some optimised backend), but I would recommend C++ for a heavy GUI.
Both Python and Golang have good support for building HTTP APIs/backends. Therefore, any web frontend can be completely decoupled from the backend. The structure I suggest is the following:
|
Yes, @eine i agree with all these views. Thanks for clearing out all these things in one go. Also, 😅 I must had used "execution time compilation", instead of "interpretion" for "go run" cmd. But people around me just call it interpretion(informally), as interpretion of a language which supports multiple threads and share memory between threads, is just not feasible, hence node.js relies on clustering and python has global interpreter lock. Thaanks for correcting me there.
So far what I have read, I feel that every feature from x11docker is useful for community, except ultra heavy environments like deepin and rest. There is this fact that, I want to make sure, that if we are investing in rewriting a repository, it must be contributable by a major section of community. IMHO I find all the modern platform level applications are being written in go. To be more clear, developers are divided. We are relying on companies like google for kubernetes. I just feel that I should invest in a homogeneous environment to make passers by stop and take a look here. People love the application when they understand the source code, even at modular level |
@velcrine, I like your vision. However, I believe the effort is non trivial. It will require some full-time weeks, maybe months, to have a feature complete replacement of x11docker in golang. I do absolutely agree that it will then make it much easier to have robust testing, code quality checking, documentation (it's built-in), etc. That will sure make contributions easier, given that golang is so used in the container ecosystem. I do also think that this project is underrated, partially because some people find it to be "just an script", while that is precisely a very valuable feature: it's an stupidly easy to install yet surprisingly powerful toolbox. Still, someone needs to do that initial time investment for providing a working equivalent that convinces both mviereck and the user community about the replacement being absolutely shameless for the current users of x11docker. |
Yeah I have already done an amount of reading and experimentation. Let me clear out other items in 3/4 days. Will get it there. |
@eine @mviereck have come up with this https://github.com/velcrine/lightmachine.dockerfile for dockerfiles. While reading x11docker, I found dockerimages themselves are hiding a lot, hence, came with them first. Its highly maintainable and less-complex, hence user can customize his docker images for his needs easily. Comments need to be enhanced though. Have used this repo for temporary purpose, lets discuss further. BTW, its like one dockerfile, driven via build-args, to produce different desktop environments |
guys done with dockerfiles at https://github.com/velcrine/lightmachine.dockerfile . Also, have experimented quite a bit via custom commands for actual gui mode. Have to tackle with gnome3, not able to run it yet. |
Hi mviereck. I am very much loving this project. What about migrating it to a programming language to
My extreme push to this request is because, I can't bring this into my company, unless I am able to debug any error that arises in it. As this is a tool, which deals a lot on dependencies, environments and badly shaked from version changes, it would be critical to debug it out at any time. And bash is very hard to debug in most cases.
Also, it may take virtualization to an extreme next level, if this project will be easy to contribute. Also have u thought about this prior?
The text was updated successfully, but these errors were encountered: