Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Porting x11docker to a programming language #353

Closed
velcrine opened this issue May 12, 2021 · 23 comments
Closed

Feature Request: Porting x11docker to a programming language #353

velcrine opened this issue May 12, 2021 · 23 comments

Comments

@velcrine
Copy link

Hi mviereck. I am very much loving this project. What about migrating it to a programming language to

  1. Bring modularity.
  2. Better ease of code for other contributors.
  3. Easy readability for those who want to apply this project as a mainstream debug environment throught the company and colleges.

My extreme push to this request is because, I can't bring this into my company, unless I am able to debug any error that arises in it. As this is a tool, which deals a lot on dependencies, environments and badly shaked from version changes, it would be critical to debug it out at any time. And bash is very hard to debug in most cases.

Also, it may take virtualization to an extreme next level, if this project will be easy to contribute. Also have u thought about this prior?

@velcrine
Copy link
Author

velcrine commented May 12, 2021

Also I saw using x11docker, a guy made dx11. Which is just one more layer of abstraction. With programming language's extensibility, we can add multiple layers of abstraction, to a single codebase, instead of making a seperate project.

@mviereck
Copy link
Owner

I am open to this.
However, bash is the only language I am familiar with. Several years ago I've started this project to learn bash deeper than the few daily commands and miniscripts.

I gave x11docker a structure that should accord to good practice. So it should be possible to port it to another language.

Currently I am looking for a new language to learn, maybe C++. The reason behind is that I need a GUI for another project (microscopy-tools) that provides some image editing features.
However, the learning curve is steep, and currently I am frustrated about beginner's issues for likely stupid simple tasks if one knows what to do.

One possible way: If one tries to rewrite x11docker in C++, I would assist wherever I can with experience of the setup and minor but important details. While this goes on, I learn C++ looking at the new code.

As this is a tool, which deals a lot on dependencies, environments and badly shaked from version changes, it would be critical to debug it out at any time. And bash is very hard to debug in most cases.

The core dependencies are an X server, some X utils, Docker and linux coreutils. Other dependencies depend on desired features like pulseaudio support. Another language would not really decrease the amount of dependencies. Including the X utils or pulseaudio as libraries just hides the dependencies, but does not decrease them.

x11docker itself rarely changes nowadays. I consider it to be stable. If you look at the code frequency, you'll see that only minor changes where done the last 18 month.
Upcoming issues are mostly related to changes within Docker, and even those are rare.
I don't think bash on itself is hard to debug. That is less a question of the chosen language but of the programmer's style.

@velcrine
Copy link
Author

velcrine commented May 12, 2021

The core dependencies are an X server, some X utils, Docker and linux coreutils. Other dependencies depend on desired features like pulseaudio support. Another language would not really decrease the amount of dependencies. Including the X utils or pulseaudio as libraries just hides the dependencies, but does not decrease them.

And they are a lot of things, weston, Xorg, Xephyr, pulseaudio, diferrent environments(gnome, xfce, etc) + (none of them is what we find a lot of documentation for, neither good amount of discussions). And at all the time we want to add something or another to all these tools, tinkering around with that or something, it breaks one or two things manytimes, I have witnessed that myself.

  • I think I can add integration tests too.

x11docker itself rarely changes nowadays. I consider it to be stable. If you look at the code frequency, you'll see that only minor changes where done the last 18 month.
Upcoming issues are mostly related to changes within Docker, and even those are rare.
I don't think bash on itself is hard to debug. That is less a question of the chosen language but of the programmer's style.

Its about adding even more. Though linux is stable, did the commit race stopped? Lets take it to heights. For ex, I wanted to make multiseat setup possible with x11docker, such that u will not need docking device, which costs a lot, if u wanted to open a computer lab for African villagers. Right now, I have to make a seperate project, but If we first port it to a language, then we can just extend it to provide modularity at a level, that people like me or dx11 can just add-in as a feature.

Currently I am looking for a new language to learn, maybe C++. The reason behind is that I need a GUI for another project (microscopy-tools) that provides some image editing features.
However, the learning curve is steep, and currently I am frustrated about beginner's issues for likely stupid simple tasks if one knows what to do.

For languages, I love cpp from my heart for its truthfulness to the kernel, but because every major application level project I see is in golang, hence i have invested much in Golang. Docker and everything new is going go. Also a heck lot of community there for go.

@mviereck
Copy link
Owner

mviereck commented May 12, 2021

And at all the time we want to add something or another to all these tools, tinkering around with that or something, it breaks one or two things manytimes, I have witnessed that myself.

If something comes up, you can just ask for it. A lot of x11docker code consists of checking possible interferences, avoiding them and giving messages about it. So yes, changes have to be done carefully and well checked.

If we first port it to a language, then we can just extend it to provide modularity at a level, that people like me or dx11 can just add-in as a feature.

I am not sure about this. What could be modularized?
Basically x11docker is just a wrapper that does:

  • Run an X server.
  • Run a container with X credentials and set up a container user.
  • Watch them until one exits and terminate the other.
  • Provide additional features like sound, GPU, webcam, clipboard.

One big part is the X server setup because several different X servers are supported where each has its own advantages and disadvantages. I thought about separating this part into another script, just did not do the work without seeing an obvious advantage.

Another complex part is init support in containers; maybe I should not have included it, because it is lot of code for less needed setups. Though, some desktop environments and even some applications depend on it.

x11docker codebase could be shrinked a lot if:

  • Dropping init in container
  • Dropping GPU support
  • Reducing to few, maybe only one X server

For languages, I love cpp from my heart for its truthfulness to the kernel, but because every major application level project I see is in golang, hence i have invested much in Golang.

I am looking at C++ because it has direct support by imagemagick, a tool that I heavily use in my current project where I need a GUI in future.

I had a look at Golang; there is only an unmaintained wrapper to imagemagick, not sure if I could use the imagemagick C++ libs directly without it.
One point I dislike: It seems Golang only supports static linking. I prefer the concept of shared libraries.

I also had a look at python, but the imagemagick support is unmaintained, too.

@velcrine
Copy link
Author

velcrine commented May 12, 2021

Yeah u are right in all this points. Static linking increases size,but giving hundred percent guarantee that it will run in any environment, any distro, without saying. "some.so.3 not found".
We can start with any, Cpp or Go.
Let me put up some final points from my side on these:
About Go:

  1. For imagemagick wrapper, the development is still active in other branches https://github.com/gographics/imagick/commits/im-7+
    Also they had anounced a release 18 days ago, https://github.com/gographics/imagick/releases/tag/v3.4.0
  • For a wrapper, I wouldn't expect much code upgrade there as go code is good to debug and lower bugs are left untracked.
  1. Really big community. With more people to commit, more useful this tool. Also, more then the use, X11docker has given me learning about a lot of things.

  2. For GUI u are looking . If QT is for cpp, then fyne is for GO https://github.com/fyne-io/fyne

About cpp:

  1. Built for performance, not for developer's comfort. Only performance is the goal, for its architecture and all the libraries it gives are dedicated for low memory footprint and low cpu consumption.
  2. As x11docker will just run once, only for initating other programs, performance isn't the toss. Cpp has a much higher learning curve, but with that if u see, almost all newer cool application level projects are being written in go.

So now, whatever u choose, cpp or go, we should lets GO!

@velcrine
Copy link
Author

Let me add some more,

  1. At the core there will be x11docker with
Without init in container
Without GPU support
Reducing to few, maybe only one X server
  1. Then there be middlewares for adding GPU, adding init in container or any other additional, but not core feature. Middlewares add to the functionality, but stay outside the core.

  2. Then we provide one another layer of abstraction, to manage sessions like dx11.

  3. Then one more layer of abstraction, to provide multi-seat support with this.

Believe me, multiseat with docker is going to be the best use of it, it will reduce the costs of computer labs all around the world. Yeah!!

@mviereck
Copy link
Owner

So now, whatever u choose, cpp or go, we should lets GO!

Please don't hurry me. :-)
The decision I have to make now influences my work for years, so I have to choose wisely what language to learn.
For my goals of the image processing project I have to look at these points:

  • Direct support by imagemagick developers. Looking at the imagemagick github page I find MagickCore (C), MagickWand (C), Magick++ (C++) and Perlmagick (Perl).
  • GUI support (no matter which toolkit yet)
  • Ideally low memory footprint. My current setups already need every GB of RAM I can provide.

Currently I have the impression that C++ is the best choice here.
Looking at the support list, I also consider Perl now; it might be easier to learn than C++. Developing time is a resource to save, too. As you said about C++, "Built for performance, not for developer's comfort."
However, there are some more interesting and free C/C++ graphic libraries around that are likely not supported in perl.

For imagemagick wrapper, the development is still active in other branches https://github.com/gographics/imagick/commits/im-7+

Good to know, thanks! Though, as said above, direct support of imagemagick developers is a critical point, so I won't choose golang for me.

At the core there will be x11docker with
Without init in container
Without GPU support
Reducing to few, maybe only one X server

At this low base there is a short example in the wiki: https://github.com/mviereck/x11docker/wiki/Short-setups-to-provide-X-display-to-container#extended-xephyr-script

Then there be middlewares for adding GPU, adding init in container or any other additional, but not core feature. Middlewares add to the functionality, but stay outside the core.

Providing GPU support already requires multiple X server support. None of them fulfills all desirable setups. Compare the attributes of X servers: https://github.com/mviereck/x11docker/wiki/X-server-and-Wayland-Options#attributes-of-x-server-and-wayland-options

Then we provide one another layer of abstraction, to manage sessions like dx11.

Don't consider dx11 too much. It is just a shortcut wrapper for x11docker commands. At its current stage it looks rather unpolished. I dislike its approach to drop security features by default just for ease of use. Though, I am honored by the developer's excitement of x11docker, and his advertisement gave x11docker some more popularity.

Then one more layer of abstraction, to provide multi-seat support with this.
Believe me, multiseat with docker is going to be the best use of it, it will reduce the costs of computer labs all around the world. Yeah!!
For ex, I wanted to make multiseat setup possible with x11docker, such that u will not need docking device, which costs a lot, if u wanted to open a computer lab for African villagers.

This point sounds most interesting to me.
I am not experienced in multiseat setups at all. Could you elaborate a bit?

  • What is the core concept of multiseat?
  • What is needed to support this?
  • Where are the current hurdles with x11docker?

@velcrine
Copy link
Author

velcrine commented May 13, 2021

What is the core concept of multiseat?
What is needed to support this?
Where are the current hurdles with x11docker?

Multiseat allow one running linux, to be shared among multiple users. Each user has a set of keyboard, mouse and monitor, that is exclusively for them. This is the most cost effective solution, for setting up a computer lab. As one single tower(powerful cpu) can be used.
Most importantly, cpu resources sudden hikes for any one user are compensated by rest users. Also, In a usual lab, CPUs need to be upgraded from time to time, and thus the old one cpu is wasted, as two processors can never coexist there, they have to remove(waste) the old one. Same happens with inteegrated graphics. Single CPU will save from this, as it will barely get old.
There has been surveys that multiseat result in less than half the price for setting up a computer lab, then for a traditional system, if taken care of expenses for a total of 5 years. This setup is however not used in real life because of two reasons :

  1. Users can't be sudo with any possibility. Which blocks you from using it as your personal computer.
  2. When sysadmins install something for you, it gets installed for every single user. Imagine if you have to eat food, all the users will eat same food, but drinks are of their choice. Funny!!
  3. Many utilities like Xinput, can be used for bad purpose by any user. Loopholes are always there. Such as chmod a+r is the default permission, and by default u will let everyone see your files. Your browser history is also stored in such file, and can be accessed.
  4. U need one soundcard, one gpu per user. (really costly!)

Currently they are just used in cybercafes or customer-care centers,because of these short comings.
With x11docker, I have won over all of them shortcomings, and successfully demonstrated this. I assume gui drawing via Xephyr utilizes gpu, however graphics processing inside the docker image require seperate setup to utilize gpu( which include playing video & games, for both labs are not made for.)
Imagine how many school students in un-developed country will gain access to computers!!

@mviereck
Copy link
Owner

I like to support this.
We should point out your requirements. From what I understand, the only major drawback of x11docker is that you have to entirely know and understand the code so you are able to debug it by all means.
This requirement can only be fulfilled by own code.
So it might make sense that you write your own version of x11docker.

To encircle your needs (please extend/correct):

  • Only desktop environments in container. Seamless applications are not needed.
  • multiseat support for keyboard, mouse and sound. In Xephyr help !! isolating keyboard #351 you told of issues and need an outdated Xephyr yet for evdev support. I am not sure how to solve this.
  • GPU support yes/no? NVIDIA support yes/no?
  • Some desktops need systemd support, others don't. Is it ok to drop desktops like GNOME3 and Cinnamon, but use e.g. Xfce, LXDE, others instead?
  • x11docker always runs a fresh container from image and deletes them on exit. There are hurdles to re-use old containers because of X unix sockets. Is re-using containers a requirement? (X over tcp is an alternative to allow re-using containers, but does not support GPU usage well.)

Proposal:

  • Outline your needs so we can focus on needed features and drop others. Based on the needs, we might reduce to one X server only.
  • Write in C++ so I could learn from your code. (If you insist on golang, I would help you, too; but it would help me more if you use C++.)
  • Together we develop this; you code, I assist explaining the X and docker setups that are needed.

@velcrine
Copy link
Author

velcrine commented May 13, 2021

thanks mviereck for getting in.

Our proposal:

Write in C++ so I could learn from your code. (If you insist on golang, I would help you, too; but it would help me more if you use C++.)

I will take 2 months to make space for Cpp as I am currently indulged in GO, dart and python3 based projects.
As your project microscopy-tools is dealing with research and education, I will try every bit to get you in comfort, hence lets do it in cpp. Also, I am fond of cpp because of performance, but never got chance to work on it.

Together we develop this; you code, I assist explaining the X and docker setups that are needed.

Yeah!!

Outline your needs so we can focus on needed features and drop others. Based on the needs, we might reduce to one X server only.

I just need Xephyr to run both on wayland and X11. We can think of keeping others also as an extension,not part of core codebase, don't know if community utilize them.

Encircling our needs:

  1. Only desktop environments in container. Seamless applications are not needed.

As mutiseat is just a layer of abstraction over x11docker, x11docker will just have both (desktop+seamless), multiseat will just hide seamless application mode.

  1. multiseat support for keyboard, mouse and sound. In Xephyr help !! isolating keyboard #351 you told of issues and need an outdated Xephyr yet for evdev support. I am not sure how to solve this.

Latest version Xephyr 20 is lacking evdev support, but version 19 had it, which I have installed currently. Currently I am experimenting Xephyr with one other way. If that doesn't work, either we keep a binary of Xephyr in code or keep updating Xephyr with the patch and make it locally, some people have done that.

  1. GPU support yes/no? NVIDIA support yes/no?

It depends on you, as it depends on the core x11docker. I have little interest in gpu, as in docker computation for gpu might be just for playing games and videos.

  1. Some desktops need systemd support, others don't. Is it ok to drop desktops like GNOME3 and Cinnamon, but use e.g. Xfce, LXDE, others instead?

People generally need less memory footprint for their docker based desktop environment, atleast me. But I really don't know abour all the people out their using x11docker out their with gnome and waiting for next big release.

  1. x11docker always runs a fresh container

Hey thats a big one. Currently its just like as if running an OS from live CD boot. Persistance of my work is atmost importance to me, even in general. There can be many ways for persisting state, I was thinking of giving user choice for what method he wants, depending on his need.

We need a .ini based file or .json/yaml file to load config from, also could be overwritten from command line. This will give user more managibility.

@mviereck
Copy link
Owner

5 > x11docker always runs a fresh container
Hey thats a big one. Currently its just like as if running an OS from live CD boot. Persistance of my work is atmost importance to me, even in general.

Currently persistance for user files is possible with options --home and --share. But e.g. an application installed with sudo apt-get install ... will be lost on next startup.

You can design multiseat-x11docker to reuse containers. That would nail down the setup to X over TCP. That in return reduces the choice of possible X servers. Noteable, that disallows Xwayland setups that are needed for multiple X servers with GPU access.

So we have an odd choice: We can't have GPU acceleration along with persistant containers. (Possible untestet way: If each user has its own monitor plugged into its own graphic card, all containers could run with Xorg that supports both X over tcp and GPU acceleration.)

Outline your needs so we can focus on needed features and drop others. Based on the needs, we might reduce to one X server only.
I just need Xephyr to run both on wayland and X11. We can think of keeping others also as an extension,not part of core codebase, don't know if community utilize them.

We need to get clear about the desired setups, because no X server supports all possibly desired setups.
For the special target multiseat we can at least say:

  • seamless mode is not needed because every client gets its own desktop environment on its own monitor.
  • multiseat needs lightweight clients. init/systemd in container is needed for heavy desktop environments like Gnome3. So it's ok to drop systemd support in favor of thin clients only.

Open questions:

  • GPU yes/no?
  • Persistant container yes/no?

It depends on you, as it depends on the core x11docker.

This doesn't depend on me, but on your target. :-) The more choices you provide, the more work you'll have. Rather have a concept design that is slim but fits most of the targeted multiseat users.
x11docker is already a bit blown up, and meanwhile I wish I did not include that much.

Latest version Xephyr 20 is lacking evdev support, but version 19 had it, which I have installed currently.

It seems to be a bug and not an intended deprecation. Hopefully it will be fixed. In general it is no good idea to keep old versions of anything alive. Their problems grow with their age. nxagent would be an alternative, but has its own issues.

@velcrine
Copy link
Author

velcrine commented May 14, 2021

Open questions:

GPU yes/no?
Persistant container yes/no?

GPU is not a priority, if it comes along with rest of things then ok, otherwise I wouldn't crave for it.
Persistant containers are most important to save user's work.

It seems to be a bug and not an intended deprecation. Hopefully it will be fixed. In general it is no good idea to keep old versions of anything alive. Their problems grow with their age. nxagent would be an alternative, but has its own issues.

I am stuck with this actually. They intentionally removed support for evdev. Let me see nxagent. Atleast, for me, its not an issue till yet. But as ubuntu 22 comes, it might break. Hence, I will not hurry on a solution for this.

Meanwhile closig this issue, as I will take 1-2 months to learn Cpp along with all other tasks, will continue after that.
Thanks mviereck.

@eine
Copy link
Contributor

eine commented May 14, 2021

IMHO, using a compiled language for this project does not make much sense. There is no intense computing which would benefit from the advantages that either C++, golang or Rust might provide. However, the need to compile the tool for each target architecture and OS is a huge maintenance burden which is not justified except by your excitement to learn a new language (for other purposes). Therefore, I would strongly discourage using any of those.

Do not take me wrong. I like, know and use golang; I've heard very good things about Rust, and C++ is a very used language in lots of tools. However, a relevant aspect of engineering is picking the appropriate tool for a task. x11docker and runx are scripting projects. It's sensible to consider Python, Perl, TCL or even JS/TS, as any of those would provide some better data abstraction. Yet, a compiled language is overkill. I would personally suggest Python if you really wanted to rewrite this tool.

Python is a very interesting language, because it's an scripting/interpreted language with first class support for C/C++ under the hood. Therefore, it allows using scripting only, and then switching to shared libraries written in C/C++ for better performance requirements. For didactic purposes, this is a very interesting project about wrapping a golang tool in Python: https://github.com/asottile/dockerfile.

@velcrine velcrine reopened this May 14, 2021
@velcrine
Copy link
Author

velcrine commented May 14, 2021

@eine python is not statically typed language which makes debugging bad.

IMHO, using a compiled language for this project does not make much sense.

However I totally agree with your views and believer of same. Go can be used in interpreted mode using "go run", hence making it platform independent too. Also I do favor, it should not be the language we have learnt, but the language project needs. This has been an important factor in widespreading of projects in community.

  1. Also, I want to provide web based interface to whole setup, as that also will be platform independent.

However, a relevant aspect of engineering is picking the appropriate tool for a task. x11docker and runx are scripting projects. It's sensible to consider Python, Perl, TCL or even JS/TS, as any of those would provide some better data abstraction. Yet, a compiled language is overkill

I think u mean any interpreted language.

@velcrine
Copy link
Author

velcrine commented May 14, 2021

Also @mviereck , about evdev, this is the commit they removed it from xserver. https://gitlab.freedesktop.org/xorg/xserver/-/commit/27819950e4158326e0f83a30f2e8968b932625ef, If I add that code back and compile with master, I believe it will work. However, I am willing to debug if any issues arise. Just created an issue also there. https://gitlab.freedesktop.org/xorg/xserver/-/issues/1174

@mviereck
Copy link
Owner

mviereck commented May 14, 2021

However, the need to compile the tool for each target architecture and OS is a huge maintenance burden
However, a relevant aspect of engineering is picking the appropriate tool for a task. x11docker and runx are scripting projects. It's sensible to consider Python, Perl, TCL or even JS/TS, as any of those would provide some better data abstraction. Yet, a compiled language is overkill.

That's a good point, thank you!

this is the commit they removed it from xserver. https://gitlab.freedesktop.org/xorg/xserver/-/commit/27819950e4158326e0f83a30f2e8968b932625ef

Thanks for the link. However, it is still not clear to me if that only affects Xephyr or the entire X11 base.

Meanwhile closig this issue, as I will take 1-2 months to learn Cpp along with all other tasks, will continue after that.

Some basic thoughts:

  • Transferring entire x11docker into another language is a far to great task. It works as it is.
  • Due to the size of x11docker (9k lines of code) it is a bit difficult to debug and maintain it for others than myself.
  • The requirement of your company is to have a code that fits the multiseat target and is entirely maintainable by yourself.

Conclusions:

  • Keep it simple.
  • Clearly outline the requirements for the multiseat setups. Keep them as small as possible.
  • Write a small code in the language you prefer and that fits the task.

Proposal:

The code base could be ready within a week.

@velcrine
Copy link
Author

Thanks for the link. However, it is still not clear to me if that only affects Xephyr or the entire X11 base.

Currently, I just change Xephyr version, to make it work.
Also, the dir kdrive in that repo is responsible for bulding Xephyr + many other things. If some dependency issue comes, I will build for every tag exclusively. There is just 1, v1.20 , till now.

Transferring entire x11docker into another language is a far to great task. It works as it is.

I think you mean it is too huge to transfer. If yes, then no problem there. I think I can do it in few months.

The requirement of your company is to have a code that fits the multiseat target and is entirely maintainable by yourself.

My company wants to use it for general debug environment, in place of a vm. Hence all the features I wish to port, all of them. Multiseat is just a social help from me.

Clearly outline the requirements for the multiseat setups. Keep them as small as possible.

Yeah let me get started, and write the https://github.com/mviereck/x11docker/wiki/Short-setups-to-provide-X-display-to-container#extended-xephyr-script . I have worked on audio, but webcam might take a bit of time.

Yeah I get start in 3 days, after killing my current leftover tasks. Thanks

@eine
Copy link
Contributor

eine commented May 14, 2021

@eine python is not statically typed language which makes debugging bad.

@velcrine, that is a tradeoff. You decide to have static typing by default or add it afterwards. By the same token, you decide to have generic interfaces or you add them later.

  • With Python, you can quickly iterate and try different approaches, because it is interpreted and not strongly typed, by default. However, "type hints" are supported, and you can run tools such as mypy :

Mypy is an optional static type checker for Python that aims to combine the benefits of dynamic (or "duck") typing and static typing. Mypy combines the expressive power and convenience of Python with a powerful type system and compile-time type checking. Mypy type checks standard Python programs; run them using any Python VM with basically no runtime overhead.

  • With Golang, iterating and testing is slower, because everything needs to be typed from the beginning, and you cannot even have an unused statement (otherwise it refuses to compile/run). You can then try using interfaces for an attempt to introduce some generics in the logic. However, that needs to be done once the base data types (structs) are defined already.

Therefore, from a robustness point of view, both Python and Golang do allow very similar use cases, albeit with very different learning/effort curves. The Python learning curve is exponential and Golang's is inverse exponential.

Go can be used in interpreted mode using "go run", hence making it platform independent too.

That is not correct. Golang has a blazing fast compiler. Therefore, you can execute go run for achieving the illusion of running an interpreted script. However, it is actually building the executable and running it straightaway. It is not running it from memory. See https://golang.org/pkg/cmd/go/internal/run/.

Conversely, in Python execution is interpreted by default. However, it has some internal heuristics and it compiles the pieces of the software that are used/executed frequently. That is done transparently to the user.

Also I do favor, it should not be the language we have learnt, but the language project needs. This has been an important factor in widespreading of projects in community.

Honestly, I believe that the line between Python and Golang is very blurry in this case. If x11docker was to be written from scratch, I would suggest Python because of the faster iteration. However, since x11docker is written already (and the codebase itself is pretty well structured), it might be feasible to go for golang straightaway. Personally, I will be good with supporting and helping if you pick any of those. My concern was that I perceived you were "convinced" about C++; that's what I find a no-go.

Overall, there is a very relevant topic that went unnoticed in your dialogue above: Docker is written in golang, and therefore, their libraries/modules can be used in golang projects! Any other language needs to duplicate the data structures and API, but using golang avoids all of that. @velcrine did comment it, but the attention was not focused longer.

Very precisely, I do have a tool written in golang which uses the docker library for creating and running containers:

See how are flags, args and config options declared: https://github.com/eine/issue-runner/blob/master/tool/main.go. See where are the docker libs used for creating, pulling, removing containers: https://github.com/eine/issue-runner/blob/master/tool/docker.go. Find the official examples in https://docs.docker.com/engine/api/sdk/examples/.

Therefore, as you might imagine, I'm absolutely ok if you/we decide to use golang. Nevertheless, you saw that official bindings and examples for Python are available too. Therefore, from the point of view of mviereck, who would need to learn Python or golang for the purpose of this project only, I cannot make an strong defense of golang. I honestly think he needs to evaluate both languages, and we need to help him decide which one he feels more comfortable with. The value of this project is the knowledge in his mind, we need to make it feasible to translate that into a different language, without the language being a burden for him. On the other hand, if x11docker itself is not going to be rewritten, the choice is irrelevant, because reusing data structures won't be possible.


With regard to the side discussion, I believe that you don't want to use neither Python nor golang for serious GUI applications. None of those languages were designed for that purpose. Python is bash on steroids. A lot of projects do provide Python bindings (ScyPy, ONXX, Blender...). It's for scripting and interacting with optimised tools or shared libraries. You can build standalone GUIs with Python (e.g. https://www.youtube.com/watch?v=QWqxRchawZY or https://www.pygame.org), but it's not the best language for that purpose. On the other hand, golang is a "system language", so it's neither designed for GUI tools. Certainly there have been attempts (https://golangr.com/gui/), but it's absolutely out of the scope of what the language is good for. Any GUI framework in golang is several years behind the solutions available in Python. So, you can use Python if you want a GUI with a low frontend activity (just a viewer for some optimised backend), but I would recommend C++ for a heavy GUI.

  1. Also, I want to provide web based interface to whole setup, as that also will be platform independent.

Both Python and Golang have good support for building HTTP APIs/backends. Therefore, any web frontend can be completely decoupled from the backend. The structure I suggest is the following:

  • golang/python:
    • lib: sources used by api and cli, and expected to be used as a library by external projects (core).
    • cli: CLI tool for using the resources implemented in lib.
    • api: REST/AJAX service and file server (backend).
  • HTML + CSS + JavaScript/TypeScript:
    • app: sources of a web-based GUI (frontend).
  • doc: documentation.

@velcrine
Copy link
Author

velcrine commented May 14, 2021

Yes, @eine i agree with all these views. Thanks for clearing out all these things in one go. Also, 😅 I must had used "execution time compilation", instead of "interpretion" for "go run" cmd. But people around me just call it interpretion(informally), as interpretion of a language which supports multiple threads and share memory between threads, is just not feasible, hence node.js relies on clustering and python has global interpreter lock. Thaanks for correcting me there.

, if x11docker itself is not going to be rewritten, the choice is irrelevant, because reusing data structures won't be possible

So far what I have read, I feel that every feature from x11docker is useful for community, except ultra heavy environments like deepin and rest.

There is this fact that, I want to make sure, that if we are investing in rewriting a repository, it must be contributable by a major section of community. IMHO I find all the modern platform level applications are being written in go. To be more clear, developers are divided. We are relying on companies like google for kubernetes.
While there was a similar project called Apache aurora which inspired kubernetes, why it failed : lack of community. X11docker can completely replace VMs for testing environments. 2.8k stars are pretty underrated. Also we will write integration tests somehow.

I just feel that I should invest in a homogeneous environment to make passers by stop and take a look here. People love the application when they understand the source code, even at modular level

@eine
Copy link
Contributor

eine commented May 14, 2021

@velcrine, I like your vision. However, I believe the effort is non trivial. It will require some full-time weeks, maybe months, to have a feature complete replacement of x11docker in golang. I do absolutely agree that it will then make it much easier to have robust testing, code quality checking, documentation (it's built-in), etc. That will sure make contributions easier, given that golang is so used in the container ecosystem. I do also think that this project is underrated, partially because some people find it to be "just an script", while that is precisely a very valuable feature: it's an stupidly easy to install yet surprisingly powerful toolbox. Still, someone needs to do that initial time investment for providing a working equivalent that convinces both mviereck and the user community about the replacement being absolutely shameless for the current users of x11docker.

@velcrine
Copy link
Author

velcrine commented May 14, 2021

Still, someone needs to do that initial time investment for providing a working equivalent.

Yeah I have already done an amount of reading and experimentation. Let me clear out other items in 3/4 days. Will get it there.

@velcrine
Copy link
Author

velcrine commented May 23, 2021

@eine @mviereck have come up with this https://github.com/velcrine/lightmachine.dockerfile for dockerfiles. While reading x11docker, I found dockerimages themselves are hiding a lot, hence, came with them first. Its highly maintainable and less-complex, hence user can customize his docker images for his needs easily. Comments need to be enhanced though. Have used this repo for temporary purpose, lets discuss further.
P.S. thanks @mviereck for help on clearing some issues in dockerfiles.

BTW, its like one dockerfile, driven via build-args, to produce different desktop environments

@velcrine
Copy link
Author

velcrine commented Jun 2, 2021

guys done with dockerfiles at https://github.com/velcrine/lightmachine.dockerfile . Also, have experimented quite a bit via custom commands for actual gui mode. Have to tackle with gnome3, not able to run it yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants