feat(dev): add devcontainers support #824
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #824 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 139 139
Lines 4356 4389 +33
Branches 823 855 +32
=========================================
+ Hits 4356 4389 +33 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Thanks for this PR. I have not played with devcontainers before. The concept looks fantastic. I see how this can be very helpful for new devs to get started on the project, so this update has my full support.
Unfortunately, I couldn't get this working on my machine. When I ran yarn inside of the devcontainer, it gave an error about make not found. Then when I ran yarn dev, the react dev server started up fine, but electron couldn't start.
View full logs
debian@eed5e49ee1e6:/workspaces/polar$ yarn
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning " > @mrblenny/[email protected]" has incorrect peer dependency "react@^16.8.4".
warning " > @mrblenny/[email protected]" has incorrect peer dependency "react-dom@^16.8.4".
warning " > @mrblenny/[email protected]" has incorrect peer dependency "styled-components@^5.1.0".
warning "@mrblenny/react-flow-chart > [email protected]" has unmet peer dependency "prop-types@^15.5.4".
warning "@mrblenny/react-flow-chart > [email protected]" has incorrect peer dependency "react@^15.0.0 || ^16.0.0".
warning "@mrblenny/react-flow-chart > [email protected]" has incorrect peer dependency "react-dom@^15.0.0 || ^16.0.0".
warning " > @testing-library/[email protected]" has unmet peer dependency "@testing-library/dom@>=7.21.4".
warning " > [email protected]" has incorrect peer dependency "react-redux@^6.0.0 || ^7.1.0".
warning " > [email protected]" has incorrect peer dependency "webpack@^5.0.0".
warning "react-lazylog > [email protected]" has incorrect peer dependency "react@^15.3.0 || ^16.0.0-alpha".
warning "react-lazylog > [email protected]" has incorrect peer dependency "react-dom@^15.3.0 || ^16.0.0-alpha".
warning " > [email protected]" has incorrect peer dependency "react@^16.8.0".
warning " > [email protected]" has incorrect peer dependency "react-dom@^16.8.0".
warning "react-router-transition > [email protected]" has incorrect peer dependency "react@^0.14.9 || ^15.3.0 || ^16.0.0".
warning " > [email protected]" has incorrect peer dependency "stylelint@>= 11.x < 15".
warning Workspaces can only be enabled in private projects.
[4/4] Building fresh packages...
[1/13] ⠄ protobufjs
[2/13] ⠄ protobufjs
[3/13] ⠄ cpu-features
[4/13] ⠄ core-js-pure
warning Error running install script for optional dependency: "/workspaces/polar/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node buildcheck.js > buildcheck.gypi && node-gyp rebuild
Arguments:
Directory: /workspaces/polar/node_modules/cpu-features
Output:
/workspaces/polar/node_modules/buildcheck/lib/index.js:115
throw new Error('Unable to detect compiler type');
^
Error: Unable to detect compiler type
at BuildEnvironment.getKind (/workspaces/polar/node_modules/buildcheck/lib/index.js:115:9)
at BuildEnvironment.tryCompile (/workspaces/polar/node_modules/buildcheck/lib/index.js:537:15)
at BuildEnvironment.checkHeader (/workspaces/polar/node_modules/buildcheck/lib/index.js:423:25)
at Object.<anonymous> (/workspaces/polar/node_modules/cpu-features/buildcheck.js:16:4)
at Module._compile (node:internal/modules/cjs/loader:1198:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
at Module.load (node:internal/modules/cjs/loader:1076:32)
[10/13] ⠐ core-js
[9/13] ⠐ dtrace-provider
[3/13] ⠐ cpu-features
[11/13] ⠐ utf-8-validate
error /workspaces/polar/node_modules/bufferutil: Command failed.
Exit code: 1
Command: node-gyp-build
Arguments:
Directory: /workspaces/polar/node_modules/bufferutil
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | arm64
gyp info find Python using Python version 3.11.2 found at "/usr/bin/python3"
gyp http GET https://nodejs.org/download/release/v16.20.2/node-v16.20.2-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v16.20.2/node-v16.20.2-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v16.20.2/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v16.20.2/SHASUMS256.txt
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args '/home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/workspaces/polar/node_modules/bufferutil/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/debian/.cache/node-gyp/16.20.2/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/debian/.cache/node-gyp/16.20.2',
gyp info spawn args '-Dnode_gyp_dir=/home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/debian/.cache/node-gyp/16.20.2/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/workspaces/polar/node_modules/bufferutil',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]
gyp ERR! build error
gyp ERR! stack Error: not found: make
gyp ERR! stack at getNotFoundError (/home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/which/which.js:10:17)
gyp ERR! stack at /home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/which/which.js:57:18
gyp ERR! stack at new Promise (<anonymous>)
gyp ERR! stack at step (/home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/which/which.js:54:21)
gyp ERR! stack at /home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/which/which.js:71:22
gyp ERR! stack at new Promise (<anonymous>)
gyp ERR! stack at subStep (/home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/which/which.js:69:33)
gyp ERR! stack at /home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/which/which.js:80:22
gyp ERR! stack at /home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/isexe/index.js:42:5
gyp ERR! stack at /home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/isexe/mode.js:8:5
gyp ERR! System Linux 6.4.16-linuxkit
gyp ERR! command "/home/debian/.nvm/versions/node/v16.20.2/bin/node" "/home/debian/.nvm/versions/node/v16.20.2/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /workspaces/polar/node_modules/bufferutil
debian@eed5e49ee1e6:/workspaces/polar$ yarn dev
yarn run v1.22.19
$ concurrently --kill-others --success first "yarn:dev:*"
$ yarn theme && cross-env BROWSER=none yarn start
$ wait-on http://localhost:3000 && nodemon -I --watch ./electron/ --watch ./src/shared/ --ext ts --exec electron ./public/dev.js
$ yarn theme:light && yarn theme:dark
$ lessc --include-path=node_modules --js ./src/theme/light.less ./public/themes/light.css
$ lessc --include-path=node_modules --js ./src/theme/dark.less ./public/themes/dark.css
$ rescripts start
[app] (node:10625) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./" in the "exports" field module resolution of the package at /workspaces/polar/node_modules/postcss-safe-parser/node_modules/postcss/package.json.
[app] Update this package.json to use a subpath pattern like "./*".
[app] (Use `node --trace-deprecation ...` to show where the warning was created)
[app] ℹ ｢wds｣: Project is running at http://172.17.0.3/
[app] ℹ ｢wds｣: webpack output is served from
[app] ℹ ｢wds｣: Content not from webpack is served from /workspaces/polar/public
[app] ℹ ｢wds｣: 404s will fallback to /
[app] Starting the development server...
[app]
[app] Browserslist: caniuse-lite is outdated. Please run:
[app] npx browserslist@latest --update-db
[app]
[app] Why you should do it regularly:
[app] https://github.com/browserslist/browserslist#browsers-data-updating
[app] Browserslist: caniuse-lite is outdated. Please run the following command: `yarn upgrade`
[app] Compiled with warnings.
[app]
[app] ./node_modules/ssh2/lib/protocol/crypto.js
[app] Module not found: Can't resolve './crypto/build/Release/sshcrypto.node' in '/workspaces/polar/node_modules/ssh2/lib/protocol'
[app]
[app] ./node_modules/dtrace-provider/dtrace-provider.js
[app] Module not found: Can't resolve './src/build' in '/workspaces/polar/node_modules/dtrace-provider'
[app]
[app] ./node_modules/ssh2/lib/protocol/constants.js
[app] Module not found: Can't resolve 'cpu-features' in '/workspaces/polar/node_modules/ssh2/lib/protocol'
[app]
[app] Search for the keywords to learn more about each warning.
[app] To ignore, add // eslint-disable-next-line to the line before.
[app]
[electron] [nodemon] 3.0.2
[electron] [nodemon] to restart at any time, enter `rs`
[electron] [nodemon] watching path(s): electron/**/* src/shared/**/*
[electron] [nodemon] watching extensions: ts
[electron] [nodemon] starting `electron ./public/dev.js`
[electron] /workspaces/polar/node_modules/electron/dist/electron: error while loading shared libraries: libgobject-2.0.so.0: cannot open shared object file: No such file or directory
[electron] [nodemon] failed to start process, "electron" exec not found
[electron] [nodemon] Error
[electron] at Bus.<anonymous> (/workspaces/polar/node_modules/nodemon/lib/nodemon.js:158:25)
[electron] at Bus.emit (node:events:525:35)
[electron] at ChildProcess.<anonymous> (/workspaces/polar/node_modules/nodemon/lib/monitor/run.js:199:11)
[electron] at ChildProcess.emit (node:events:513:28)
[electron] at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)
[electron] yarn run dev:electron exited with code 0
--> Sending SIGTERM to other processes..
node:events:491
throw er; // Unhandled 'error' event
^
Error: spawn ps ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:285:19)
at onErrorNT (node:internal/child_process:485:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
Emitted 'error' event on ChildProcess instance at:
at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
at onErrorNT (node:internal/child_process:485:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn ps',
path: 'ps',
spawnargs: [ '-o', 'pid', '--no-headers', '--ppid', 10233 ]
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
I can see how devcontainers are useful for web apps which just need to listen on a specified port, but do they also support electron-based desktop apps? I'm not sure how Polar can run inside of a docker container which has no UI. Can you provide some additional info on how you plan to use this with Polar?
|
Hi. I built a dev container and was able to install, test and compile polar inside the container, but I have not tried the access UI or hot-reload. Here is an example:
docker image build --tag polardev:node16 .devcontainer/
cd polar/
docker container run \
--volume $PWD:/app \
--name polar-dev \
--detach \
polardev:node16 \
sleep infinity
docker exec polar-dev yarn install
docker exec polar-dev yarn lint
docker exec polar-dev yarn test
docker exec polar-dev yarn packageThe examples above is to manually "use" devcontainers, but it does work in Vscode using the DevContainers extension. I could only get devcontainer to work in VSscode, but not Vscodium, the FLOSS fork. Regarding the UI, I believe it is possible to forward the port from the container to the rust and access it via a standard browser. I also noticed I forgot to expose a port in the Dockerfile, so that needs to be addressed. Let's work on this. Adding documentation will be helpful to future developers. |
|
Ah ok. Yes, I think we should have some comments in the CONTRIBUTING doc about how to use this. |
uwla
force-pushed
the
feat-devcontainers
branch
from
67520c0 to
0c8c2e2
Compare
|
Hi @jamaljsr, can you take a look again? Also, I'd like to point out that I added the flag |
uwla
force-pushed
the
feat-devcontainers
branch
from
0c8c2e2 to
51745de
Compare
|
I just quickly skimmed through the diff. I'm shocked that you got the GUI working, I didn't think that would be possible. I haven't tested but it seems like this would only work for linux systems since the |
|
@jamaljsr running GUI applications using Docker is currently only possible using Linux distributions with X11 backend. It is not even possible in Linux Wayland. That's due to the flexibility of X11 protocol. There is a project called x11docker, https://github.com/mviereck/x11docker, which automates running generic GUI applications in Docker, such as web brwosers, mail clients, file managers, video editors, and so on. But it is possible to use the devcontainers in Windows and Mac in headless mode to install packages, lint code, run tests and compile Polar. I'll create a Maybe, we should official polar development images. There is already Sounds good? |
Ok, thanks for the confirmation. Since this only works for Linux, I'd suggest adding a note in the instructions stating that. Also, I don't think these instructions should be at the top of the CONTRIBUTING doc since these instructions won't work for everyone. Can we keep the "Local approach" section first with these instructions below it. My thought is that the majority of people that want to contribute changes in the app (requiring the GUI to work), or they want to package a binary from source code to run on their system. I don't think many Mac/Windows users would go through this process just to lint, test, and build a Linux binary.
I like this idea. I can work on pushing these images to docker hub. Since the Node v20 update is now merged, I wouldn't do a v16 image since it won't work any longer. |
|
That makes sense. I'll move the docs to the bottom of CONTRIBUTING and update node version to 20. What is the minor version? I mean, is it 20.1 or 20.2, or something else. |
uwla
force-pushed
the
feat-devcontainers
branch
from
51745de to
b8b1154
Compare
|
@jamaljsr, I improved the PR |
uwla
force-pushed
the
feat-devcontainers
branch
from
b8b1154 to
28601c9
Compare
There was a problem hiding this comment.
LGTM 👌
I've pushed up images to Docker Hub. Unfortunately the ui image doesn't build for arm64, so I could only push that one for amd64. I attached the logs in case you have any suggestions. If not, we skip it.
Here's the logs with the error:
docker_build.log
Do you want to mention the Docker Hub images in the instructions before merging this?
Automate the setup of development environment via devcontainers.
uwla
force-pushed
the
feat-devcontainers
branch
from
28601c9 to
269e528
Compare
|
I added mention to DockerHub images in the instructions. |
|
Great! Thanks for the updates. |
Development feature: automate the setup of development environment via devcontainers.
This facilitates developing Polar by configuring a reproducible environment to build, test and develop.
The devcontainers specification can be integrated into IDEs like Vscode/Vscodium.
Alternativaley, it is possible to manually create a container and run commands inside of it.