build: Dockerfile and docker-compose to launch notebook with pygenn resolves genn-team/genn#546 #548

Stevinson · 2022-10-20T15:03:42Z

Run pyGeNN models from a docker image. docker-compose is used to launch a jupyter notebook with files from the local filesystem.

Run GeNN models from a docker image that launches a jupyter notebook

neworderofjamie

Thank you so much for your hard work on this - as a docker n00b, there's just a few things I'd like explained

neworderofjamie · 2022-10-20T16:21:02Z

build_steps/Dockerfile

+
+COPY . ${HOME}
+
+RUN adduser --disabled-password --gecos "" $USERNAME


Why not just use root as containers are kinda throwaway? My knowledge of docker is pretty crap so I could very well be missing something...

It’s something I was taught was best practice so have got into the habit of doing. I believe this is because:

Certain execution environments block containers running as root

Force container immutability

Limits the processes that can be executed and who can execute them

Ahh, that is really good to know, thanks

Running the container as root (from the host) could be bad, sure, but I can't see any issue being root inside the container. Do you have an example of root inside containers being blocked by providers? The mutability argument, I'm not sure how relevant that is, since all state is lost once the container is killed anyway. As for limiting processes, sure, it stops the user running e.g. apt inside the container as interactive, but is that a good thing?

Btw, I don't mean to be pedantic. I'm no expert, and I'm very interested to hear yours and others' reasons for and against!

Ahh.... I see the issue with containers starting as root now.....

I just pwnt my box with a container that begins as root! :)

https://flast101.github.io/docker-privesc/

neworderofjamie · 2022-10-20T16:28:48Z

build_steps/docker-compose.yml

@@ -0,0 +1,28 @@
+version: "3.4"


I don't quite see why docker compose is required....I thought it was intended to combine together multiple containers whereas, couldn't what you do here be achieved with a single docker run incantation?

Also, more fundamentally, the way I imagine the GeNN docker image being used is that it would be hosted on Dockerhub (probably produced by CI when we make releases/merge master) so users wouldn't have to checkout our git repro and hence wouldn't have access to the docker compose script outside of the container.

Again, I may be missing things....

Agreed, it facilitates having multiple containers interacting together. I only used it as I find it helpful to set up port bindings, volumes, health checks, etc in a configurable way that is easy to use (and was used only to launch jupyter).

However, if it is not going to be useful for extendability then maybe, as you say, a command such as docker run -p 127.0.0.1:8888:8888/tcp -v '$(CURDIR)'/pygenn/notebooks:/root/pygenn/notebooks pygenn jupyter notebook --ip 0.0.0.0 --no-browser is better suited. I have added this to the makefile (as make docker-jupyter) and can delete the docker-compose.yml?

Just for saving such brutally long command lines, docker compose seems like a good idea (especially as it's optional).

…er-compose

neworderofjamie

I am going to fiddle a bit more (hence the change of target) but, unless my colleague @jamesturner246 has any further concerns, I'm happy to accept this (and will add the relevant hacktoberfest label). Thank you again for your help with this!

jamesturner246

It looks good. A couple of points that might be worth implementing - maybe @neworderofjamie also has preferences on them. Thanks!

jamesturner246 · 2022-10-21T14:09:00Z

Makefile

@@ -69,3 +69,11 @@ clean:
 	@# Delete libGeNN
 	@rm -f $(LIBGENN)
 	@rm -f $(BACKEND_LIBS)
+


It might be cool and convenient to have a makefile rule to setup an interactive pygenn container with -it, and bind host pwd to some directory (e.g. 'workspace') in the working directory of the container. Just for people who don't want to use jupyter.

build_steps/Dockerfile

jamesturner246 · 2022-10-21T14:10:44Z

pygenn/README.md

@@ -13,3 +13,8 @@ PyGeNN wraps the C++ GeNN API using SWIG, allowing GeNN to be used either direct
 - Navigate to the GeNN directory and build GeNN as a dll using ``msbuild genn.sln /t:Build /p:Configuration=Release_DLL`` (if you don't have CUDA installed, building the CUDA backend will fail but it should still build the CPU backend).
 - Copy the newly built DLLs into pygenn using ``copy /Y lib\genn*Release_DLL.* pygenn\genn_wrapper``
 - Build the Python extension with setup tools using ``python setup.py develop`` command
+


Should probably add pygenn/README.md documentation for the docker-jupyter option, plus for the interactive container rule mentioned above, if implemented.

build: Dockerfile and docker-compose to launch pygenn

db06ad2

Run GeNN models from a docker image that launches a jupyter notebook

Stevinson changed the title ~~build: Dockerfile and docker-compose to launch notebook with pygenn~~ build: Dockerfile and docker-compose to launch notebook with pygenn resolves genn-team/genn#546 Oct 20, 2022

Stevinson mentioned this pull request Oct 20, 2022

Docker install of GeNN #546

Closed

3 tasks

neworderofjamie added the hacktoberfest label Oct 20, 2022

neworderofjamie requested review from neworderofjamie and jamesturner246 October 20, 2022 16:19

neworderofjamie reviewed Oct 20, 2022

View reviewed changes

Add docker run alternative to launch jupyter as opposed to using dock…

9566210

…er-compose

neworderofjamie changed the base branch from master to docker_stevinson October 21, 2022 10:54

neworderofjamie self-requested a review October 21, 2022 10:56

neworderofjamie approved these changes Oct 21, 2022

View reviewed changes

jamesturner246 requested changes Oct 21, 2022

View reviewed changes

neworderofjamie added the hacktoberfest-accepted label Oct 24, 2022

neworderofjamie merged commit 130b600 into genn-team:docker_stevinson Oct 24, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build: Dockerfile and docker-compose to launch notebook with pygenn resolves genn-team/genn#546 #548

build: Dockerfile and docker-compose to launch notebook with pygenn resolves genn-team/genn#546 #548

Stevinson commented Oct 20, 2022

neworderofjamie left a comment

neworderofjamie Oct 20, 2022

Stevinson Oct 21, 2022

neworderofjamie Oct 21, 2022

jamesturner246 Oct 21, 2022 •

edited

Loading

jamesturner246 Oct 21, 2022

jamesturner246 Oct 21, 2022 •

edited

Loading

neworderofjamie Oct 20, 2022

Stevinson Oct 21, 2022

neworderofjamie Oct 21, 2022

neworderofjamie left a comment •

edited

Loading

jamesturner246 left a comment

jamesturner246 Oct 21, 2022

jamesturner246 Oct 21, 2022


		COPY . ${HOME}

		RUN adduser --disabled-password --gecos "" $USERNAME

build: Dockerfile and docker-compose to launch notebook with pygenn resolves genn-team/genn#546 #548

build: Dockerfile and docker-compose to launch notebook with pygenn resolves genn-team/genn#546 #548

Conversation

Stevinson commented Oct 20, 2022

neworderofjamie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesturner246 Oct 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesturner246 Oct 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neworderofjamie left a comment • edited Loading

Choose a reason for hiding this comment

jamesturner246 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesturner246 Oct 21, 2022 •

edited

Loading

jamesturner246 Oct 21, 2022 •

edited

Loading

neworderofjamie left a comment •

edited

Loading