-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker dev env #5583
Merged
Merged
Docker dev env #5583
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
8c27a31
add docker development environment
maxgrossman ff9d8dd
newlines
maxgrossman 4a63997
do not make build dep of up. this way build can be separate way to ha…
maxgrossman 35f9578
just mount hoot-ui-2x dir
maxgrossman ffcd681
removing extra lines
maxgrossman 92f4a4a
adding missing and fixing incorrect volumes
maxgrossman 00aadc2
let us attach a remote debugger for services
maxgrossman d34649a
only copy files/war when we build them
maxgrossman 1597bef
add db_host_osmapi to DatabaseConfigLocal target
maxgrossman a146a0a
correct osmapi host name
maxgrossman 68b5d30
handle older versions of docker/docker-compose
brianhatchl 6a3454a
git ignore .npm
brianhatchl f011038
update hoot-ui-2x submodule
maxgrossman eba2ea6
add missing new line
maxgrossman d365fc8
Merge remote-tracking branch 'origin/master' into docker_dev_env
brianhatchl 19abb86
Updates to docker PR to work in fresh git repo clone (#5614)
brianhatchl 388e63c
restore FORCE to always attempt VersionDefines.h rebuild
brianhatchl 335e4a9
this seems to be causing ./HOOT_VERSION_GEN to loop
brianhatchl f062ae9
don't need this
brianhatchl 2204e2d
revert changes to see if it affects 'make archive'
brianhatchl 5a6d332
try .PHONE instead of FORCE
brianhatchl e0ad41b
Merge remote-tracking branch 'origin/master' into docker_dev_env
brianhatchl 7ced311
Merge remote-tracking branch 'origin/master' into docker_dev_env
brianhatchl 89553be
Submodule hoot-ui-2x 521e79585..d2bbe9541:
brianhatchl 5de4d6c
wip: add some docker command refs
brianhatchl 405490c
need to run configure each time because generated Makefile can get stale
brianhatchl c2e8e00
need to call this too when core builds services
brianhatchl f06d3bb
Move versions out to .env file and expose in docker-compose.yml (#5675)
brianhatchl 2504791
Merge remote-tracking branch 'origin/master' into docker_dev_env
brianhatchl 974780c
Submodule hoot-ui-2x 7e0099d18...804b1502f:
brianhatchl File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| POSTGRESQL_VERSION=13 | ||
| GLPK_VERSION=4.64 | ||
| LIBOAUTHCPP_VERSION=0.1.0 | ||
| LIBPHONENUMBER_VERSION=8.12.39 | ||
| LIBPOSTAL_VERSION=1.1 | ||
| NODE_VERSION=14.16.1 | ||
| NPM_VERSION=6.14.12 | ||
| V8_VERSION=8.4.371.19 | ||
| STXXL_VERSION=1.3.1 | ||
| ARMADILLO_VERSION=10.8.2 | ||
| GDAL_VERSION=3.2.3 | ||
| GEOS_VERSION=3.9.3 | ||
| LIBGEOTIFF_VERSION=1.6.0 | ||
| PROJ_VERSION=7.2.1 | ||
| DEVTOOLSET_VERSION=8 | ||
| JDK_VERSION=1.8.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -185,4 +185,11 @@ config.rpath | |
| 0 | ||
| tbs/0 | ||
| .DS_Store | ||
| .ccache | ||
| .ccache/**/* | ||
| .m2 | ||
| .env | ||
| hoot.env | ||
| .bash_history | ||
| .config | ||
| .npm | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| ## Docker development environment cheat sheet | ||
|
|
||
|
|
||
| To start the containers | ||
| >make -f Makefile.docker up | ||
|
|
||
|
|
||
| To stop the containers | ||
| >make -f Makefile.docker down | ||
|
|
||
|
|
||
| To tail tomcat logs | ||
| >docker-compose logs --follow core-services | ||
|
|
||
|
|
||
| To bash to the core-services container | ||
| >docker-compose exec core-services bash | ||
|
|
||
|
|
||
| To force clean/build on container start, modify hoot.env _e.g._ | ||
| ``` | ||
| HOOT_CLEAN=0 | ||
| HOOT_BUILD_CORE=1 | ||
| HOOT_BUILD_HOOT_SERVICES=1 | ||
| HOOT_BUILD_JS_SCHEMA=0 | ||
| HOOT_BUILD_TRANSLATION_SERVER=0 | ||
| HOOT_BUILD_NODE_EXPORT_SERVER=0 | ||
| HOOT_SERVICES_HOST=core-services | ||
| HOOT_SERVICES_PORT=8080 | ||
| HOOT_NPM_INSTALL=0 | ||
| ``` | ||
| to set any properties from 0 to 1 and then stop/start the containers. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| UID=$(shell id -u) | ||
| GID=$(shell id -g) | ||
|
|
||
| DOCKER_VERSION := $(shell docker --version 2>/dev/null) | ||
|
|
||
| ifndef DOCKER_VERSION | ||
| $(error "command docker is not available, please install Docker") | ||
| endif | ||
|
|
||
| # The "new" version integrates compose in the docker command. | ||
| # It also fixes a bug with stopping log output when the "--follow" option is set | ||
| COMPOSE_COMMAND=docker compose | ||
|
|
||
| DOCKER_COMPOSE_NEW := $(shell docker compose version 2>/dev/null) | ||
| ifndef DOCKER_COMPOSE_NEW | ||
| DOCKER_COMPOSE_OLD := $(shell docker-compose --version 2>/dev/null) | ||
| ifdef DOCKER_COMPOSE_OLD | ||
| COMPOSE_COMMAND = docker-compose | ||
| else | ||
| $(error "docker compose is not available, please install it") | ||
| endif | ||
| endif | ||
|
|
||
| hoot.env: | ||
| cp hoot.env.example hoot.env | ||
|
|
||
| .env: hoot-ui-2x/README.md | ||
| cp .env_versions .env | ||
| mkdir -p bin lib .ccache | ||
| echo "HOOT_UID=$(UID)" >> .env | ||
| echo "HOOT_GID=$(GID)" >> .env | ||
| echo "HOOT_USER=hoot" >> .env | ||
| echo "HOOT_GROUP=hoot" >> .env | ||
| echo "HOOT_HOME=/var/lib/hootenanny" >> .env | ||
|
|
||
| conf/database/DatabaseConfigLocal.sh: | ||
| touch conf/database/DatabaseConfigLocal.sh | ||
| chmod +x conf/database/DatabaseConfigLocal.sh | ||
| echo "#!/bin/bash;" >> conf/database/DatabaseConfigLocal.sh | ||
| echo "set -euo pipefail;" >> conf/database/DatabaseConfigLocal.sh | ||
| echo "export DB_HOST=postgres" >> conf/database/DatabaseConfigLocal.sh | ||
| echo "export DB_HOST_OSMAPI=postgres" >> conf/database/DatabaseConfigLocal.sh | ||
|
|
||
| hoot-ui-2x/README.md: | ||
| git submodule update --init | ||
|
|
||
| build: .env hoot.env conf/database/DatabaseConfigLocal.sh hoot-ui-2x/README.md | ||
| DOCKER_BUILDKIT=1 $(COMPOSE_COMMAND) build --no-cache | ||
|
|
||
| up: .env hoot.env conf/database/DatabaseConfigLocal.sh hoot-ui-2x/README.md | ||
| DOCKER_BUILDKIT=1 $(COMPOSE_COMMAND) up -d | ||
|
|
||
| down: .env hoot.env | ||
| DOCKER_BUILDKIT=1 $(COMPOSE_COMMAND) down | ||
|
|
||
| distclean: .env hoot.env | ||
| DOCKER_BUILDKIT=1 $(COMPOSE_COMMAND) down --volumes --rmi all | ||
| rm -fr .ccache lib bin \ | ||
| .env hoot.env hoot-ui-2x/node_modules \ | ||
| conf/database/DatabaseConfigLocal.sh | ||
|
|
||
| PHONY: \ | ||
| build \ | ||
| distclean \ | ||
| down \ | ||
| up |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| version: "3.7" | ||
| x-args: &default-args | ||
| hoot_uid: "${HOOT_UID}" | ||
| hoot_gid: "${HOOT_GID}" | ||
| hoot_user: "${HOOT_USER}" | ||
| hoot_group: "${HOOT_GROUP}" | ||
| hoot_home: "${HOOT_HOME}" | ||
| postgresql_version: "${POSTGRESQL_VERSION}" | ||
| glpk_version: "${GLPK_VERSION}" | ||
| liboauthcpp_version: "${LIBOAUTHCPP_VERSION}" | ||
| libphonenumber_version: "${LIBPHONENUMBER_VERSION}" | ||
| libpostal_version: "${LIBPOSTAL_VERSION}" | ||
| node_version: "${NODE_VERSION}" | ||
| npm_version: "${NPM_VERSION}" | ||
| v8_version: "${V8_VERSION}" | ||
| stxxl_version: "${STXXL_VERSION}" | ||
| armadillo_version: "${ARMADILLO_VERSION}" | ||
| gdal_version: "${GDAL_VERSION}" | ||
| geos_version: "${GEOS_VERSION}" | ||
| libgeotiff_version: "${LIBGEOTIFF_VERSION}" | ||
| proj_version: "${PROJ_VERSION}" | ||
| devtoolset_version: "${DEVTOOLSET_VERSION}" | ||
| stxxl_version: "${STXXL_VERSION}" | ||
| jdk_version: "${JDK_VERSION}" | ||
| services: | ||
| frontend: | ||
| build: | ||
| context: . | ||
| dockerfile: docker/Dockerfile.frontend | ||
| args: *default-args | ||
| hostname: frontend | ||
| ports: | ||
| - 8080:8080 | ||
| env_file: | ||
| - hoot.env | ||
| volumes: | ||
| - ./hoot-ui-2x:${HOOT_HOME}/hoot-ui-2x:rw | ||
| - ./.git/modules/hoot-ui-2x:${HOOT_HOME}/.git/modules/hoot-ui-2x:rw | ||
| postgres: | ||
| build: | ||
| context: . | ||
| dockerfile: docker/Dockerfile.postgres | ||
| args: *default-args | ||
| hostname: postgres | ||
| env_file: | ||
| - hoot.env | ||
| volumes: | ||
| - postgres-data:/var/lib/pgsql/${POSTGRESQL_VERSION}/data:rw | ||
| core-services: | ||
| depends_on: | ||
| postgres: | ||
| condition: service_healthy | ||
| frontend: | ||
| condition: service_healthy | ||
| build: | ||
| context: . | ||
| dockerfile: docker/Dockerfile.core-services | ||
| args: *default-args | ||
| hostname: core-services | ||
| env_file: | ||
| - hoot.env | ||
| ports: | ||
| - 8888:8080 | ||
| - 8094:8094 | ||
| - 8096:8096 | ||
| - 8101:8101 | ||
| - 8000:8585 | ||
| volumes: | ||
| - ./:${HOOT_HOME}:rw | ||
| - tomcat8-hoot-services:/var/lib/tomcat8/webapps/hoot-services:rw | ||
|
|
||
| volumes: | ||
| postgres-data: | ||
| tomcat8-hoot-services: |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bmarchant I made this change because we need
HOOT_VERSION_GENto always run. I had whacked aFORCEtarget that I didn't consider the impact of here cbbaa64 that I think necessitated your change that I'm now fixing with.PHONY.For some reason the
make archivestep was failing in some iterations where I tried to restoreFORCEand in other iterations it was leading to an infinite loop callingHOOT_VERSION_GEN.This configuration seems to work in forcing the regeneration of VersionDefines.h on every build without the above defect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will regenerating
VersionDefines.hevery timemakeis run cause a major rebuild? If so, this would slow down compilation during development of Hootenanny outside of docker containers. Maybe somewhere in the docker container the change could be made usingsedor something like that.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think with the older
FORCEtarget this should have also been the case. But theHOOT_VERSION_GENscript only regeneratesVersionDefines.hif the version has actually changed so I think it should be ok. If it has changed that would necessitate a core rebuild of the three cpp classes that include it.