Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker updates #82

Merged
merged 10 commits into from
Jan 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 0 additions & 40 deletions .travis.yml

This file was deleted.

3 changes: 0 additions & 3 deletions Dockerfile

This file was deleted.

20 changes: 8 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# NLDI Database Setup

[![Build Status](https://travis-ci.org/ACWI-SSWD/nldi-db.svg?branch=master)](https://travis-ci.org/ACWI-SSWD/nldi-db)

This repository contains Liquibase scripts for creating the NLDI PostGIS database.

## Docker
Also included are Docker Compose scripts to:
* Create PostgreSQL and Liquibase containers for testing the scripts.
* Create a continuous integration PostgreSQL database container.
* Create a continuous integration (CI) PostgreSQL database container.
* Create a PostgreSQL database container for local development containing a sampling of data.

### Docker Network
Expand All @@ -18,9 +16,7 @@ docker network create --subnet=172.26.0.0/16 nldi
```

### Environment variables
In order to use the docker compose scripts, you will need to create a .env file in the project directory containing

the following (shown are example values):
In order to use the docker compose scripts, you will need to create a .env file in the project directory containing the following (shown are example values):

```
POSTGRES_PASSWORD=<changeMe>
Expand Down Expand Up @@ -98,30 +94,30 @@ The PostGIS database will be available on your localhost's port $DB_PORT, allowi

### CI Database
```
docker-compose up ciDB
docker-compose up ci
```
It will be available on you localhost's port $DB_CI_PORT

You can also pull the image from Docker Hub and run it with
You can also pull the image from the GitHub Package Repository and run it with

```
docker run -it --env-file ./.env -p 127.0.0.1:5445:5432 usgswma/wqp_db:ci
docker run -it --env-file ./.env -p 5445:5432 ghcr.io/acwi-sswd/nldi-db:ci-latest
```
where __./.env__ is the environment variable file you have locally and __5445__ can be changed to the port you wish to access it via.

### Demo Database

```
docker-compose up demoDB
docker-compose up demo
```

It will be available on your localhost's port $DB_DEMO_PORT


You can also pull the image from Docker Hub and run it with
You can also pull the image from the GitHub Package Repository and run it with

```
docker run -it --env-file ./.env -p 127.0.0.1:5446:5432/tcp usgswma/nldi-db:demo
docker run -it --env-file ./.env -p 5446:5432/tcp ghcr.io/acwi-sswd/nldi-db:demo-latest
```

where __./.env__ is the environment variable file you have locally and __5446__ can be changed to the port you wish to access it via.
Expand Down
14 changes: 13 additions & 1 deletion database/ci/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
FROM usgswma/iow_test_db:postgis11-jre11
ARG LIQUIBASE_DOCKER_VERSION=3.10
ARG POSTGIS_DOCKER_VERSION=12-3.0

FROM liquibase/liquibase:${LIQUIBASE_DOCKER_VERSION} as Liquibase

FROM postgis/postgis:${POSTGIS_DOCKER_VERSION} AS Postgres

LABEL [email protected]

ENV LIQUIBASE_HOME /liquibase
ENV LIQUIBASE_WORKSPACE /liquibase
COPY --from=Liquibase ${LIQUIBASE_WORKSPACE} ${LIQUIBASE_WORKSPACE}
ENV JAVA_HOME /usr/local/openjdk-11
COPY --from=Liquibase ${JAVA_HOME} ${JAVA_HOME}
RUN apt-get update && apt-get install -y curl

############################################
# Grab files for initializing the database
############################################
Expand Down
14 changes: 13 additions & 1 deletion database/demo/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
FROM usgswma/iow_test_db:postgis11-jre11
ARG LIQUIBASE_DOCKER_VERSION=3.10
ARG POSTGIS_DOCKER_VERSION=12-3.0

FROM liquibase/liquibase:${LIQUIBASE_DOCKER_VERSION} as Liquibase

FROM postgis/postgis:${POSTGIS_DOCKER_VERSION} AS Postgres

LABEL [email protected]

ENV LIQUIBASE_HOME /liquibase
ENV LIQUIBASE_WORKSPACE /liquibase
COPY --from=Liquibase ${LIQUIBASE_WORKSPACE} ${LIQUIBASE_WORKSPACE}
ENV JAVA_HOME /usr/local/openjdk-11
COPY --from=Liquibase ${JAVA_HOME} ${JAVA_HOME}
RUN apt-get update && apt-get install -y curl

############################################
# Grab files for initializing the database
############################################
Expand Down
35 changes: 16 additions & 19 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
version: '3.3'
services:

ciDB:
image: nldi_db_ci
ci:
image: nldi-db-ci
build:
context: .
dockerfile: ./database/ci/Dockerfile
Expand All @@ -23,10 +23,10 @@ services:
- NLDI_DATABASE_ADDRESS=127.0.0.1
ports:
- ${DB_CI_PORT}:5432
container_name: ${NLDI_DATABASE_ADDRESS}_CI
container_name: nldi-db-ci

demoDB:
image: nldi_db_demo
demo:
image: nldi-db-demo
build:
context: .
dockerfile: ./database/demo/Dockerfile
Expand All @@ -47,34 +47,31 @@ services:
- NLDI_DATABASE_ADDRESS=127.0.0.1
ports:
- ${DB_DEMO_PORT}:5432
container_name: ${NLDI_DATABASE_ADDRESS}_DEMO
container_name: nldi-db-demo

db:
image: nldi_db
build:
context: .
image: postgis/postgis:12-3.0
networks:
nldi:
ipv4_address: ${DB_IPV4}
aliases:
- nldi-db
environment:
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${NLDI_DATABASE_NAME}
ports:
- ${DB_PORT}:5432
container_name: ${NLDI_DATABASE_ADDRESS}
container_name: nldi-db

liquibase:
image: nldi_liquibase
image: nldi-liquibase
depends_on:
- db
build:
context: ./liquibase
args:
- LIQUIBASE_VERSION=${LIQUIBASE_VERSION}
- A_JDBC_JAR=${JDBC_JAR}
networks:
nldi:
ipv4_address: ${LIQUIBASE_IPV4}
aliases:
- nldi-liquibase
environment:
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${NLDI_DATABASE_NAME}
Expand All @@ -86,11 +83,11 @@ services:
- NHDPLUS_SCHEMA_OWNER_USERNAME=${NHDPLUS_SCHEMA_OWNER_USERNAME}
- NLDI_READ_ONLY_USERNAME=${NLDI_READ_ONLY_USERNAME}
- NLDI_READ_ONLY_PASSWORD=${NLDI_READ_ONLY_PASSWORD}
- NLDI_DATABASE_ADDRESS=${NLDI_DATABASE_ADDRESS}
- NLDI_DATABASE_ADDRESS=nldi-db # using the network alias from the db service
volumes:
- ./liquibase/changeLogs:/home/java/workspace/
- ./liquibase/changeLogs:/liquibase/workspace/
- ./liquibase/scripts/dbInit:/docker-entrypoint-initdb.d
container_name: nldi_liquibase
container_name: nldi-liquibase

networks:
nldi:
Expand Down
27 changes: 7 additions & 20 deletions liquibase/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,29 +1,16 @@
FROM usgswma/openjdk:debian-stretch-openjdk-11
ARG LIQUIBASE_DOCKER_VERSION=3.10

LABEL maintainer="[[email protected]](mailto:[email protected])"
FROM liquibase/liquibase:${LIQUIBASE_DOCKER_VERSION}

############################################
# Install Liquibase
############################################
LABEL maintainer="[[email protected]](mailto:[email protected])"

ARG LIQUIBASE_VERSION
ARG A_JDBC_JAR
ENV JDBC_JAR=$A_JDBC_JAR
ENV LIQUIBASE_HOME $HOME
ENV LIQUIBASE_HOME /liquibase
ENV LIQUIBASE_WORKSPACE $LIQUIBASE_HOME/workspace
ENV LOCALONLY "-c listen_addresses='127.0.0.1, ::1'"

ADD https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/wait-for-it.sh

ADD https://github.com/liquibase/liquibase/releases/download/liquibase-parent-$LIQUIBASE_VERSION/liquibase-$LIQUIBASE_VERSION-bin.tar.gz $LIQUIBASE_HOME/

ADD https://jdbc.postgresql.org/download/$JDBC_JAR $LIQUIBASE_HOME/lib/

RUN tar -xzf $LIQUIBASE_HOME/liquibase-$LIQUIBASE_VERSION-bin.tar.gz -C $LIQUIBASE_HOME/

RUN chmod -R 777 $LIQUIBASE_HOME

USER root
COPY ./docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
USER liquibase

ENTRYPOINT ["docker-entrypoint.sh"]
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ BEGIN
,pstartcomid
,pMaxDistanceKm
,pstopcomid
,(abstime(('now'::text)::timestamp(6) with time zone))
,now()
);

END IF;
Expand Down
7 changes: 6 additions & 1 deletion liquibase/changeLogs/postgres/nldi/changeLog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,13 @@ databaseChangeLog:
- rollback: drop extension if exists postgis_tiger_geocoder;

- changeSet:
author: drsteini
author: egrahn
id: "create.extension.plv8"
preConditions: # check if the plv8 extension is available to install
- onFail: CONTINUE
- sqlCheck:
expectedResult: 1
sql: select count(*) from pg_available_extensions where name = 'plv8'
changes:
- sql: create extension if not exists plv8;
- rollback: drop extension if exists plv8;
Expand Down