Skip to content

Commit

Permalink
Update LLI container (#2)
Browse files Browse the repository at this point in the history
Updates the LLI container and its build process to integrate with updates to lli-infra. This includes:
- Extended parameterization including IdP configuration
- Changes build and boot processes

Additionally updates READMEs.
  • Loading branch information
robinsaxifrage-agr-gc authored Sep 5, 2024
1 parent d870dd9 commit 99a4ad3
Show file tree
Hide file tree
Showing 7 changed files with 247 additions and 141 deletions.
6 changes: 3 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,6 @@
[submodule "apps/cdsp"]
path = apps/cdsp
url = https://gitlab.com/aafc/cdsp.git
[submodule "apps/user_saml"]
path = apps/user_saml
url = https://gitlab.com/aafc/cdsp-saml.git
[submodule "apps/activity"]
path = apps/activity
url = https://github.com/nextcloud/activity.git
Expand All @@ -19,3 +16,6 @@
[submodule "apps/notifications"]
path = apps/notifications
url = https://github.com/nextcloud/notifications.git
[submodule "apps/user_saml"]
path = apps/user_saml
url = https://github.com/aafc-lli/cdsp-saml
18 changes: 12 additions & 6 deletions .lli/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,11 @@ RUN echo \
$PHP_VER-redis \
| xargs apt-get install -y

# Install sudo for running commands as www-data.
RUN apt-get install -y sudo
# Install:
# - sudo for running commands as www-data.
# - postgresql-client for configuration loading.
# - curl for self-init requests.
RUN apt-get install -y sudo postgresql-client curl

# Set up Nginx.
RUN apt-get install -y nginx
Expand All @@ -108,14 +111,17 @@ RUN ln -sf /dev/stdout /var/log/nginx/access.log && \
# Populate service directory.
COPY --from=build_run /ncloud /ncloud

# Chown app.
RUN chown -R www-data:www-data /ncloud/server

# Load configs and boot script.
COPY container/fpm.conf /etc/php/$PHP_VER_NUM/fpm/php-fpm.conf
COPY container/fpm-pool.conf /etc/php/$PHP_VER_NUM/fpm/pool.d/www.conf
COPY container/nginx.conf nginx.conf.template
COPY container/nginx.template.conf nginx.template.conf
COPY container/boot.sh boot.sh
COPY container/post-install.sh post-install.sh
COPY container/local-update.sh local-update.sh

RUN chown -R www-data:www-data /ncloud/server
# Init supplimentary log.
RUN touch /ncloud/ncloud.log && \
chown www-data:www-data /ncloud/ncloud.log

CMD ./boot.sh
88 changes: 2 additions & 86 deletions .lli/README.md
Original file line number Diff line number Diff line change
@@ -1,75 +1,4 @@
# Local Quickstart

You must have the following dependencies installed and in your `PATH`:
* Docker
* Git + Git Bash
* JQ

First, ensure the `lli-infra` repo is cloned next to our NextCloud `server` repo:

```
<any directory>
lli-infra/
server/
```

Always use Git Bash when working with this NextCloud setup. In VSCode, you can open a Git Bash terminal by:
- Press Ctrl + ~ to open the terminal panel
- Click the caret next to the "+" icon at the top of the terminal panel
- Select "Git Bash"

All scripts should be run from this directory:

```bash
cd ncloud/docker/modern
```

To start NextCloud locally, run:

```bash
./local.sh up then-init
```

This builds and starts a Docker Compose of NextCloud and supporting services, then automatically initializes the database and triggers the NextCloud install.

NextCloud will now be installed and running at http://localhost. The default administrator credentials are:

```bash
Username: admin
Password: admin
```

After making a code change in your local repos, run the following to apply the changes to the deployment.

```bash
./local.sh update
```

If the change involves Vue or other built JS code, add the `rebuild-js` flag:

```bash
./local.sh update rebuild-js
```

To shut the deployment down, run:

```bash
./local.sh down
```

Or, if you want to retain all the data for next time:

```bash
./local.sh down save-vols
```

This will shut down and destroy all containers, but leave volumes in place. If you shut down this way, you do not need to re-initialize the next time you run NextCloud, so start it with simply:

```bash
./local.sh up
```

# Container
# LLI NextCloud Container

This container uses Nginx as a webserver and PHP Fast Process Manager (FPM) as the PHP executor upstream.

Expand All @@ -86,17 +15,4 @@ The filesystem layout is:
/etc/nginx/nginx.conf Nginx config.
```

The required runtime environment variables are:

```
NGINX_PORT The port number Nginx should listen on.
NGINX_INTERNAL_HOST The internal URL on which the site is hosted, without the protocol.
NGINX_EXTERNAL_HOST The external URL on which the site is hosted, without the protocol.
NGINX_PATH_PREFIX The URL prefix NextCloud lives on, on the hostname.
CDSP_UNASSIGNED_GROUPS A comma-separated list of groups new users are assigned to.
CDSP_RESTRICTED_GROUPS A comma-separated list of groups whose members logins are restricted by realm.
CDSP_ADMIN_GROUPS A comma-separated list of groups whose members are considered administrators.
AWS_ACCESS_KEY_ID Access Key for the AWS credential.
AWS_SECRET_ACCESS_KEY Secret Key for the AWS credential.
AWS_BUCKET AWS Bucket name.
```
[TODO: Expand doc.]
138 changes: 120 additions & 18 deletions .lli/container/boot.sh
Original file line number Diff line number Diff line change
@@ -1,27 +1,129 @@
#!/bin/bash
set -e

# Boot script for ncloud container.
# Boot script. Installs, launches, and configures NextCloud.

# Check for existing config.php.
chown -R www-data:www-data /ncloud/conf
if [[ -f /ncloud/conf/config.php ]]; then
echo "Pulling existing config.php..."
cp /ncloud/conf/config.php /ncloud/server/config/config.php
fi
# ---- App configuration.
disable_apps=(
"dashboard"
"user_status"
"files_reminders"
"comments"
)
enable_apps=(
# Default.
"admin_audit"
"files_external"
# Third-party.
"activity"
"announcementcenter"
"notifications"
"user_saml"
# First-party.
"cdsp"
)
# ----

# De-template Nginx config.
cat nginx.conf.template | \
sed -E s/__INTERNAL_HOST/$NGINX_INTERNAL_HOST/ | \
sed -E s/__EXTERNAL_HOST/$NGINX_EXTERNAL_HOST/ | \
sed -E s/__LISTEN_PORT/$NGINX_PORT/ | \
sed -E s#__PATH_PREFIX#$NGINX_PATH_PREFIX# \
> /etc/nginx/nginx.conf
rm nginx.conf.template
protocol=http
service_url=http://localhost
postgres_conn_str=postgresql://$POSTGRES_USER:$POSTGRES_PW@$POSTGRES_HOST/$POSTGRES_DB
if [[ $LLI_ENV != "local" ]]; then
protocol=https
service_url=https://$NCLOUD_HOST
fi

# Update data directory ownership. Performed on boot since this is a volume.
echo "Chowning data mount..."
chown -R www-data:www-data /ncloud/data

# Boot.
echo "Configuring Nginx..."
cat nginx.template.conf | \
sed -E s/__NCLOUD_HOST/$NCLOUD_HOST/ | \
sed -E s/__LISTEN_PORT/$NGINX_PORT/ \
> /etc/nginx/nginx.conf

echo "Booting..."
nginx & /usr/sbin/php-fpm8.1 -F -R
nginx &
nginx_pid=$!
/usr/sbin/php-fpm8.1 -R

# On local, Postgres needs to be initialized before we can continue.
echo "Awaiting Postgres..."
postgres_up=0
while ! psql $postgres_conn_str -c '\q' > /dev/null 2>&1; do
printf "."
sleep 2
done
echo

echo "Triggering install..."
curl \
-sS \
-X POST \
-H "Content-Type: multipart/form-data" \
-F install=true \
-F adminlogin=$ADMIN_USER \
-F adminpass=$ADMIN_PW \
-F directory=/ncloud/data \
-F dbtype=pgsql \
-F dbuser=$POSTGRES_USER \
-F dbpass=$POSTGRES_PW \
-F dbpass-clone=$POSTGRES_PW \
-F dbname=$POSTGRES_DB \
-F dbhost=$POSTGRES_HOST \
$service_url

echo "Configuring enabled apps..."
cd server

exec_occ() {
sudo -u www-data php occ "$@"
}

printf "%s\n" "${disable_apps[@]}" | xargs -I {} sudo -u www-data php occ app:disable {}
printf "%s\n" "${enable_apps[@]}" | xargs -I {} sudo -u www-data php occ app:enable --force {}

echo "Applying configuration..."
if [[ $NCLOUD_DEBUG == "1" ]]; then
exec_occ config:system:set debug --value="true"
fi

exec_occ config:system:set log_type --value="file"
exec_occ config:system:set logfile --value="/ncloud/ncloud.log"
exec_occ config:system:set loglevel --value="0" # TODO: Inject config.

exec_occ config:system:set redis host --value="$REDIS_HOST"
exec_occ config:system:set redis port --value="$REDIS_PORT"

exec_occ config:system:set memcache.local --value="\OC\Memcache\Redis"
exec_occ config:system:set memcache.distributed --value="\OC\Memcache\Redis"
exec_occ config:system:set memcache.locking --value="\OC\Memcache\Redis"

exec_occ config:system:set trusted_domains 1 --value="$NCLOUD_HOST"
exec_occ config:system:set overwriteprotocol --value="$protocol"
exec_occ config:system:set overwritewebroot --value="$NCLOUD_PATH_PREFIX"
exec_occ config:system:set overwritehost --value="$NCLOUD_HOST"
exec_occ config:system:set overwrite.cli.url --value="$service_url"
exec_occ config:system:set theme --value="cdsp-theme"

exec_occ config:app:set user_saml general-allow_multiple_user_back_ends --value="1" # todo rm this
exec_occ config:app:set user_saml type --value "saml"

saml_conf_json="$(echo "$SAML_CONF_B64" | base64 -d)"
cat <<EOF___ | psql $postgres_conn_str
DELETE FROM oc_user_saml_configurations;
INSERT INTO oc_user_saml_configurations (id, name, configuration)
VALUES (1, 'Primary IdP', '$saml_conf_json');
EOF___

exec_occ config:app:set cdsp restrictedgroups --value="$CDSP_RESTRICTED_GROUPS"
exec_occ config:app:set cdsp unassignedgroups --value="$CDSP_UNASSIGNED_GROUPS"
exec_occ config:app:set cdsp admingroups --value="$CDSP_ADMIN_GROUPS"

exec_occ config:app:set cdsp awsaccesskey --value="$AWS_ACCESS_KEY_ID"
exec_occ config:app:set cdsp awssecretkey --value="$AWS_SECRET_ACCESS_KEY"
exec_occ config:app:set cdsp awsbucket --value="$AWS_BUCKET"
exec_occ config:app:set cdsp awsregion --value="ca-central-1"

echo "Up."
tail -f /ncloud/ncloud.log
Loading

0 comments on commit 99a4ad3

Please sign in to comment.