This post is a follow-up to Dockerized Traefik Host Using ACME DNS-01 Challenge and Staging ISLE Installation: Migrate Existing Islandora Site - with Annotations, specifically Step 11 in the later document. It introduces a Digital.Grinnell-specific implementation of the Traefik with Acme.sh.

Testing with McFateM/docker-traefik2-acme-host

I started work on this implementation with a test, by cloning https://github.com/McFateM/docker-traefik2-acme-host and proceeding as directed in the repository’s README.md document, as user islandora on node DGDockerX, like so:

Working in ~/host/acme

As suggested, I made a copy of the .env file from the corresponding acme directory on Grinnell’s dgdocker3.grinnell.edu server, something like this:

╭─islandora@dgdockerx ~/host/acme ‹master*›
╰─$ rsync -aruvi islandora@dgdocker3.grinnell.edu:/home/islandora/host/acme/.env . --progress
The authenticity of host 'dgdocker3.grinnell.edu (132.161.151.***)' can't be established.
ECDSA key fingerprint is SHA256:************************************************.
ECDSA key fingerprint is MD5:****************************************.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'dgdocker3.grinnell.edu,132.161.151.***' (ECDSA) to the list of known hosts.
islandora@dgdocker3.grinnell.edu's password:
receiving incremental file list
>f+++++++++ .env            896 100%  875.00kB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 43 bytes  received 1,011 bytes  39.77 bytes/sec
total size is 896  speedup is 0.85

I made the following edits/changes on the DGDocker3 host as suggested in the project repository README.md file, using nano.

Working in ~/host/traefik

As suggested in the documentation, I moved into the traefik directory and had a look at the README.md file there. Following it’s advice, like so:

╭─islandora@dgdockerx ~/host/acme ‹master*›
╰─$ cd ../traefik
╭─islandora@dgdockerx ~/host/traefik ‹master*›
╰─$ ll
total 12K
-rw-rw-r--. 1 islandora islandora 1.9K Sep 15 14:29 docker-compose.yml
-rw-rw-r--. 1 islandora islandora  673 Sep 15 14:29 README.md
-rw-rw-r--. 1 islandora islandora  442 Sep 15 14:29 traefik-tls.toml
╭─islandora@dgdockerx ~/host/traefik ‹master*›
╰─$ cat README.md
---
Host: Defined as ${HOST} in .env
Service: traefik
URL: https://${HOST}/dashboard/
---

This document should be used to launch the `traefik` service on ANY host as part of a `docker-traefik2-acme-host` stack.

> Note that this process should be started only AFTER the `acme` service!

## Preparation

Before entering the prescribed "Command Sequence", below, the user should take steps to copy any pertinent `.env` files from an existing deployment. Try something like this:

  - `rsync -aruvi administrator@static.grinnell.edu:/home/administrator/host/traefik/.env . --progress`

## Command Sequence

  - cd ~/host/traefik
  - docker-compose up -d; docker-compose logs
╭─islandora@dgdockerx /opt/containers/host/traefik ‹master*›
╰─$ rsync -aruvi administrator@static.grinnell.edu:/home/administrator/host/traefik/.env . --progress
administrator@static.grinnell.edu's password:
receiving incremental file list
>f+++++++++ .env
             25 100%   24.41kB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 43 bytes  received 140 bytes  11.09 bytes/sec
total size is 25  speedup is 0.14
╭─islandora@dgdockerx /opt/containers/host/traefik ‹master*›
╰─$ nano .env

Working in the Remaining Directories

I did the same as above in each of the remaining directories: portainer, watchtower, and whoami. Afterward, I returned to the ~/host directory and made necessary changes to restart.sh.

Ultimately the changes I made were intended to create the following services and addresses for testing purposes only:

Test Launching the Stack

With my test configured I should be able to launch the Traefik/Portainer/WhoAmI stack for testing by simply running the ~/host/destroy.sh script followed by ~/host/restart.sh. Initially when I did this I had configured restart.sh with an acme.sh command like this:

docker exec -it acme --issue --dns dns_azure --server https://acme-staging-v02.api.letsencrypt.org/directory -d dgdockerx.grinnell.edu -d isle-staging.grinnell.edu -d dg-staging.grinnell.edu --domain-alias _acme-challenge.leverify.info --key-file /certs/dgdockerx.grinnell.edu.key --cert-file /certs/dgdockerx.grinnell.edu.cert --standalone --force --log --debug 2

That acme command failed because it tried to create and validate a cert for three different subdomains, dgdockerx.grinnell.edu, isle-staging.grinnell.edu, and dg-staging.grinnell.edu. Validation like this requires that each target subdomain has three things in place:

• A valid A record in the Grinnell College external and/or internal DNS tables directing the subdomain to an appropriate service endpoint.
• A corresponding CNAME record in the college’s Azure DNS. acme uses this to generate TXT records that are subsequently used for validation.
• A working service endpoint capable of returning a valid response.

When I initiated that first test by running ~/host/restart.sh, none of my three subdomains had the necessary CNAME records and only dgdockerx.grinnell.edu had a working service endpoint. Naturally, that test failed, and I learned from subsequent tests that any error in the running of the acme script apparently negates the entire command. So, with that in mind, I am taking steps to limit each acme command I run to a single subdomain. For example:

docker exec -it acme --issue --dns dns_azure --server https://acme-staging-v02.api.letsencrypt.org/directory -d dgdockerx.grinnell.edu --domain-alias _acme-challenge.leverify.info --key-file /certs/dgdockerx.grinnell.edu.key --cert-file /certs/dgdockerx.grinnell.edu.cert --standalone --force --log --debug 2

docker exec -it acme --issue --dns dns_azure --server https://acme-staging-v02.api.letsencrypt.org/directory -d isle-staging.grinnell.edu --domain-alias _acme-challenge.leverify.info --key-file /certs/isle-staging.grinnell.edu.key --cert-file /certs/isle-staging.grinnell.edu.cert --standalone --force --log --debug 2

docker exec -it acme --issue --dns dns_azure --server https://acme-staging-v02.api.letsencrypt.org/directory -d dg-staging.grinnell.edu --domain-alias _acme-challenge.leverify.info --key-file /certs/dg-staging.grinnell.edu.key --cert-file /certs/dg-staging.grinnell.edu.cert --standalone --force --log --debug 2

Another Test Launch

Before attempting to engage the acme.sh validation scheme with my staging instance of ISLE, I elected to run another test with my whoami service and a new dgdockerx-landing-page service as well. I would do so using the one-subdomain-per-command approach mentioned above. This new test needs to introduce a new service and URL as well, so I elected to add a static site “landing page” to this server. The services and addresses I intend to create will include:

I am exceptionally pleased to report that… IT JUST WORKS. ❗ 😀 ❗ 😀 ❗

All of the updated files, sans any .env files needed for complete configuration, have been pushed back into the docker-traefik2-acme-host public repository. On September 28, 2020, I also took the liberty of upgrading the aforementioned project to use Portainer v2.0.0 as well as Traefik:latest (currently equates to 2.3.0) and docker-compose 3.3. All of those changes have also been pushed back and merged into master. Enjoy.

Next: Repeat with DG-STAGING (ISLE v1.5.1) at https://dg-staging.grinnell.edu

This could be a real challenge because docker-traefik2-acme-host uses Traefik v2 while ISLE still employ Traefik v1. Fortunately, the key component to making this work is the acme.sh script and service, and that should work with any version of Traefik. So let’s take some baby steps…

Introduce acme.sh Into ISLE

All that’s necessary to introduce acme.sh into ISLE is copying the current ~/host/acme directory to /opt/dg-isle/acme, changing a litte of the configuration there, and invoking the acme service to obtain our certs before we bring up the ISLE stack. Working on DGDockerX as user islandora I made the copy and edits like so:

cp -f ~/host/destroy.sh /opt/dg-isle/destroy.sh
cp -f ~/host/restart.sh /opt/dg-isle/restart.sh
cp -fr ~/host/acme /opt/dg-isle/acme
cd /opt/dg-isle
nano /opt/dg-isle/acme/docker-compose.yml
# In nano I changed all instances of the external network name from 'proxy' to 'isle-external', and our host '../certs:' directory to '../config/proxy/ssl-certs' in order to match ISLE conventions.
nano /opt/dg-isle/restart.sh
# In nano I changed the external network name from 'proxy' to 'isle-external' and modified directory names for other portions from `~/host` to `/opt/dg-isle`
# The next command will use the new `restart.sh` script to launch ISLE
./restart.sh

And that’s a wrap for this episode. Until next time…