This post is a follow-up to previous post 034 where I successfully completed a “local” build of ISLE v1.2.0, but did no “customization” of that local instance. So, this post’s intent is to complete the goal stated in post 034, specifically to:

The goal of this project is to spin up a pristine, local Islandora stack using an updated fork of the ISLE project at https://github.com/DigitalGrinnell/dg-isle, then introduce elements like the Digital Grinnell theme and custom modules like DG7. Once these pieces are in-place and working, I’ll begin adding other critical components as well as a robust set of data gleaned from https://digital.grinnell.edu.

As before, this effort will involve an ld, or local development, instance of Digital.Grinnell on one of my Mac workstations. Unlike my previous work, this instance will follow the guidance of a different document, specifically install-local-migrate.md.

Fork Synchronization

Before beginning this process I need to get my Github environment updated by synchronizing my ISLE fork with the canonical copy. I followed this workflow to do so, like this:

Workstation Commands
cd ~/Projects
git clone https://github.com/McFateM/ISLE.git
cd ISLE
git remote add upstream https://github.com/Islandora-Collaboration-Group/ISLE.git
git fetch upstream
git pull upstream master
git push
atom .

Nice! In case you haven’t seen it before, the last command in that sequence, atom ., simply opens my Atom editor to the new local instance of the ISLE project.

Installing per ISLE’s install-local-migrate.md

This is first-and-foremost a local development copy of ISLE, but with considerable Digital Grinnell customization, so I’m following the process outlined in the project’s ./docs/install/install-local-migrate.md. References to Step X that follow refer to corresponding sections of https://github.com/Islandora-Collaboration-Group/ISLE/blob/master/docs/install/install-local-migrate.md. Each section or “Step” listed below is also a link back to the corresponding section of the canonical document.

Step 0: Copy Production Data to Your Local

Drupal Site Files & Code

Per the aforementioned guidance, I did this…

Opened a zsh shell (terminal) on Digital.Grinnell production node DGDocker1 as user islandora using iTerm2 at ssh://islandora@dgdocker1.grinnell.edu. Then, in that shell on the DGDocker1 host I did:

DGDocker1 Commands
cd ~
mkdir -p migration-copy/var/www/html/sites/default/files
cd migration-copy
docker cp isle-apache-dg:/var/www/html/sites/default/files/. var/www/html/sites/default/files/

Note that the -p option on mkdir is critical, it will create all, or port of, the entire directory tree as-specified, if it does not already exist. The final command, docker cp..., subsequently takes advantage of this new directory tree and copies all of the Drupal .../default/files from the production container to a similar directory on the host, DGDocker1 in this case.

For the 2nd bullet item in this step, I’ve identified that my Drupal/Islandora customizations should eventually reside in the private repo that is https://github.com/McFateM/dg-islandora.

Drupal Site Database

For this process I already have a workflow in place, and it’s a little different than what’s documented. So here’s what I did:

  • Visit my production site at https://digital.grinnell.edu.
  • Login as the System Admin, that’s User 1 or the super-user in Drupal terms.
  • The home page at https://digital.grinnell.edu provides, in the right-hand menu bar, a Management menu with a first option to Clear cache. Click it.
  • The home page also provides a Quick Backup block where the default options do a great job of backing up only what’s needed. Accept all defaults and click the Backup Now button. This feature takes the site offline, makes and downloads a backup of the database (in my case it created digital.grinnell.edu-2019-08-13T15-51-20.mysql), and brings the site back online…“automagically”.

Fedora Hash Size (Conditional)

I found this section a little confusing, which I assume means that it does not apply to Digital.Grinnell. Moving on.

Solr Schema & Islandora Transforms

This section definitely applies to Digital.Grinnell! 😊 Since I recently completed spin-up of a “Demo” ISLE using install-local-new.md, and since this is “not my first rodeo”, I choose to take the “advanced” path here and will “diff & merge current production customization edits into ISLE configs”. Wish me luck. 🍀

To begin this portion of the process I created a new ~/diff-and-merge-customizations directory structure and with https://isle.localdomain running the Demo, I copied files from it like so:

Workstation Commands
mkdir -p ~/diff-and-merge-customizations/ld
mkdir -p ~/diff-and-merge-customizations/prod
cd ~/diff-and-merge-customizations/ld
docker cp isle-solr-ld:/usr/local/solr/collection1/conf/schema.xml schema.xml
docker cp isle-fedora-ld:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/foxmlToSolr.xslt foxmlToSolr.xslt
docker cp isle-fedora-ld:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/islandora_transforms islandora_transforms

I subsequently pulled the three diff targets from my production instance of ISLE using these commands on DGDocker1, followed by another set on my workstation:

DGDocker1 Commands
mkdir -p ~/migration-copy/solr
mkdir -p migration-copy/fedora
cd migration-copy/solr
docker cp isle-solr-dg:/usr/local/solr/collection1/conf/schema.xml .
cd ../fedora
docker cp isle-fedora-dg:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/foxmlToSolr.xslt .
docker cp isle-fedora-dg:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/islandora_transforms islandora_transforms
Workstation Commands
cd ~/diff-and-merge-customizations/prod
rsync -aruvi islandora@dgdocker1.grinnell.edu:migration-copy/solr/. .
rsync -aruvi islandora@dgdocker1.grinnell.edu:migration-copy/fedora/. .

This left me with a local ~/diff-and-merge-customizations directory with sub-dirs ld and prod. The diff-and-merge operation subsequently compared the contents of these two sub-dirs, merging the differences from prod into the corresponding files found in ld. I did both the diff and merge operations using Atom.

The command and output from the first diff of the islandora_transforms was:

╭─markmcfate@ma8660 ~
╰─$ cd ~/diff-and-merge-customization/ld
╭─markmcfate@ma8660 ~/diff-and-merge-customization/ld
╰─$ diff -r islandora_transforms ../prod/islandora_transforms   1 ↵
Only in islandora_transforms: .git
diff -r islandora_transforms/WORKFLOW_to_solr.xslt ../prod/islandora_transforms/WORKFLOW_to_solr.xslt
95c95
</xsl:stylesheet>
\ No newline at end of file
---
</xsl:stylesheet>

So, no significant differences here. Yay!

Next, I compared the two files, one-at-a-time, using Atom and its Split Diff package. There were very few differences in foxmToSolr.xslt so it was easy to merge. I took the significant differences in the prod copy and merged them into the ld copy. I did the same with schema.xml, but it was an entirely different beast. There were 34 sections of differences, and in many cases it was not clear if our existing customizations should be merged. As a result, the changes I made in schema.xml, and some I did NOT make, will need to be carefully re-evaluated as this local migrate “test” proceeds.

At this point I’ve saved my “merged” customizations in ~/diff-and-merge-customizations/ld until Step 2a, below.

Phew, that was a lot of Step 0!

Step 1: Edit /etc/hosts File

Easy peasy, relatively speaking. 😄

Step 2: Setup Git for the ISLE Project

OK, I already have two private Github repositories…

However, in my case neither of these repositories is “complete” because they are products of the install-local-new.md process, so my focus in this step will be to get these two repos in line with the Git workflow that’s being established.

Regarding the dg-isle repo…

  • I cloned it to my local workstation and set things up as directed with:
Workstation Commands
cd ~/Projects
git clone https://github.com/McFateM/dg-isle.git
cd dg-isle
git remote add icg-upstream https://github.com/Islandora-Collaboration-Group/ISLE.git
git fetch icg-upstream
git pull icg-upstream master
git push -u origin master

Step 2a: Add Customizations from Step 0

Before executing the documented commands I elected to create a new branch in my local dg-isle repo, so my workflow was this:

Workstation Commands
cd ~/Projects/dg-isle
git checkoput -b solr-and-gsearch-customizations
mkdir -p ./config/solr
mkdir -p ./config/fedora/gsearch
cp -f ~/diff-and-merge-customizations/ld/schema.xml ./config/solr/
cp -f ~/diff-and-merge-customizations/ld/foxmlToSolr.xslt ./config/fedora/gsearch/
cp -fr ~/diff-and-merge-customizations/ld/islandora_transforms ./config/fedora/gsearch/

Next, I edited docker-compose.local.yml as prescribed, and then saved it all like so:

Workstation Commands
cd ~/Projects/dg-isle
git checkout solr-and-gsearch-customizations
git add -A
git commit -m “Customizations from Step 2a”
git push –set-upstream origin solr-and-gsearch-customizations

Step 3. git clone the Production Drupal Site Code

OK, this section will deal with my Islandora/Drupal code repository with all my customization in it, in my case that’s https://github.com/McFateM/dg-islandora, but my copy isn’t complete yet, so…

Post 038 - Building My dg-islandora Code Repository

I’m creating a new blog post that will become 038-Building-My-dg-islandora-Code-Repository. It will combine elements of post 021 with post 034 to create a “customized” local ISLE v1.2.0 instance with features of Digital.Grinnell.

Once that process is complete, I’ll return here to continue Step 3. Until then…