Migrating Digital.Grinnell (DG) to ISLE 1.2.0 (ld) for Local Development
This post is a follow-up to previous post 034 where I successfully completed a “local” build of ISLE v1.2.0, but did no “customization” of that local instance. So, this post’s intent is to complete the goal stated in post 034, specifically to:
The goal of this project is to spin up a pristine, local Islandora stack using an updated fork of the ISLE project at https://github.com/DigitalGrinnell/dg-isle, then introduce elements like the Digital Grinnell theme and custom modules like DG7. Once these pieces are in-place and working, I’ll begin adding other critical components as well as a robust set of data gleaned from https://digital.grinnell.edu.
As before, this effort will involve an
local development, instance of Digital.Grinnell on one of my Mac workstations. Unlike my previous work, this instance will follow the guidance of a different document, specifically
Before beginning this process I need to get my Github environment updated by synchronizing my ISLE fork with the canonical copy. I followed this workflow to do so, like this:
git clone https://github.com/McFateM/ISLE.git
git remote add upstream https://github.com/Islandora-Collaboration-Group/ISLE.git
git fetch upstream
git pull upstream master
Nice! In case you haven’t seen it before, the last command in that sequence,
atom ., simply opens my
Atom editor to the new local instance of the
Installing per ISLE’s
This is first-and-foremost a
local development copy of ISLE, but with considerable Digital Grinnell customization, so I’m following the process outlined in the project’s
./docs/install/install-local-migrate.md. References to
Step X that follow refer to corresponding sections of https://github.com/Islandora-Collaboration-Group/ISLE/blob/master/docs/install/install-local-migrate.md. Each section or “Step” listed below is also a link back to the corresponding section of the canonical document.
Step 0: Copy Production Data to Your Local
Drupal Site Files & Code
Per the aforementioned guidance, I did this…
zsh shell (terminal) on Digital.Grinnell production node
DGDocker1 as user
ssh://firstname.lastname@example.org. Then, in that shell on the
DGDocker1 host I did:
mkdir -p migration-copy/var/www/html/sites/default/files
cd migration-copydocker cp isle-apache-dg:/var/www/html/sites/default/files/. var/www/html/sites/default/files/
Note that the
-p option on
mkdir is critical, it will create all, or port of, the entire directory tree as-specified, if it does not already exist. The final command,
docker cp..., subsequently takes advantage of this new directory tree and copies all of the Drupal
.../default/files from the production container to a similar directory on the host,
DGDocker1 in this case.
For the 2nd bullet item in this step, I’ve identified that my Drupal/Islandora customizations should eventually reside in the private repo that is https://github.com/McFateM/dg-islandora.
Drupal Site Database
For this process I already have a workflow in place, and it’s a little different than what’s documented. So here’s what I did:
- Visit my production site at https://digital.grinnell.edu.
- Login as the
System Admin, that’s
User 1or the super-user in Drupal terms.
- The home page at https://digital.grinnell.edu provides, in the right-hand menu bar, a
Managementmenu with a first option to
Clear cache. Click it.
- The home page also provides a
Quick Backupblock where the default options do a great job of backing up only what’s needed. Accept all defaults and click the
Backup Nowbutton. This feature takes the site offline, makes and downloads a backup of the database (in my case it created
digital.grinnell.edu-2019-08-13T15-51-20.mysql), and brings the site back online…“automagically”.
Fedora Hash Size (Conditional)
I found this section a little confusing, which I assume means that it does not apply to Digital.Grinnell. Moving on.
Solr Schema & Islandora Transforms
This section definitely applies to Digital.Grinnell! 😊 Since I recently completed spin-up of a “Demo” ISLE using
install-local-new.md, and since this is “not my first rodeo”, I choose to take the “advanced” path here and will “diff & merge current production customization edits into ISLE configs”. Wish me luck. 🍀
To begin this portion of the process I created a new
~/diff-and-merge-customizations directory structure and with https://isle.localdomain running the
Demo, I copied files from it like so:
|mkdir -p ~/diff-and-merge-customizations/ld|
mkdir -p ~/diff-and-merge-customizations/prod
docker cp isle-solr-ld:/usr/local/solr/collection1/conf/schema.xml schema.xml
docker cp isle-fedora-ld:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/foxmlToSolr.xslt foxmlToSolr.xslt
docker cp isle-fedora-ld:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/islandora_transforms islandora_transforms
I subsequently pulled the three diff targets from my production instance of ISLE using these commands on
DGDocker1, followed by another set on my workstation:
|mkdir -p ~/migration-copy/solr|
mkdir -p migration-copy/fedora
docker cp isle-solr-dg:/usr/local/solr/collection1/conf/schema.xml .
docker cp isle-fedora-dg:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/foxmlToSolr.xslt .
docker cp isle-fedora-dg:/usr/local/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/islandora_transforms islandora_transforms
rsync -aruvi email@example.com:migration-copy/solr/. .
rsync -aruvi firstname.lastname@example.org:migration-copy/fedora/. .
This left me with a local
~/diff-and-merge-customizations directory with sub-dirs
prod. The diff-and-merge operation subsequently compared the contents of these two sub-dirs, merging the differences from
prod into the corresponding files found in
ld. I did both the
diff and merge operations using
The command and output from the first
diff of the
╭─markmcfate@ma8660 ~ ╰─$ cd ~/diff-and-merge-customization/ld ╭─markmcfate@ma8660 ~/diff-and-merge-customization/ld ╰─$ diff -r islandora_transforms ../prod/islandora_transforms 1 ↵ Only in islandora_transforms: .git diff -r islandora_transforms/WORKFLOW_to_solr.xslt ../prod/islandora_transforms/WORKFLOW_to_solr.xslt 95c95 </xsl:stylesheet> \ No newline at end of file --- </xsl:stylesheet>
So, no significant differences here. Yay!
Next, I compared the two files, one-at-a-time, using
Atom and its
Split Diff package. There were very few differences in
foxmToSolr.xslt so it was easy to merge. I took the significant differences in the
prod copy and merged them into the
ld copy. I did the same with
schema.xml, but it was an entirely different beast. There were 34 sections of differences, and in many cases it was not clear if our existing customizations should be merged. As a result, the changes I made in
schema.xml, and some I did NOT make, will need to be carefully re-evaluated as this local migrate “test” proceeds.
At this point I’ve saved my “merged” customizations in
Step 2a, below.
Phew, that was a lot of Step 0!
Step 1: Edit
Easy peasy, relatively speaking. 😄
Step 2: Setup Git for the ISLE Project
OK, I already have two private Github repositories…
- https://gitubh.com/McFateM/dg-isle for my customized copy of the ISLE project, and
- https://github.com/McFateM/dg-islandora for my customized copy of the Drupal and Islandora code.
However, in my case neither of these repositories is “complete” because they are products of the
install-local-new.md process, so my focus in this step will be to get these two repos in line with the Git workflow that’s being established.
- I cloned it to my local workstation and set things up as directed with:
git clone https://github.com/McFateM/dg-isle.git
git remote add icg-upstream https://github.com/Islandora-Collaboration-Group/ISLE.git
git fetch icg-upstream
git pull icg-upstream master
git push -u origin master
Step 2a: Add Customizations from
Before executing the documented commands I elected to create a new branch in my local
dg-isle repo, so my workflow was this:
git checkoput -b solr-and-gsearch-customizations
mkdir -p ./config/solr
mkdir -p ./config/fedora/gsearch
cp -f ~/diff-and-merge-customizations/ld/schema.xml ./config/solr/
cp -f ~/diff-and-merge-customizations/ld/foxmlToSolr.xslt ./config/fedora/gsearch/
cp -fr ~/diff-and-merge-customizations/ld/islandora_transforms ./config/fedora/gsearch/
Next, I edited
docker-compose.local.yml as prescribed, and then saved it all like so:
git checkout solr-and-gsearch-customizations
git add -A
git commit -m “Customizations from Step 2a”
git push –set-upstream origin solr-and-gsearch-customizations
git clone the Production Drupal Site Code
OK, this section will deal with my Islandora/Drupal code repository with all my customization in it, in my case that’s https://github.com/McFateM/dg-islandora, but my copy isn’t complete yet, so…
Post 038 - Building My
dg-islandora Code Repository
I’m creating a new blog post that will become 038-Building-My-
dg-islandora-Code-Repository. It will combine elements of post 021 with post 034 to create a “customized” local ISLE v1.2.0 instance with features of Digital.Grinnell.
Once that process is complete, I’ll return here to continue
Step 3. Until then…