This post replaces my earlier post 021 with the same title. It is intended to chronicle my efforts to build a new ISLE v1.3.0 local development instance of Digital.Grinnell on my work-issued MacBook Air, MA7053.

Goal

The goal of this project is to spin up a pristine, local Islandora stack using the ISLE project at https://github.com/Digital-Grinnell/dg-isle/, then introduce elements like the Digital Grinnell theme and custom modules like DG7. Once these pieces are in-place and working, I’ll begin adding other critical components as well as a robust set of data gleaned from https://digital.grinnell.edu/.

Using This Document

There are just a couple of notes regarding this document that I’d like to pass along to make it more useful.

• Gists - You will find a few places in this post where I generated a gist to take the place of lengthy command output. Instead of a long stream of text you’ll find a simple link to a gist like this.

• Workstation Commands - There are lots of places in this document where I’ve captured a series of command lines along with output from those commands in block text. Generally speaking, after each such block you will find a Workstation Commands table that can be used to conveniently copy and paste the necessary commands directly into your workstation. The tables look something like this:

• Apache Container Commands - Similar to Workstation Commands, a tabulated list of commands may appear with a heading of Apache Container Commands. *Commands in such tables can be copied and pasted into your command line terminal, but ONLY after you have opened a shell into the Apache container. The asterisk (*) at the end of the table heading is there to remind you of this! See the next section of this document for additional details. These tables looks something like this:

Opening a Shell in the Apache Container

This is something I find myself doing quite often during ISLE configuration, so here’s a reminder of how I generally do this…

Cloning to Local

The first step is to clone my fork of ISLE, namely dg-isle, to my workstation at ~/Projects/GitHub/dg-isle, checkout the local-dg-fedora branch there, and begin, like so…

Cleaning Up

I typically use the following command stream to clean up any Docker cruft before I begin anew. Note: Uncomment the third line ONLY if you want to delete images and download new ones. If you do, be patient, it could take several minutes depending on connection speed.

Connecting to FEDORA

The docker-compose.override.yml file in the local-dg-fedora branch of my dg-isle project includes 3 lines that direct FEDORA and FGSearch to use the mounted and pre-configured /Volumes/DG-FEDORA USB stick for object storage. The commands and process required to use the USB stick are presented in post 046, “DG-FEDORA: A Portable Object Repository”.

Restarting the Stack

Moving to Step 7 in the install documentation produced this gist.

Running the Drupal Installer Script

Moving on to Step 8 according to the documentation…

It was at this point I discovered a new gem in iTerm2: If you hit Command + shift + A the terminal will select/highlight all of the output from the last command. Exactly what I was hoping for. I’ve copied all that output and stuck it in this gist rather than pasting it all here.

Testing the Site

Moving on to Step 9 in the documentation…

A web browser visit to https://dg.localdomain/ shows that the standard ISLE stack is working~, and I was able to successfully login as admin with my super-secret password.

Installing the DG Theme

Step 10 in the documentation calls for ingest of some sample objects, but this is where I depart from the script since I’ve done this a number of times before.

So my focus here turned to installing the digital_grinnell_bootstrap theme instead. Initially I did this with a pair of git clone... commands, but later in this process I’m tasked with saving my Islandora / Drupal code as-a-whole into a larger Git repository that will include these themes. Cloning a Git repository inside another can lead to significant workflow problems, so lets use Git submodules instead.

Success! The theme is in place and active on my dg.localdomain site. Just one more tweak here…

I visited #overlay=admin/appearance/settings/digital_grinnell_bootstrap and made sure ONLY the following boxes are checked:

• Logo
• Shortcut Icon
• Use the default logo
• Use the default shortcut icon

All other theme settings should be default values and need not be changed.

A visit to the site with a refresh showed that this worked!

Install the Islandora Multi-Importer (IMI)

It’s important that we take this step BEFORE any that follow, otherwise the module will not install properly for reasons unknown. Note that I’m installing a Digital.Grinnell-specific fork of the module here, and the process is this:

Install the Missing Backup and Migrate Module

The Backup and Migrate module will be needed to quickly get our new ISLE configured as we’d like. Install it like so:

Backup and Restore the Database Using Backup and Migrate

From the https://digital.grinnell.edu production site…

• Login as System Admin
• From the Development menu (on the right) select Clear Cache
• On the home page (https://digital.grinnell.edu), scroll to the bottom of the right-hand column.
• Use the Quick Backup dialog, with all the defaults, to create and download a fresh backup.

Alternatively, you could…

• Navigate to https://digital.grinnell.edu/admin/config/system/backup_migrate/export/advanced
• In the Load Settings box select Default Settings w/ Users
• Click Backup now to backup the site
• Click Save to save the file to your workstation Downloads folder

In the https://dg.localdomain site…

• Visit #overlay=admin/config/system/backup_migrate/restore
• Click the Restore tab
• Select the Restore from an uploaded file option
• Click Browse in the Upload a Backup File
• Navigate to your workstation Downloads folder and choose the backup file created moments ago
• Click Restore now
• Navigate your browser back to https://dg.localdomain/
• Take note of any warnings or errors that may be present.

Restore Results…Lots of Warnings

OK, so when I did all of the above backup/restore process what I got back in the “Navigate your browser…” step was an unreadable host of warnings. Without panic I very calmly returned to my terminal and the shell open in the Apache container and:

This returned a number of warnings about missing modules and such. No matter, that was to be expected and the full list of warnings is captured in this gist.

The remedy for most of these missing bits was to do the following while still in my open Apache terminal/shell:

Visiting the site again shows that most of the Drupal missing modules are happy now, but there are still a number of Islandora bits missing, so I was left with the output as shown in this gist.

Next steps and sections, still working “off-script”, will install all of these missing parts.

Installing the Missing Islandora and Custom Modules

If I recall correctly, all of the missing Islandora and custom modules listed above can be found in the Apache container on DGDocker1, our production instance of ISLE, at /var/www/html/sites/all/modules/islandora. So I started this process by opening a new shell in the aforementioned container on DGDocker1 like so:

╭─mcfatem@dgdocker1 ~
╰─$ docker exec -it isle-apache-dg bash
root@90ae0691e764:/# cd /var/www/html/sites/all/modules/islandora
root@90ae0691e764:/var/www/html/sites/all/modules/islandora# l
dg7/                                    islandora_multi_importer/            islandora_solution_pack_compound/
idu/                                    islandora_oai/                       islandora_solution_pack_disk_image/
islandora/                              islandora_object_lock/               islandora_solution_pack_entities/
islandora_bagit/                        islandora_ocr/                       islandora_solution_pack_image/
islandora_batch/                        islandora_openseadragon/             islandora_solution_pack_large_image/
islandora_binary_object/                islandora_paged_content/             islandora_solution_pack_newspaper/
islandora_book_batch/                   islandora_pathauto/                  islandora_solution_pack_oralhistories/
islandora_bookmark/                     islandora_pdfjs/                     islandora_solution_pack_pdf/
islandora_checksum/                     islandora_pdfjs_reader/              islandora_solution_pack_video/
islandora_checksum_checker/             islandora_premis/                    islandora_solution_pack_web_archive/
islandora_collection_search/            islandora_scholar/                   islandora_sync/
islandora_context/                      islandora_simple_workflow/           islandora_videojs/
islandora_feeds/                        islandora_solr_collection_view/      islandora_webform/
islandora_fits/                         islandora_solr_facet_pages/          islandora_xacml_editor/
islandora_image_annotation/             islandora_solr_metadata/             islandora_xml_forms/
islandora_importer/                     islandora_solr_search/               islandora_xmlsitemap/
islandora_internet_archive_bookreader/  islandora_solr_views/                islandora_xquery/
islandora_jwplayer/                     islandora_solution_pack_audio/       objective_forms/
islandora_marcxml/                      islandora_solution_pack_book/        php_lib/

My recollection has been confirmed! So the next step was to visit each missing module’s folder to see what git remote each is tied to, like so:

root@90ae0691e764:/var/www/html/sites/all/modules/islandora# cd dg7; git remote -v
origin	https://github.com/DigitalGrinnell/dg7.git (fetch)
origin	https://github.com/DigitalGrinnell/dg7.git (push)
root@90ae0691e764:/var/www/html/sites/all/modules/islandora/dg7# cd ../idu; git remote -v
origin	https://github.com/DigitalGrinnell/idu.git (fetch)
origin	https://github.com/DigitalGrinnell/idu.git (push)
root@90ae0691e764:/var/www/html/sites/all/modules/islandora/idu# cd ../islandora_binary_object/; git remote -v
origin	git://github.com/discoverygarden/islandora_binary_object.git (fetch)
origin	git://github.com/discoverygarden/islandora_binary_object.git (push)
root@90ae0691e764:/var/www/html/sites/all/modules/islandora/islandora_binary_object# cd ../islandora_collection_search/; git remote -v
origin	https://github.com/discoverygarden/islandora_collection_search (fetch)
origin	https://github.com/discoverygarden/islandora_collection_search (push)
root@90ae0691e764:/var/www/html/sites/all/modules/islandora/islandora_collection_search# cd ../islandora_mods_display/; git remote -v
origin	https://github.com/DigitalGrinnell/islandora_mods_display.git (fetch)
origin	https://github.com/DigitalGrinnell/islandora_mods_display.git (push)
root@90ae0691e764:/var/www/html/sites/all/modules/islandora/islandora_mods_display# cd ../islandora_solution_pack_oralhistories/; git remote -v
origin	https://github.com/Islandora-Labs/islandora_solution_pack_oralhistories.git (fetch)
origin	https://github.com/Islandora-Labs/islandora_solution_pack_oralhistories.git (push)
root@90ae0691e764:/var/www/html/sites/all/modules/islandora/islandora_solution_pack_oralhistories# cd ../islandora_pdfjs_reader/; git remote -v
origin	git://github.com/nhart/islandora_pdfjs_reader.git (fetch)
origin	git://github.com/nhart/islandora_pdfjs_reader.git (push)
root@90ae0691e764:/var/www/html/sites/all/modules/islandora/islandora_pdfjs_reader# cd ../islandora_solr_collection_view/; git remote -v
origin	https://github.com/Islandora-Labs/islandora_solr_collection_view.git (fetch)
origin	https://github.com/Islandora-Labs/islandora_solr_collection_view.git (push)

Note that I did NOT bother with the islandora_multi_importer (IMI) directory since I know for a fact that IMI requires installation via Composer. I also didn’t bother looking for transcript_ui because it is a known sub-module of islandora_solution_pack_oralhistories.

It looks like all of the others can just be added as Git submodules like so:

The chown command line above was required to bring ALL of the new modules’ ownership into line with everything else in dg.localdomain. Also note that two of the lines, for islandora_binary_object and islandora_pdfjs_reader, are commented out because of known issues with installation of those modules.

Temporarily Eliminate Warnings

So my site, https://dg.localdomain/, is still issuing a few annoying warnings about missing pieces. It’s a safe bet that we don’t need these modules, at least not right now, so just do this:

You saw that correctly, I did drush cc all twice, just for good measure. Now, just a couple more issues to deal with…

Need a private File System

At this point the system is still issuing some warnings, and the most annoying is:

Warning: file_put_contents(private:///.htaccess): failed to open stream: "DrupalPrivateStreamWrapper::stream_open" call failed in file_put_contents() (line 496 of /var/www/html/includes/file.inc).

A visit in my browser to https://dg.localdomain/#overlay=admin/reports/status helps to pinpoint the problem… we don’t yet have a private file system. Let’s create one like so:

Now that same status report, https://dg.localdomain/#overlay=admin/reports/status, shows that we are still operating in maintenance mode, and some of our newest modules may require database updates. To remedy those two conditions:

Solr Schema is NOT Right

Ok, at this point I believe that I have a good Drupal database, a working Fedora repository, and a solid Islandora/Drupal codebase; but the Solr schema associated with this configuration is NOT up-to-speed with Digital.Grinnell’s so the dg7 code and the dg7_collection view are expecting Solr fields that do not yet exist here. What to do?

1. Save the current database using Backup and Migrate.
2. Save the current codebase to the host using mkdir -p ../dg-islandora; docker cp isle-apache-ld:/var/www/html/. ../dg-islandora and putting all of it into a new Digital-Grinnell/dg-islandora repository on GitHub for safe-keeping.
3. Update the FEDORA and Solr schema and configuration using the guidance found in https://github.com/Digital-Grinnell/ISLE-DG-Essentials/blob/master/README.md
4. Try pulling up the site again.

Huzzah! It works!

Final Step…Capture the Working Code in dg-islandora

To wrap this up I followed Step 11 in the install-local-new.md document to capture the state of my Islandora/Drupal code. In doing so I created my PRIVATE code repository, dg-islandora.

And that’s a wrap. Until next time…