About Interrupted Ingests...

A number of users sent in notes concerning interrupted ingests due to the emergency Drupal patch that needed to be deployed last Wednesday. The questions had the common theme of "What do I do when the objects I am ingesting are affected by restarting/shutting down the CTDA servers?" Below you will find our thoughts and some recommendations if you ever find yourself in this situation.

We find that the issue of interrupted ingests mainly impact content that is in the middle of a batch ingest, mostly page images of book and newspaper objects. Many users set the batch ingest to run automatically and do not return to the repository until the ingest is complete. Any type of server disruption could effect a large number of objects in a batch ingest. We use an example of a newspaper object below, but derviatives can be regenerated for all content types.

Instead of deleting all of objects of an interrupted batch and starting all over again, we recommend that you identify where the ingest ended and pick up your batch ingest at that point with the next object. If you need assistance in determining if any of your ingests were impacted or cut short, please email ctda@uconn.edu and we would be happy to help.

For book and newspaper objects, you can view which pages have been ingested into the repository by clicking on the "Pages" tab when viewing your book or newspaper objects. You may notice that some, or all, of your page files may display a blank silver folder instead of a thumbnail image of the page.

This indicates that the derivatives for your objects were not created as a result of the interrupted ingest.

The repository allows you to manually regenerate any derivatives and any datastreams of any object.

We have created a Regenerating Derivatives and Datastreams Guide to walk you thorough the process. 

A tip from the CTDA team: you can run multiple derivative regeneration processes at the same time. When we have to regenerate derivatives for a bunch of objects in the repository, we open each page of the book or newspaper object in different tabs in the browser and then run the regeneration procedure for each page at the same time.