The Missing Link: Metadata Conversion Workflows for Everyone

Editor's Notes

1. No ArchivesSpace Hand code EAD (or use template) Used CONTENTdm as digital repository Contributor to two consortiums, need to meet both standards ArchivesWest (for EADs) MWDL (for digital content) Some batch loading of digital content Relied on spreadsheets populated row by row
2. Sara and Andrea will be demonstrating two processes – converting HTML finding aid inventories into Dublin Core metadata and Digital Content Linking. All the step-by-step procedures are available at the links above, if you want to try them out later. We will also show these links at the end of the presentation.
3. In the days before standardization, finding aid formats were as unique as the people creating them. This made legacy finding aids difficult to convert into spreadsheets. In addition, we also found that XML stylesheets vary with each collection. We needed a simple, low tech option to convert our legacy finding aids into Dublin Core compliant data for digitization. After extensive research, we opted for the copy/paste method process because it was by far the easiest method to develop and teach. Everybody can copy the html table-formatted container list and paste it into an Excel spreadsheet. We also wanted to utilize the tools we already had on hand – Excel, Oxygen, CONTENTdm. We did not want to purchase or design new software- since such an approach would have been counterproductive to our goal of maintaining a low technological bar. So by developing a strategy that involves less than 10 steps, we adjusted data using common spreadsheet formulas and an XML Editor to batch import the data into the digital collection management system.
4. Step 1: We copied the table formatted container list from the online finding aid
5. We then open our plain, old, run-of-the mill spreadsheet. Or is it?
6. We pasted the html table-formatted container list into a blank spreadsheet, which we titled “Raw HTML Copy”. We want to separate out the identifying numbers in Column B – the 01:01: and so forth, from the title and place the data into its own column.
7. We accomplish this by inserting a column, enter our formula =RIGHT(C1, LEN(C1)-7, with 7 representing the number of characters you want removed from the cell.
8. We now have the title isolated into its own cell.
9. Step 3: Insert another column to include our identifiers. Type in the first three identifiers: 1:01, 1:02, 1:03, highlight the first three rows and grab the black square at the bottom and drag down to your last item to autofill the cells with consecutive numbers.
10. The identifiers have now been separated from the title into their own column.
11. Step 4: Copy corresponding columns from the Raw HTML sheet into the EAD sheet. But beware: make sure you select Paste Special when copying instead of just Paste to make sure only the data is exported over and NOT the formulas otherwise your data fields will not export correctly and will display hashtags.
12. Step 5: Insert the collection name, Herald Journal Photograph Collection; Collection Number, P0001 & Collection URL into the first three rows.
13. Through the use of embedded Excel formulas, collection information is then effortlessly exported over to the Dublin Core sheet from the EAD sheet into Source, Physical Collection Name,
14. Physical Collection Number, Box, Item, Call Number, & Collection Inventory URL for each item. Review the Dublin Core sheet for complete exportation and clean up the sheet to remove empty columns. For instance, this collection did not have folder information so the sheet exported zeros for folder information. Those will need to be re
15. Step 6: Insert filenames provided by our Digital Initiatives staff Step 7: Save spreadsheet as a tab delimited file Step 8: Open the file in a text editor such as Notepad, delete any trailing spaces, and save the file again for batch uploading into CONTENTdm. And now Andrea will explain the batch linking digital content.
16. Overview This is a brief outline of the procedures involved in the workflow we have created to batch upload and embed links to digital content in EAD finding aids (this process works best when an entire collection has been digitized). The 3 main processes include first, the exporting of digital collection metadata into a tab delimited file then editing that metadata in order to repurpose it for linking. Second, we then use the mail merge function in Microsoft Word to automatically create an xml format container list that can be copied then pasted directly into the xml document for the EAD finding aid. Finally, you perform quality control on the xml document then upload the content to Archives West.
17. Here is a more detailed look at the process The first step is to export metadata from your digital asset management system – in our case the system is CONTENTdm and the process is pretty simple. In CONTENTdm administration you select the collections tab and then go to the export option from the menu – you make the appropriate selections for the metadata export – then CONTENTdm creates a tab-delimited text file > you right click on the file to “Save Link As” and save it to your computer. This text file can now be opened in Microsoft Excel. You click through the text import wizard until the process is finished. The result should be a spreadsheet that looks something like this with a lot of fields for the collection metadata Which you will then edit to only include information needed to create an EAD container list with the necessary elements for the xml document including component numbers, component levels, and any necessary hierarchical containers for box, folder, or item, and title, format, date, and the ARK URLs for linking the digital content.
18. Once you have finished making the necessary edits to your spreadsheet you can move on to the next step which is to utilize the mail merge function in Microsoft Word to create a new xml container list for EAD with links to digital content embedded. To begin you will need to use a template like the one you see here This template should represent the xml coding needed for a single item in your EAD finding aid and you want to be sure to include the digital access object and xlink tagging (which are necessary for the content linking to operate effectively). The parts of the xml template that are highlighted here in the angle brackets are variable while the rest of the text is constant, or fixed. Mail merge will use each row of data in your spreadsheet to populate these variable fields and duplicate this template for each item in your collection. To perform the mail merge you first go to the mailings tab in Word and click “Start Mail Merge” and then make sure “Normal Word Document” is selected. Second, you click “Select Recipients” and choose “Use Existing List”, a new window opens to select a table > you select your spreadsheet then another new window opens for you to select your spreadsheet again as the data source for the merge. Next, you will assign fields from your spreadsheet to the corresponding EAD elements in the xml template. You begin by highlighting the first EAD element, then you go to “Insert Merge Field” then select the matching field from the drop down list of data source options. You repeat the same process for each of the EAD elements in your template. Once you have finished, you complete the merge by selecting “Finish & Merge,” then you select “Edit Individual Document” then you choose “All” You will now have a new word document that should look like this. You should see xml for individual items in your collection on each page with information inserted from your spreadsheet. You will then want to make any necessary edits to the xml (like removing empty tags or getting rid of all the extra white space).
19. Then for the final phase of the process you copy the entire container list in Word and paste it into the <dsc> section of your master xml file for your collection’s EAD finding aid. You can then perform quality control on the xml, once finished you can upload your new EAD finding aid complete with links to the digital objects
20. Throughout the creation of this workflow we have learned a few things and we can make some suggestions of things to keep in mind for anyone seeking to implement this process: First, there will most likely be a training needs - you will need be prepared to teach and re-teach as necessary, also make sure those involved in the process understand the overall purpose and benefits from the results of their work – for example teach about how users are accessing material and also what the description differences are in each system You will also need to be sure that everyone is aware of and using best practices and standards for your institution to ensure consistency from all parties involved in the process This workflow involves the use of Excel quite a bit – so there needs to be a certain level of fluency with the program – for example: formatting cells in the spreadsheet can be tricky especially when working with dates for your collection You will need to also make sure that there is compliance across multiple standards – for example, DACS allows “circa” dates but ISO standards do not - you will need to keep in mind that there is an overall need for the information to be machine readable as well as human readable Finally – be aware of and consider the future applications for the process (for example we anticipate adopting Archives Space at some point and we will no doubt have to adapt our workflows for that)
21. If you would like to try out the process you can access our detailed workflows as well and the slides from this presentation today at these sites.
22. You are also welcome to contact any of us if you have further questions