6 Replies Latest reply on Jul 31, 2017 10:03 AM by Michael Tomko

    Adding Electronic Records to the Catalog

    techhistorynerd

      I'm wondering what the status/plans are for National Archives records that are already in digital form with regards to the Catalog.  For example, the following Series:

       

      https://catalog.archives.gov/id/6307936

       

      has several hundred documents identified as electronic records.  Currently, none of them seem to be available from the catalog, despite their already being "digital."

       

      I realize the amount of digital material the NARA holds is vast and putting it all unrestricted records immediately online is impractical to say the very least, but I was wondering what options might be available to Catalog users to request or even assist as citizen archivists with getting records of interest that are already digital online?  Or is this not really practical for financial reasons?  (I was quite surprised to see the fees for "Born digital" copying at $14-$17 per file - is this intended as a "rate limiter" to prevent huge data requests from overwhelming the system or is it necessary for supporting digital archival systems?  Or (never having done it) is the process of born-digital reproduction more complicated than just a USB file coping operation?)

        • Re: Adding Electronic Records to the Catalog
          Electronic Records Reference Branch

          The National Archives currently makes some “born-digital” or electronic records available in the National Archives Catalog.  However, you are correct that the volume of records makes it a daunting task to try to make all the electronic records available online at this time.

           

          My unit, the Electronic Records Division, has custody of over 800 million “born-digital” files (over 397 terabytes) created by Federal agencies.  We have over 100,000 files (1 terabyte) available in the National Archives Catalog.  A list of our holdings available in the Catalog (as of April 1, 2017) is available at: https://www.archives.gov/files/research/electronic-records/efiles-in-catalog.pdf.

           

          In general, we add records to the Catalog based on user interest and feasibility.  It is an involved process to make the records available in the Catalog and NARA has limited resources.  Thus we are not in a position to make electronic records files available in the Catalog on demand.  But we are continually improving the process with the hopes of being able to do so in the future.  If there are series of records you are interested in, such as the Altair Program and Project Records, please contact us at cer@nara.gov to discuss how you may access the records.

           

          The National Archives Trust Fund establishes the reproduction fees to recover the cost of copying the files, which is more complicated than just a desktop copying operation.  Because we are preparing copies of archival records, all orders are filled as custom copies.

           

          Other National Archives units also make their “born-digital” holdings available in the Catalog.  For example, the Still Pictures unit provides access to digital photographs through the Catalog.

           

          Lastly, additional information about accessing the Electronic Records Division’s holdings from the National Archives is at: https://www.archives.gov/research/electronic-records/access-in-catalog-faqs.html.

           

          Thank you for your interest in NARA’s electronic records.  If you have additional questions, please contact us at cer@nara.gov.

          1 of 1 people found this helpful
            • Re: Adding Electronic Records to the Catalog
              techhistorynerd
              In general, we add records to the Catalog based on user interest and feasibility.  It is an involved process to make the records available in the Catalog and NARA has limited resources.  Thus we are not in a position to make electronic records files available in the Catalog on demand.  But we are continually improving the process with the hopes of being able to do so in the future.

               

              Is the difficulty with making records available in the Catalog technical (difficulty obtaining space on servers, complicated interfaces, etc...), or regulatory/process driven (lots of manual entering of meta-data, long approval process, etc...)?  Is there anything the citizen archivist community can do to help alleviate the problems?  (Designing improved web-based software for more efficient records handling, for example?)

               

              The National Archives Trust Fund establishes the reproduction fees to recover the cost of copying the files, which is more complicated than just a desktop copying operation.  Because we are preparing copies of archival records, all orders are filled as custom copies.

               

              If a researcher is able to be present in person at one of the NARA's physical sites, does that offer any possibilities for simplifying the copying process?  I'm undoubtedly not properly understanding the challenges, but I'd be curious what the barriers are (at least, for in-person file duplication.)

               

              Thanks for your time and the quick response!

                • Re: Adding Electronic Records to the Catalog
                  Electronic Records Reference Branch

                  There are many factors, some of which you suggest, that make providing access to “born-digital” records in the Catalog an involved process.  For our unit, the process involves compiling metadata, via manual data entry, from multiple internal systems in order to submit it to the Catalog.  There may be paper documentation needed to understand or use the electronic records, which we need to digitize in order to also make it available in the Catalog.  We need to review the records to confirm they are fully open, which is a step we take when providing access to records for the first time.  We need to copy the files in order to provide them to the Catalog.  For data files, the copy process also involves determining the output format for access.   

                   

                  There are regulations and processes we need to follow in developing systems or tools for handling and providing access to electronic records.  You may be interested to learn more about NARA’s Electronic Records Archives (ERA) system at: https://www.archives.gov/era/about

                   

                  Unfortunately, there isn’t anything the citizen archivist community can do to help add electronic records to the Catalog at this time.  However, there may be opportunities to help if individuals are interested in becoming volunteers to work on-site.  Information about volunteering at the National Archives in College Park (where our unit is located) is at: https://www.archives.gov/careers/volunteering/dc-metro.

                   

                  Regarding on-site access, unfortunately, we currently do not readily have the means to allow researchers to view or copy electronic records in our custody on-site.  Until very recently, we preserved electronic records on magnetic tape and thus did not have the hardware and software in the research rooms to support on-site access to the electronic records.  Even now, the copy process to retrieve our holdings from our electronic storage systems essentially precludes on-site access.  Traditionally we have had very little to almost no demand for viewing electronic records on-site as researchers rather use the records with their own computers and software.  So we have not really explored providing this type of service.   

                   

                  Some of the other NARA units with electronic records, such as Presidential Libraries and Motion Pictures, may have means for on-site access or copying.

                   

                  P.S. We plan to look at the feasibility of making NASA’s Altair Program and Project Records available in the Catalog.

                   

                  Lynn Goodsell

                  Reference Branch Chief, Electronic Records Division

                  email: cer@nara.gov

                    • Re: Adding Electronic Records to the Catalog
                      Michael Tomko

                      Dear Reference Branch Chief,  ERD

                       

                      I have been researching Desert Storm and have found that there are no textual records or moving images n the Online Public Catalog in electronic format. The records are currently restricted again even though there was a 1995 Declassification Project related to the Khamisiyah Sarin nerve agent plume.  There was a MDR release of the 101st Airborne Division in 2016 which is not in the catalog. I also submitted a Mandatory Declassification Review toward the RG 518 , A1 23, Joint Operations Directorate , Command Files, consisting of the February 91, daily executive summaries, Warbook, Emergency Action Procedure Manual and Significant Event Log. I have also FOIA'd the Command Report, Daily Executive Summaries file. I also FOIA'd the scientific conclusions related to the NARA record testing for contamination  in1995 using mass spectrometry and gas chromatography by the Wisconsin Health Dept, National Media Lab and 3M. The documents  have not been released. I have also requested that NARA place the records for public review in the Online Public Catalog In electronic format per the legislative rules of the FOIA Improvement Act of June 30, 2016, whereby the "Rule of 3" and public access is addressed. I also sent a letter to the Deputy Archivist which discussed the new rules of the FOIA Improvement Act.

                       

                      The Desert Storm records have been restricted beyond the 25 year time frame and there are no textual records in an electronic format at the NARA Online Catalog, even though a Desert Storm Memorial is proposed at the D.C. National Mall? The Moving Images and Motion Pictures of the aerial Mocking Bird / Opus Missions or the BLU 82 Air Fuel Bombing Missions and ground missions has not been released. It is unusual that footage of Vietnam and the 2nd Gulf War has been released, while Desert Storm film remains restricted?

                       

                      Can strategic portions of the Desert Storm Textual Records and Motion Pictures be placed in the Online Public Catalog once they are declassified by the agencies involved?  The MDR of the 101st, listed 7" s of record released in 2016 ?

                       

                      I also emailed the Archivist and Deputy Archivist, and she responded that the FOIA Team would be notified.

                       

                      Would the Executive  Summaries  used by the commanding general and the actual bombing mission footage be of public interest?