EOSDIS Common Metadata Repository Surpasses 1 Billion Data Records

The CMR provides users with immediate access to the metadata of more Earth science data that ever before.
Image
words Common Metadata Repository next to a stylized globe

The science metadata holdings in the Common Metadata Repository (CMR) of NASA’s Earth Observing System Data and Information System (EOSDIS) broke the 1 billion records mark on June 18, 2022. This number represents the number of data files in the approximately 54,000 dataset collections from satellite, airborne, field campaign, and other missions in the EOSDIS Earth science data community as well as data from national and international partner agencies.

The CMR is a high-performance, high-quality, continuously evolving metadata system that catalogs all data and service metadata records for the EOSDIS system and is the authoritative management system for all EOSDIS metadata. 

According to Valerie Dixon,  search and discovery lead with NASA’s Earth Science Data and Information Systems (ESDIS) Project, the CMR’s ability to reach this milestone without any impacts to performance proves the system is not only robust, but able to evolve apace with the growing needs and expectations of NASA’s Earth observation community.

“With the growing global importance of Earth science data and increasing number of and interest in Earth Observation missions, it is crucial to have a metadata system that ensures both historical and newly observed data continue to be discoverable,” said Dixon.  “After all, the best data in the world are useless if no one can find or figure out how to use them, and metadata both enables data discovery and provides necessary context.”

Metadata are what make the data in the EOSDIS collection manageable and searchable. At their most basic, metadata are simply data that describe data, and they provide information such as when and where Earth observation data were collected, what instrument and instrument settings were used to collect them, and how the data were processed.

Modern Earth science applications strive to provide end users with nearly immediate access and interactivity across massive stores of Earth science data. These data are discovered, navigated, and often interrogated through science metadata. As the range of applications grow and more information moves from the underlying science data to metadata, the challenges of navigating even just the metadata increases. Along with the 54,000 collection records, the CMR houses roughly 1 billion granule records and receives tens of millions of ingest actions (collection and granule creation, update, and deletion requests) and tens of millions of queries every day; the CMR strives to resolve 99% of its queries in under a second, and consistently achieves over 95%.

As a single, shared, scalable metadata repository, the CMR merges all current capabilities and metadata from legacy NASA Earth science metadata systems. In addition, the CMR:

  • Provides very fast (sub-second) searches of the Earth science data available from EOSDIS
  • Allows for expansion (scalability) to enable new capabilities as users’ metadata needs evolve
  • Ensures high-quality metadata through a process of continual curation and assessment, both automated and manual
  • Provides a metadata model—the Unified Metadata Model (UMM)—that unifies multiple metadata formats under a common umbrella of continual evolution and development of advanced metadata concepts

The CMR benefits data providers by letting them ingest their metadata into a single system. Similarly, the CMR benefits data users by letting them search a single system to locate and retrieve EOSDIS and International Data Network (IDN) data.

Learn more about the CMR and other components of the EOSDIS system.