After Three Decades, a Key Earth Data System Enters Its Final Chapter

Every day, people rely on Earth science data to better understand our changing planet. Scientists use it to study melting glaciers, changing forests, and shifting weather patterns. Emergency responders monitor floods, droughts, and wildfires. Water managers make decisions about reservoirs and water supplies. Teachers bring real-world satellite observations into classrooms.  

Behind their countless research papers, forecasts, decisions, and discoveries is an enormous infrastructure built to collect, manage, and deliver that data.

At ASDC, the team focused on minimizing the impact on users as systems moved to Earthdata Cloud. “My primary goal during the transition was ensuring minimal disruption to the user experience,” said Hazem Mahmoud, ASDC DAAC project scientist. Clear communication with data producers and users, he said, was essential as teams navigated unfamiliar cloud-based infrastructure.

At LP DAAC, teams migrated more than 20 petabytes of data into the cloud while continuing to support an ever-expanding portfolio of missions and users. Teams reconciled cloud archives with existing systems, preserved metadata integrity, and maintained operational continuity throughout the transition.

At NSIDC DAAC, where ECS was woven into nearly every operation, the transition required teams to rethink how systems were built and managed, while navigating the many dependencies of a mature operational system. "The scale of rebuilding it with something new was huge," said Amanda Leon, manager of NSIDC DAAC.

A Shared Effort Across NASA

The success of the migration depended not only on technology. Delivering Earth science data at NASA's scale is a team sport, and the ECS-to-cloud effort made it clear just how interconnected NASA’s archive centers have become. 

NASA’s Earth Science Data and Information System (ESDIS), NSIDC DAAC, LP DAAC, ASDC, and contractor teams—engineers, data managers, system administrators, user support specialists, developers, scientists, project managers, and communications professionals—worked closely together, sharing solutions, lessons learned, and technical expertise.

Although each archive center manages different types of Earth science data, the ECS DAACs spent decades solving many of the same challenges together, noted Chris Torbert, former LP DAAC Project Manager. The archive centers have "a special bond because of our common goals and system," he said, and that foundation of trust and cooperation proved invaluable. Teams frequently shared lessons learned, tools, and solutions such that what began as a shared reliance on ECS evolved into a shared effort to move beyond it.

For example, when ICESat-2, MODIS, and VIIRS data ingest streams transitioned from GridFTP and SFTP to the Science Data Transfer Protocol (SDTP), operators and developers from the three DAACs worked together to test and operationalize the new protocol.

What Users Gain From the Cloud

One of the immediate benefits of moving to the Earthdata Cloud is improved reliability. Users accustomed to scheduled maintenance windows under ECS are finding that data and services are available more consistently than before.

"In the cloud, we rarely have downtime anymore," said Christopher Bond, NSIDC systems engineer. "Data is available to the user all week." 

Cole Krehbiel, project scientist for LP DAAC, similarly pointed to the disappearance of the Wednesday morning outages that had long been associated with ECS operations.

The move to the Earthdata Cloud also has changed how people work with data. Imagine an emergency manager monitoring wildfire conditions across multiple states. Instead of downloading and sorting through large volumes of satellite data from multiple sources, the responder can increasingly access information through shared cloud-based tools and services. The water manager assessing drought conditions, the educator building classroom lessons, the scientist studying decades of environmental change: all benefit from faster, more flexible access to information.

With Earthdata Cloud, users now spend less time moving data and more time working with it. Developers can build their own tools and applications on top of cloud-hosted data rather than waiting for mission-specific tools to be developed. Users can access data through more consistent interfaces across NASA's Earth science archives.

Researchers are already seeing the benefits. 

Zach Fair, a researcher who uses NASA's CryoCloud platform, said cloud-based tools have accelerated both data discovery and analysis. In a recent study of snow depth observations collected during NASA's SnowEx campaigns, Fair and colleagues aggregated multiple datasets— including ICESat-2 observations, airborne lidar measurements, and weather station records—directly through cloud-based services, allowing the team to spend more time on analysis. He also noted that shared cloud environments have strengthened collaboration and reproducibility, particularly during community hackweeks, where participants work from common datasets, tutorials, and workflows.

"Cloud-based systems such as CryoCloud and earthaccess are what allowed me to jumpstart my research and gain a better understanding of package management and efficient coding," Fair said. 

That kind of cross-dataset work reflects what Leon described as increasingly "permeable" boundaries between NASA’s data collections and disciplines, opening new opportunities for discovery, analysis, and collaboration.