When you turn on a faucet, the expectation is that water will flow and that this flowing water will be in a form useful for activities such as cooking, cleaning, and drinking. The same is true for data. When you search for NASA Earth observing data and imagery using Earthdata Search or NASA Worldview, the expectation is that the data will be readily available and in the forms required for activities such as research, near real-time natural hazard monitoring, and model development.
The sudden shift in mid-March 2020 to telecommuting for almost everyone responsible for processing, archiving, and distributing data in NASA’s Earth Observing System Data and Information System (EOSDIS) collection has not stopped the flow of data. In fact, metrics from NASA’s Earth Science Data and Information System (ESDIS) Project, which manages Earth observing data in the EOSDIS collection as well as the systems that process, archive, and deliver these data, indicate overall consistency in the total amount of monthly data distributed by the discipline-specific EOSDIS Distributed Active Archive Centers (DAACs).
A look at the strategies for ensuring the steady flow of NASA Earth observing data shows how NASA’s EOSDIS is able to meet the challenge of providing data to a data-hungry world (even when much of the world is working from home).
Let’s start with the instruments aboard satellites orbiting hundreds of kilometers above Earth. Their function is to collect the raw data and observations about Earth processes that are the basic ingredients for the tens of thousands of EOSDIS data products. Regardless of what is going on at the planet’s surface, these instruments continue their job of collecting data 24 hours a day, seven days a week. But an instrument has only limited storage for these data; the data must return to Earth.
Getting the raw data from orbiting sensors to a ground station is accomplished by NASA’s Earth Science Mission Operations (ESMO) Project. These vital data transfers have not been impacted by remote work, and ESMO satellite platform operators are continually downlinking data from satellites to ground stations. As mission-essential personnel, ESMO staff are hard at work at the operations center at NASA’s Goddard Space Flight Center in Greenbelt, Maryland. After being downloaded from satellites, data are transferred to teams located at NASA facilities, universities, and other centers of excellence for processing.
Transforming raw sensor data into the multitude of data products required by EOSDIS data users is accomplished by instrument processing teams who continue to maintain a steady flow of data from their processing streams to the appropriate DAACs for distribution. This work is enabled through the use of encrypted virtual private networks (VPNs) that not only provide data security, but also safely enable interaction between processing teams and the DAACs receiving their processed data.
Along with ESDIS, ESMO, and data processing teams, DAACs, too, had to transition to remote operations in March. For DAACs located in large cities with robust internet connectivity, operations quickly shifted off-site, and DAAC staff are securely running programs and monitoring systems using VPNs, cloud services, and other means. For some DAACs, though, transitioning to remote activities was not as simple.
For NASA’s Alaska Satellite Facility DAAC (ASF DAAC) located approximately 200 miles south of the Arctic Circle at the University of Alaska Fairbanks (UAF), providing the necessary internet connectivity in private homes was a challenge. “This was the biggest hurdle we had [in the shift to telecommuting],” observes Dr. Nettie La Belle-Hamer, ASF Director and ASF DAAC Manager, via email. “For a place that has very little cell coverage and fiber networks, having a lack of access to the internet is an issue.”
ASF DAAC archives and distributes Synthetic Aperture Radar (SAR) data in the EOSDIS collection, some of which are distributed under international agreements with foreign space agencies. During the first two weeks after UAF closed to the public on March 26, the home work environments of ASF staff were augmented with necessary equipment via a check-out system to help compensate for internet bandwidth or connectivity issues. Currently, DAAC operations are continuing without any outages or slowdowns and data distribution remains consistent.
ASF operations staff, including information technology (IT) staff, still have access to systems located at UAF for the DAAC and the ASF Ground Station, which is part of the NASA Near Earth Network (NEN). As Dr. La Belle-Hamer notes, for the Ground Station this means daily in-person operations at UAF.
Four-thousand miles east of Fairbanks, NASA’s Socioeconomic Data and Applications Center (SEDAC) near New York City has maintained almost all of its operations since mandatory telecommuting began in mid-March. SEDAC is the EOSDIS DAAC responsible for archiving and distributing socioeconomic data in the EOSDIS collection, and is hosted at Columbia University’s Center for International Earth Science Information Network (CIESIN) located in Palisades, NY, on the Hudson River. All online tools and services are operational, and new socioeconomic data are being added to the EOSDIS archive.
Through CIESIN, SEDAC has a robust IT infrastructure designed to support secure remote access for all staff and provide a flexible virtual environment for development, sustaining engineering, and operations. When staff members had to quickly adjust to remote work, IT staff immediately strengthened the DAAC’s VPN capacity and deployed laptops and other devices to improve staff productivity.
One early proactive step, notes Dr. Robert Chen, CIESIN Director and SEDAC Manager, was to install remote environmental monitoring devices in the SEDAC computer and data archive rooms that send out warnings when temperature or humidity exceed specified limits and enable campus facilities staff to respond to on-site problems rapidly. This system worked exactly as planed on August 8 when the temperature of the SEDAC media archive room increased above 80°F, triggering the remote alarm. Onsite personnel were notified, the air conditioning unit was restarted, and the temperature was brought back to safe operating levels.
As Dr. Chen points out, this is not the first incidence when SEDAC operations were suddenly impacted. The attacks of September 11, 2001, and Superstorm Sandy (October to November 2012) both caused significant long-term disruptions to New York City and the surrounding region, and provided lessons-learned that SEDAC is applying to the current situation. “I think these events caused us to think more carefully about backup, vulnerability, and recovery issues, and the potential for extended periods of disruption,” he writes in an email. “We have gradually virtualized most of our systems, which has given us greater flexibility in implementing remote operations and reducing the need for onsite work with physical servers, storage devices, and networks. Virtualizing also enables more seamless and secure access to all organizational resources from any remote location, more reliable back-ups, and faster response to problems.”
While VPNs provide secure connections for working with EOSDIS data and moving processed data, there still is a need for real-time interaction between ESDIS Project staff, science teams, and the DAACs. A knock on a door or an inter-office phone call has been replaced with instant messaging tools that facilitate the real-time exchange of ideas or updates. For more involved issues or for team gatherings, video meeting applications such as Teams and WebEx have proved to be valuable collaboration tools.
These tools, however, are not without their limitations. For example, video conferencing applications use a lot of bandwidth, and can slow down other systems. In addition, they can’t compensate for existing issues such as distance – a meeting between teams at the ESDIS Project office in Greenbelt, MD, and ASF DAAC in Fairbanks will always mean coordinating schedules across four time zones.
DAAC help desks are still providing rapid response to user questions. However, the inability of ESDIS and DAAC staff to physically attend numerous science meetings and interact one-on-one with data users at events like the Ecological Society of America Annual Meeting or the American Geophysical Union Fall Meeting (both of which shifted to virtual meetings for 2020) removes a valuable link in the EOSDIS data use chain.
Remote work requirements also are leading to work-arounds for internal office activities. For example, to ensure that software development is not impacted and that requirements continue to be met, staff have mapped requirements to objectives during development sprint periods on quarterly increments. Technical meetings to discuss and evaluate these software and development efforts, such as quarterly Program Increment planning meetings (which are part of the Agile Framework), shifted to virtual meetings. These virtual meetings involve more than 200 programmers across the nation and utilize several virtual collaboration tools to ensure that development, integration, and operations remain on schedule.
This is not to say that these virtual meetings seamlessly take the place of in-person meetings. While in-person planning meetings at Goddard took place over slightly more than two full days, virtual meetings require five partial days. Facilitating these large virtual meetings, which include plenary sessions along with numerous breakout sessions, necessitates a great deal of coordination. Out of respect for participating West Coast programmers, meetings don’t start until 11 am Eastern Time, and the meeting schedule incorporates many breaks to allow staff time to meet personal needs ranging from managing child care to eating lunch. Even with the extra time needed for these strategies, EOSDIS software teams continue to meet data user needs and development evolution goals.
Overall, ESDIS and DAAC teams have adapted well to the new reality of distributed collaboration, and it’s likely that some of these interactions will become more a part of normal operations. In addition, the ongoing trend towards cloud-based services should enable more flexible, distributed operational capabilities.
Both Dr. Chen at SEDAC and Dr. La Belle-Hamer at ASF DAAC observe that the responses by their teams to provide a steady, uninterrupted flow of NASA Earth observing data while conducting DAAC operations via telecommuting have strengthened relationships with their respective user communities. “I think users have been appreciative of the continued service provided by all of the EOSDIS DAACs despite the recent challenges,” writes Dr. Chen. “In SEDAC’s case, there has been a steady increase in the number of new Earthdata users accessing SEDAC, to more than 2,000 per month, on average.”
Dr. La Belle-Hamer points out that with everyone working remotely, she and her team understand exactly what their data users are experiencing. “We have had to deal with low bandwidth, no cell overage, old laptops, and long hours with video conferencing,” she writes. “Just like our data users – we are in this with them.”
Even though those responsible for NASA Earth observing data are working from home offices scattered across thousands of miles, data users can be assured that the availability of NASA Earth observing data is unaffected. In addition, new data, services, and tools are still being developed. These may be challenging times, but EOSDIS is meeting the challenge.