Significant evolutionary activities are initiated as a result of programs reviews. In 2015, NASA proactively tasked the Earth Science Division (ESD) / Earth Science Data and Information Systems (ESDIS) Project to sponsor an independent, holistic review (with appropriate stakeholders, including ESD Reserach and Analysis, ESD Applications, ESDIS, Science community leaders, and Mission stakeholders, as well as a senior representative from the NASA Chief Information Officer community) to study potential efficiencies and enhanced capabilities from a variety of perspectives based on:
- Science Discipline
- Optimizing common data operation tasks across Distributed Active Archive Centers (DAACs)
Review Finding #1
Commercial cloud environments offer potential for storage, processing, and operations efficiencies; improved cross-DAAC collaboration; and potential for new data access and service paradigms.
While the ESDIS Project and DAACs have initiated preliminary studies of relevant Cloud technology, these studies have been ad hoc.
Substantial additional work, coordination, and investment is needed in this area to identify opportunities and risks.
Review Recommendation #1
The ESDIS Project should develop, implement, and report on the outcome of prototypes to explore the advantages, risks, and costs of using commercial cloud environments for the following:
- Storage and data transfer (e.g., assess benefits and impacts of “virtual data center”, “virtual collections,” and costs for distributing data from cloud environments)
- Processing (e.g., advantages and effects of elastic computing on cost and speed for processing and reprocessing)
- Improved data access (including exploration, selection, reduction, data analytics), and the feasibility associated with enabling data analytics/on-demand processing (e.g., bringing user processing/algorithms closer to the data)
An important goal would be enabling user work across multiple large data sets managed by different DAACs without the need to transmit data over networks.
Review Finding #2
Tools across the DAACs have varying levels of usability. While many are discipline/data collection unique, there is also considerable overlap, redundancy, and duplication of effort. Redundancy of tools leads to user confusion.
Review Recommendation #2
The ESDIS Project should develop, execute, and report on efforts to:
- Develop a common software requirement and development strategy across DAACs
- Work toward a uniform user experience across DAACs based on design/development best practices
- Implement common data services and APIs for uniform search, discovery, and access to data (common APIs should augment necessary discipline specific tools)
- Consolidate and simplify duplicative software tools
Review Finding #3
Open Source software development has the potential for increased collaboration. However, the time and effort required to share software outside the original NASA development organizations, by open sourcing, can be prohibitive. Inability to share software increases cost and inhibits reproducibility of data products.
Review Recommendation #3
NASA should develop more streamlined approaches and processes to allow open source software development throughout the agency.
Adequate resources need to be applied to ensure that NASA-developed software can be open sourced on a timeline that facilitates collaboration and reuse.
Resulting Program Review Activities
- Three commercial cloud prototyping efforts are underway to evaluate foundational capabilities for DAAC and Earth Observing System Data and Information System (EOSDIS) services.
- Partnerships with commercial cloud providers are advancing to evolve NASA data and services.
- Working towards a standard collaborative process for cross-DAAC software development to improve efficiency and usability.
- Open Source Policy instituted for all new Earth Science Data System software development efforts. However, procedural obstacles remain.