Keywords

Data, Python, Tethys Platform, HydroShare

Start Date

26-6-2018 9:00 AM

End Date

26-6-2018 10:20 AM

Abstract

Most environmental modelling efforts have significant data needs, which can present technical challenges and distract from the primary modelling objectives. These challenges include finding relevant data sources, retrieving large amounts of data, performing data transformations, and finally storing and sharing results. This presentation describes the tools being used and developed at the U.S. Army Engineer Research and Development Center (ERDC) to address these challenges. These tools include (1) Quest, an extensible Python library for searching various local and public data providers, automating downloading, performing various data filters (or transformations), and finally, publishing the data to data repositories, (2) Data Depot, an internal instance of HydroShare, a web-based data repository and catalogue, which supports data archival, discovery, and sharing, and (3) Quest Web, a map-based web interface to Quest built with Tethys Platform. These tools all work together to provide a comprehensive data solution for our scientists and engineers, and though these tools have been developed specifically to support hydrologic modelling workflows, they are generic and extensible making them applicable to a broad range of data and workflows.

Stream and Session

B2: Hybrid modelling and innovative data analysis for integrated environmental decision support

COinS
 
Jun 26th, 9:00 AM Jun 26th, 10:20 AM

Automated Data Discovery, Retrieval, Manipulation, and Publication using Python, Tethys, and HydroShare

Most environmental modelling efforts have significant data needs, which can present technical challenges and distract from the primary modelling objectives. These challenges include finding relevant data sources, retrieving large amounts of data, performing data transformations, and finally storing and sharing results. This presentation describes the tools being used and developed at the U.S. Army Engineer Research and Development Center (ERDC) to address these challenges. These tools include (1) Quest, an extensible Python library for searching various local and public data providers, automating downloading, performing various data filters (or transformations), and finally, publishing the data to data repositories, (2) Data Depot, an internal instance of HydroShare, a web-based data repository and catalogue, which supports data archival, discovery, and sharing, and (3) Quest Web, a map-based web interface to Quest built with Tethys Platform. These tools all work together to provide a comprehensive data solution for our scientists and engineers, and though these tools have been developed specifically to support hydrologic modelling workflows, they are generic and extensible making them applicable to a broad range of data and workflows.