Paper/Poster/Presentation Title

Technology behind the Deltares Open Archive

Keywords

Open, archive, scientific data, THREDDS, GeoNetwork, CSW

Location

Session A2: Sharing Scientific Environmental Data and Models

Start Date

18-6-2014 10:40 AM

End Date

18-6-2014 12:00 PM

Abstract

Water agencies conducting daily operational forecasting need an archive solution that enables them to review (and defend) forecasts and decisions made, conduct post-event performance analysis, create canned datasets for training and create datasets for model calibration and hindcasting. This archive solution should store datasets that range from time series (observations and forecast), workflow definition and model-run settings, to web-reports and text products (e.g. forecaster notes). The challenge is to provide an open solution that supports both operations and science, allowing access by a variety of applications. The Deltares Open Archive is a new solution which meets those needs. The solution consists of a data storage, a catalogue and services to store, backup, discover and retrieve data. The data storage is file based using standardized data formats such as NetCDF-CF for scientific data and XML, HTML, PDF, txt, and zip to store non-scientific data. Associated metadata is held in xml. Data access is provided by the THREDDS Data Server (TDS), a web server that provides a variety of standardized remote data access protocols like OPeNDAP, WMS and WCS for scientific datasets and plain HTTP for non-scientific data. To search the Deltares Open Archive, a key-word based catalogue solution is developed using GeoNetwork, an OGC Catalogue Service compliant product. Metadata is composed of area, data source and temporal information, while more tags can be added to highlight e.g. threshold crossings and label a period as being a relevant (storm) event. Data can be provided by Delft-FEWS or any other application, as long as metadata requirements are met.

COinS
 
Jun 18th, 10:40 AM Jun 18th, 12:00 PM

Technology behind the Deltares Open Archive

Session A2: Sharing Scientific Environmental Data and Models

Water agencies conducting daily operational forecasting need an archive solution that enables them to review (and defend) forecasts and decisions made, conduct post-event performance analysis, create canned datasets for training and create datasets for model calibration and hindcasting. This archive solution should store datasets that range from time series (observations and forecast), workflow definition and model-run settings, to web-reports and text products (e.g. forecaster notes). The challenge is to provide an open solution that supports both operations and science, allowing access by a variety of applications. The Deltares Open Archive is a new solution which meets those needs. The solution consists of a data storage, a catalogue and services to store, backup, discover and retrieve data. The data storage is file based using standardized data formats such as NetCDF-CF for scientific data and XML, HTML, PDF, txt, and zip to store non-scientific data. Associated metadata is held in xml. Data access is provided by the THREDDS Data Server (TDS), a web server that provides a variety of standardized remote data access protocols like OPeNDAP, WMS and WCS for scientific datasets and plain HTTP for non-scientific data. To search the Deltares Open Archive, a key-word based catalogue solution is developed using GeoNetwork, an OGC Catalogue Service compliant product. Metadata is composed of area, data source and temporal information, while more tags can be added to highlight e.g. threshold crossings and label a period as being a relevant (storm) event. Data can be provided by Delft-FEWS or any other application, as long as metadata requirements are met.