Keywords
Cloud computing; distributed computing; parallel computing; data provisioning; devOps; full technology stack; environmental modeling
Start Date
5-7-2022 12:00 PM
End Date
8-7-2022 9:59 AM
Abstract
Scientific computing applications can be complex to design, difficult to deploy, and expensive computationally. Using the latest advances in software engineering and cloud technologies, we can greatly mitigate these challenges while also providing new opportunities to pursue research with increased flexibility. Our Hydrologic Micro Services (HMS) technology stack has been developed as a containerized set of components, orchestrated through Kubernetes, that can be managed, updated, and deployed with relative ease to any cloud platform. The general framework can be applied to a variety of different scientific software applications, that can be containerized, providing the ability to quickly create complete technology stacks for solving new or unique problems, or integrate new features into existing applications. Combining flexible cloud compute resources, automatic horizontal scaling (distributed computing) and vertical scaling (parallel processing) using cloud platforms, Kubernetes and Dask, our scientific software applications can be used to tackle environmental problems in new and efficient ways. In addition, the deployment of applications using Kubernetes can drastically reduce the amount of time required for application and server management, freeing up more time for research and solving emerging environmental problems. Our HMS technology stack contains the following components: Nginx, http traffic routing; Django, serves up web content; Flask, REST API; Dask, asynchronous, distributed, and parallel processing; .NET 6, data provision and simulation application; MongoDB, data storage.
Scalable Orchestration of Containerized Science Applications
Scientific computing applications can be complex to design, difficult to deploy, and expensive computationally. Using the latest advances in software engineering and cloud technologies, we can greatly mitigate these challenges while also providing new opportunities to pursue research with increased flexibility. Our Hydrologic Micro Services (HMS) technology stack has been developed as a containerized set of components, orchestrated through Kubernetes, that can be managed, updated, and deployed with relative ease to any cloud platform. The general framework can be applied to a variety of different scientific software applications, that can be containerized, providing the ability to quickly create complete technology stacks for solving new or unique problems, or integrate new features into existing applications. Combining flexible cloud compute resources, automatic horizontal scaling (distributed computing) and vertical scaling (parallel processing) using cloud platforms, Kubernetes and Dask, our scientific software applications can be used to tackle environmental problems in new and efficient ways. In addition, the deployment of applications using Kubernetes can drastically reduce the amount of time required for application and server management, freeing up more time for research and solving emerging environmental problems. Our HMS technology stack contains the following components: Nginx, http traffic routing; Django, serves up web content; Flask, REST API; Dask, asynchronous, distributed, and parallel processing; .NET 6, data provision and simulation application; MongoDB, data storage.
Stream and Session
false