Tubedb: An On-Demand Processing Database System For Climate Station Data

Geographers, ecologists, and other environmental scientists are typically required to utilise non-continuous measurements from various types of sensors as part of their research activities. However, data management and processing require advanced computer skills and specific knowledge of the measurement sensors.Here, we present the Tube Database (TubeDB), an easy-to-operate software system to archive, quality control, query, and further process time-series data in an efficient manner. Data are imported by loading any unprocessed raw formats as recorded by climate stations into TubeDB. When a user requests data, a query to an on-demand processing unit is created that builds an individual processing flow with the raw data as a source. The processing flow builds on the typical preparation steps for time-series data, such as temporal aggregation, quality checks, and the interpolation of missing values and transformations. The primary user interface is a web-based application to allow easy access, rapid visualisations of time-series within seconds, and powerful product-export functionality. We also provide an R package (rTubeDB) for seamless connection to the R programming environment.TubeDB enables data requesters to assemble individually processed sets of time-series within a web-browser without requiring sensor-specific knowledge or experience in time-series processing. This complements existing time-series databases, which miss climate data-specific processing and visualisation capabilities. As a self-contained application with an embedded database and webserver, TubeDB has both low hardware demands and low installation complexity. TubeDB enables easy access to time-series data for a wide range of non computer scientists and will be useful for many research activities.