Fast Approximate Evaluation of OLAP Queries for Integrated Statistical Data

We have developed a mediator architecture that integrates statistical information about energy prod- ucts from several government agencies, such as the Bureau of Labor Statistics, the Energy Information Administration, and the California Energy Commission. Our architecture has a dual mode of operation. First, our system can retrieve live data from databases and web sources from these agencies. This allows the users to obtain completely up-to-date data. However, for complex analytical queries that typically require large amounts of data and processing, live access does not offer the level of interactivity that some users require. Second, our system can warehouse the information from the data sources to allow for complex analytical queries to be executed much more efficiently. However, the data would be only as recent as the last update to the data warehouse. In this paper we describe the architecture and focus on how to perform analytical queries against the data warehouse very efficiently. We present results using a fast wavelet-based technique for progressive evaluation of range-sum queries. This technique allows for returning an approximate result to the query very efficiently and for fast convergence to the exact result. We envision users exploring many complex queries using the very fast approximate results as guidance and only obtaining the exact results for those queries that are deemed of interest. We present experimental results showing the efficiency of both approximate and exact queries.