Design and Implementation of an Efficient Data Stream Processing System

Ali Salehi

CSIRO ICT

DATE: 2010-08-19
TIME: 16:00:00 - 17:00:00
LOCATION: CSIT Seminar Room, N101
CONTACT: JavaScript must be enabled to display this email address.

ABSTRACT:
In standard database scenarios, an end-user assumes that all data (e.g., sensor readings) is stored in a database. Therefore, one can simply submit any arbitrary complex processing in the form of SQL queries or stored procedures to a database server.

Data stream oriented applications are typically dealing with huge volumes of data. Storing data and performing off- line processing on this huge dataset can be costly, time consuming and impractical. This work describes our research results while designing and implementing an efficient data management system for online and off-line processing of data streams in the ield of environmental monitoring. Our target data sources are wireless sensor networks. Although our focus is on a speciic application domain, our results are designed in a generic way, so that they can be applied to wide variety of data stream oriented applications.

We will present GSN middleware which enables fast and iexible deployment and interconnection of sensor networks. It provides simple and uniform access to a comprehensive set of heterogeneous technologies. Additionally, GSN offers zero-programming deployment and data-oriented integration of sensor networks and supports dynamic re-coniguration and adaptation at runtime. We present the virtual sensor concept, which offers a high- level view of arbitrary stream data sources, its powerful declarative speciication and query tools. Furthermore, we describe design, conceptual, architectural and optimization decisions of GSN platform in detail.
BIO:
Ali Salehi obtained Ph.D. in Computer Science from Ecole Polytechnique FAdArale de Lausanne (2010). His research interests are data stream processing, distributed data storage and financial markets. Ali is founder of Global Sensor Network (GSN) project which is an open source stream data management and data integration platform. GSN is used as the core technology in over 10 EU/Swiss funded research projects. Ali is also founder of NexTick project (specialized version of GSN optimized for use in Finance). NexTick is an open source platform for analyzing and visualizing technical indicators over securities from NYSE and NASDAQ. Ali joined CSIRO ICT center on the 1st of August 2010 as a postdoctoral fellow.