Parsimonious sensing with Active Learning: applications with context mining and environmental sensing

Abstract

The unprecedented success of Web 2.0, and with it, social media services, has resulted in massive amounts of user-generated data. Traditional techniques are no longer adequate to deal with this sheer amount of information. In an attempt to address this problem, new techniques that can be applied to big data, are being proposed in an increasingly frequent way. In this dissertation, the concept of parsimonious sensing and some of its applications are presented. Parsimonious sensing attempts to select the most relevant information from a large dataset, thus reducing the cost of its analysis. To do this, it employs different techniques such as active learning, also know as optimal experimental design in the field of statistics. We also explore some innovative methods of identifying relevant anomalies from a large dataset to be subsequently explored. This dissertation studies the application of parsimonious sensing on three unique datasets. The first main experience studies the employment of active learning in an environmental sensing network system with air quality parameters. The second experience depicts an attempt to predict the number of hits for a certain query related to events happening in Singapore, thus decreasing the number of required queries. The third and last experiment makes use of a dataset provided by a major taxi company in Singapore and tries to identify traffic anomalies and later, synthesize queries that are run through a search engine in order to identify the context of the anomalies. We found the application of parsimonious sensing to be successful when implemented in the context of environmental sensing. We have further developed a system capable of identifying traffic anomalies and returning a number of links that can potentially explain why they happened. The fully automated system has been shown to be better than a hybrid system, composed of information retrieved both automatically and manually. The findings from this dissertation can hopefully shed some light on the possible applications of parsimonious sensing to diverse contexts.