A Spatio Temporal Data Imputation Model for Internet of Things using Similarity Search and Deep Learning

Abstract

The prevalence of missing data in the Internet of Things (IoT) poses significant newlinechallenges to reliable data analysis and decision-making processes. IoT data is often newlineplagued by connectivity errors, environmental extremes, sensor inaccuracies, and newlinehuman errors, leading to incomplete datasets. The urgency of developing robust newlineimputation methods is underscored by the critical role of IoT data in various newlineapplications, including environmental monitoring, industrial automation, and smart newlineinfrastructure management. Inaccurate or incomplete data can lead to erroneous newlineconclusions and suboptimal decisions, emphasizing the necessity of accurate newlineimputation techniques tailored to the unique characteristics of IoT datasets. Despite newlinethe interconnected nature of IoT data in both spatial and temporal dimensions, existing newlineimputation techniques often overlook these spatial correlations or rely solely on newlineEuclidean frameworks, resulting in suboptimal outcomes. These gaps highlight the newlineneed for advanced methodology that can handle large missing gaps and efficiently newlineutilize available data and partially imputed values. newlineDealing with these issues is essential to improving the reliability and efficiency of newlineIoT systems to ensure the integrity of data-driven decision-making processes. This newlineresearch makes significant strides in addressing the challenges inherent in IoT data newlineanalysis, particularly focusing on the critical issue of missing data. By introducing newlineadvanced methodologies, this work aims to enhance imputation accuracy in IoT newlineapplications. This thesis include techniques that preserves spatial correlations and newlinetemporal correlations to effectively impute data from failed sensor nodes, a fast newlinesimilarity search-based approach to handle diverse missing patterns by utilising newlinepartially imputed data, and a network architecture grounded in the Variational newlineAutoEncoder (VAE) framework that captures both global spatial and temporal newlinedependencies for accurate multivariate data imputation. newlineThese methods not only provide solutions for handling mis