Distributed Data Dimensionality Reduction And Denoising

Abstract

With the recent upsurge in the amount of data transmission and use, giving rise to the era of big data, research has put a lot of emphasis on designing efficient methods to extract useful information by reducing the size of a large set of data to a much smaller set. This is the main idea in data dimensionality reduction which is a critical step in applications that involve a constantly increasing data size, and limited computational and communication resources. Such a situation appears in sensor networks where there is a large number of sensors acquiring data. However, the informative part of the sensor data in practice is created by a small number of sources/phenomena of interest and the dimension of data can be effectively reduced.Data dimensionality reduction by employing principal component analysis has proven to be extremely useful. Traditional principal component analysis involves principal subspace estimation obtained by a decomposition of the data covariance matrix in a centralized fashion. However, such centralized approaches do not account for the fact that sensors are spatially scattered while they may have to communication through noisy links. This thesis puts forth a distributed algorithmic framework for estimating the principal eigenspace of the sensor data without the need of a central fusion center. Each sensor is only responsible for estimating only a small part of the principal subspace, and it only communicates with sensors in its near vicinity, the so called single-hop neighbors. Toward this end, the standard principal component analysis framework is reformulated as a separable constrained minimization problem which is solved by utilizing coordinate descent techniques combined with the alternating direction method of multipliers. Computationally simple local updating recursions are obtained that involve only single-hop inter-sensor communications and allow sensors to estimate the principal covariance eigenspace in a distributed fashion. Two different type of algorithms are obtained that can applied in different sensor network architectures.The proposed distributed principal eigenspace estimation framework is used in a denoising application, where the sensor data are projected onto the estimated principal space to reduce the noise and improve overall data quality. To this end, a novel distributed denoising scheme is presented in this thesis. Extensive numerical tests using both synthetic and real data demonstrate that the proposed framework has the potential to achieve a considerably faster convergence rate and better steady-state estimation performance compared to existing alternatives. Further, the proposed distributed algorithms exhibit robustness in the presence of inter-sensor communication noise whereas alternative techniques do not converge in the presence of random perturbations.