Eigenspace Model Based Error Concealment And Low Bit Rate Coding Of Face Sequences

Abstract

The emerging multimedia applications address the increasing demand for novel video coding systems to provide higher compression ratio while maintaining the high quality of the reconstruction. This research makes an effort under such context to investigate the application of principal component analysis (PCA) in the video coding area, especially for error concealment and very low bit rate face coding. PCA is a well known optimal linear scheme for dimension reduction in data analysis. The central idea of PCA is to reduce the dimensionality of a data set while retaining as much as possible the variation in the data set. Since PCA captures the statistical variations and global information efficiently, it is used in the proposed research to build the model of the target object or range of interest (ROI), and thereby a new model based framework is constructed for very low bit rate face coding and error concealment. The research focuses on building an efficient and accurate PCA model for very low bit rate coding and effective error concealment. The main limitation of PCA is that it cannot model the data set with large variations efficiently. An adaptive update scheme is investigated in this research to enhance the accuracy and efficiency of the eigenspace model. Computational complexity reduction is another important consideration for real time operation. An incremental mode PCA with missing data for eigenspace updating is proposed. Its effect on the model based error concealment scheme over different quantization levels, loss patterns and loss rates is analyzed. A novel model based and waveform based hybrid coding system aimed at very low bit rate face coding is also presented. Model based coding provides great potential for bit rate savings while model failures and unknown objects can be handled by waveform based coding. The two coding modes are combined under a rate-distortion framework, where Lagrangian cost function is used to determine the most efficient prediction for each block. Simulations show that the system can achieve high compression ratios while maintaining the robustness and generality, which indicate its potential use for videophone application.