A NOVEL SUPERVISED DIMENSIONALITY REDUCTION METHOD: INTEGRATING PCA WITH SVM

Abstract

Data curation and storage methods have changed over the past few decades with the use of new technologies, and gathering data on a huge number of features (dimensions) is now very common among diverse scientific and engineering fields. Prior to classification or regression, dimensionality reduction is necessary to eliminate irrelevant features and to deal with data with high dimensions. A number of numerical methods have already been proposed to reduce the dimension of data, for example, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Supervised Principal Component Analysis (SPCA).

In this dissertation, we will introduce a novel way of reducing dimensionality and classifying data simultaneously through a supervised approach that reduces dimension while classifying data. The objective of our model is to determine the projection matrix utilized in the dimensionality reduction procedure, as well as to determine the hyperplane of the classifier used for data classification. Since the supervised model is learning both the representation of the low-dimensional data and the classification simultaneously, our supervised model has the advantage of high accuracy as well as effective representation. Additionally, our model is capable of performing multi-class classification, i.e., it is capable of classifying data with more than two categories. Due to our model’s ability to use nonlinear mappings, we can also apply it to data sets with nonlinear and complex structures. Simulating the model and comparing it with the state-of-the-art dimensionality reduction and classification techniques demonstrate the model’s effectiveness and efficiency.