A Real-time Activity Recognition in a Congested Wireless Environment

Abstract

**Please note that the full text is embargoed until 8/16/2024** ABSTRACT: This dissertation reports on how to achieve real-time activity recognition in a congested wireless environment. Recently, human activity recognition with WiFi has been the focus of many researchers due to the limitations of legacy approaches like video cameras and sensors. Users have concerns with privacy when it comes to video activity recognition. Likewise, activity recognition sensors can be expensive, obtrusive, and inconvenient to be worn for an extended period of time. The urgency for implementing contactless activity recognition has also been accelerated due to changes in society and social interaction because of the COVID-19 pandemic. Many public facilities like restaurants, shopping malls, and airports now render services with limited physical contact and require contactless activity recognition. To address the concerns, many recent works have developed activity recognition systems based on Channel State Information (CSI) from commercially available WiFi devices. CSI eliminates the cost of buying sensors and privacy concerns of using video activity recognition, and it is considered a promising alternative to legacy systems. WiFi is a suitable choice because of its signal coverage and ubiquity. However, implementing WiFi activity recognition in public places comes with several challenges: shared wireless spectrum with other wireless devices, adaptability of the recognition model to different locations with differing environmental layouts and multipath effects, and real-time recognition of several activities on a single stream. The challenges are addressed in this dissertation with a three-phase system. In the first phase, the system makes an intelligent decision by sensing the quality of the wireless spectrum and selecting a suitable channel for activity recognition using the entropy of CSI of the wireless spectrum. The second phase creates a robust and location-agnostic model that is resilient to varying environmental layouts and signal multipath. In the third and final phase, the system addresses real-time human activity recognition and its associated challenges.

Specifically, in the first phase, we employ an entropy-based WLAN channel allocation algorithm using CSI to combat the unreliability of the received signal strength indicator (RSSI) in measuring channel quality. CSI is a better measure of channel quality because it reflects both the time variation and frequency selectivity of a wireless link. Likewise, entropy has the property of representing the information-theoretical characteristics of multi-fractal systems by examining the underlying spectral power distributions and their probabilities. To measure channel quality, we capture the CSI as it is indicative of the current channel condition. The CSI values between the transmit-receive antenna pair contain the channel response for all the subcarriers. We calculate the channel singular value decomposition (SVD) for the channel response to obtain the non-negative real value across the transmit-receive antenna pair. The spectral entropy is obtained using the singular values representing the channel gains for the subcarriers and the probability of each channel gain. Consequently, we get the spectral entropy of overall wireless environment by placing the station's network interface card (NIC) in monitor mode to passively scan all the channels. Our algorithm is executed to achieve the objective of selecting a channel with maximum spectral entropy in a given measurement period.

In the second phase, we achieve location-independent activity recognition using unsupervised invariance induction framework. The technique is used to ensure fairness by eliminating the discrimination during activity recognition. Obtaining a consistent performance from an activity recognition model is hard. The challenge is due to diverse CSI variations from multipath propagation since gestures performed in different environments usually reflect signal paths differently. The environment and location information introduces bias. Our proposed work eliminates such bias by separating features for activity recognition from location-dependent features not contributing to the activity recognition. The feature separation module is an unsupervised invariance induction framework consisting of activity recognition, extraneous information removal and discriminators.

Finally in the last phase, we address the non-real-time challenge and propose a real-time object detection model for CSI-based multiple human activity recognition. Our model is an object detection deep learning that classifies an activity and localizes the activity in the context of the stream. To achieve that, we capture changes in both time and frequency of the signal simultaneously and apply a wavelet transformation by converting the CSI data from the WiFi signal to images. An object detection deep learning network using Mask R-CNN is used to achieve human activity classification, localization, and instance segmentation within a continuous stream generated from images. The result shows that model classification accuracy is 93.80 % on average and instance segmentation accuracy of 90.73 %. We evaluated the performance of our model with a non-real-time deep learning model for activity detection. We found that our model performed satisfactorily with a less than 8 % detection performance.