Detect Human Presence Using WLAN Signals and Deep Learning
This example shows how to use a convolutional neural network (CNN) to detect human presence by using the channel state information (CSI) in wireless local area networks. In this example, you can capture beacon frames from real routers with a software defined radio (SDR) to generate your own sensing dataset, or you can use prerecorded data. Then you train a CNN that detects human presence from the live SDR captures or the prerecorded data.
WLAN sensing (also known as Wi-Fi™ sensing) uses Wi-Fi networks for applications such as gesture recognition, location tracking, monitoring vital signs, and human presence detection. WLAN sensing is currently being standardized by the IEEE® 802.11bf task group . This technology can sense presence, range, velocity, and location of objects in various environments through subtle changes in the communication channel. CSI is the most common metric for determining the changes in the channel . However, CSI is not typically exposed to you as a user of commercial Wi-Fi devices.
Using WLAN Toolbox™ with a supported SDR, you can capture WLAN waveforms, detect beacon packets, and extract CSI. From this CSI, you create a collection of periodogram images to use as the training and validation datasets for deep learning. Using Deep Learning Toolbox™, you train a CNN and use the trained network for live detection of human presence. Alternatively, you can use the prerecorded dataset to train the CNN and evaluate the presence detection performance against the test dataset. Beacon packets are useful for WLAN sensing as access points (APs) transmit beacon packets periodically. As a result, the sensing resolution is independent of the traffic load at the AP in the temporal domain.
This section explains how to set up an SDR for capturing the data that will be used in training and live detection of human presence. If you do not have SDR hardware and plan to use the prerecorded data, you can skip this section.
This example supports these SDRs:
ADALM-Pluto from the Communications Toolbox Support Package for Analog Devices® ADALM-Pluto Radio
USRP™ E310/E312 from the Communications Toolbox Support Package for USRP™ Embedded Series Radio
AD936x/FMCOMMS5 from the Communications Toolbox Support Package for Xilinx® Zynq®-Based Radio
USRP™ N200/N210/USRP2/N320/N321/B200/B210/X300/X310 from the Communications Toolbox Support Package for USRP™ Radio
1) Ensure that you have installed the appropriate support package for the SDR that you intend to use and that you have configured the hardware accordingly.
2) To use your SDR as the data source for training the neural network and then inferencing with the trained CNN, select the
useSDR check box.
useSDR = false; % You can skip this section if unchecked
3) Specify your capture device (
rxsim.DeviceName), radio gain (
rxsim.RadioGain), and a channel number (
rxsim.ChannelNumber) for your operation frequency band (
rxsim.FrequencyBand). These settings determine the center frequency from which your SDR captures beacon packets. If you do not know a channel number, use the OFDM Beacon Receiver Using Software-Defined Radio example to determine which channel(s) contain beacon packets.
if useSDR % User-defined parameters rxsim.DeviceName = "Pluto"; rxsim.RadioGain = 15; % Can be 'AGC Slow Attack', 'AGC Fast Attack', or an integer value. See relevant documentation for selected radio for valid values of radio gain. rxsim.ChannelNumber = 36; % Valid values for 5 GHz band are integers in range [1, 200] rxsim.FrequencyBand = 5; % in GHz % Set up SDR receiver object rx = hSDRReceiver(rxsim.DeviceName); rx.SampleRate = 20e6; % Configured for 20 MHz since this is beacon transmission bandwidth rx.Gain = rxsim.RadioGain; rx.CenterFrequency = wlanChannelFrequency(rxsim.ChannelNumber,rxsim.FrequencyBand); rx.ChannelMapping = 1; rx.OutputDataType = 'single'; rxsim.SDRObj = rx; end
4) Configure these capture parameters:
The number of beacon packets to be extracted from each capture (
The number of captures needed to create the dataset and train the neural network (
The beacon interval (
rxsim.BeaconInterval). The default beacon interval for a commercial AP is 100 time units (TUs), where 1 TU equals 1024 microseconds. You typically do not need to adjust the beacon interval value unless you have adjusted the interval of your access point (AP).
Filter the beacons based on their SSID if more than one SSID exists on the same channel (
if useSDR %#ok<*UNRCH> % User defined parameters rxsim.NumCaptures = 10; rxsim.NumPacketsPerCapture = 8; rxsim.BeaconInterval = 100; % in time units (TUs). The default value in typical APs is 100. rxsim.BeaconSSID = ""; % Optional % Calculated parameters rxsim.CaptureDuration = rxsim.BeaconInterval*milliseconds(1.024)*rxsim.NumPacketsPerCapture + milliseconds(5.5); % Add 5.5 ms for beacons located at the end of the waveform % SDR setup is complete msgbox("SDR object configuration is complete! Run steps 1 and 2 to capture data.") return % Stop execution end
This example uses the following process to obtain CSI images for human presence detection:
When you use an SDR, the system captures an over-the-air waveform and performs receiver processing in WLAN Toolbox to extract the CSI from beacon packets. It does the same for all the packets in a capture to form a
rxsim.NumPacketsPerCapture raw CSI array. Repeat this process for multiple captures to create an array of images. When you create the training data, the system assigns "no-presence" or "presence" labels to the captures. It obtains the periodogram representation of the raw CSI image array by using 2D-FFT, applying
[0 1] range scaling, and removing DC components along the and axes. CSI periodogram images make it easier to focus on the CSI changes caused by human motion by discarding the effects of imperfections in the hardware, software, or environment. This focus also improves the training performance. Next, you use the periodogram images as an input dataset for CNN training and validation. Finally, you use the trained CNN with live SDR captures to detect presence or test the performance of the CNN by using a subset of the prerecorded dataset. When capturing training data or performing live detection, you can visualize the magnitude spectrum and periodogram representation of the raw CSI.
This process was applied to obtain the prerecorded CSI dataset, which consists of 500 samples, with 250 labeled "no-presence" and 250 labeled "presence". When this dataset was being created, the
rxsim.NumCaptures parameter was set to 250. An ADALM-Pluto SDR captured over-the-air beacon frames in a meeting room that contained furniture and had floor dimensions of 3m-by-3m. The beacon interval (
rxsim.BeaconInterval) of the AP was set to 20 TUs. Two different human presence states were recorded: no presence (empty room) and presence (one person walking a random path).
Step 1: Create "no-presence" Data
In this section, you generate the "no-presence" data (
dataNoPresence) and associated labels vector (
If you are using an SDR, make sure no human presence exists in the environment before you create the data. The
captureCSIDatasetfunction displays the current capture number, status, and timestamp (
If you are using the prerecorded dataset, the
loadCSIDatasetfunction generates the
timestampNoPresenceparameters and visualizes the CSI dataset.
To generate the data, click Create "no-presence" Data. Two plots are generated for each capture:
Magnitude spectrum of the captured CSI. Although the captured CSI is a complex valued array, this example uses the magnitude response of the CSI for visualization.
CSI periodogram image.
In "no-presence" data, the power content of the CSI spreads over both axes of the periodogram representation due to the lack of time-domain disturbance induced by the motion.
if useSDR % Capture live CSI with SDR [dataNoPresence,labelNoPresence,timestampNoPresence] = captureCSIDataset(rxsim,"no-presence"); else % No SDR hardware, load CSI dataset [dataNoPresence,labelNoPresence,timestampNoPresence] = loadCSIDataset("dataset-no-presence.mat","no-presence"); end
Dimensions of the no-presence dataset (numSubcarriers x numPackets x numCaptures): [52 8 250]
Step 2: Create "presence" Data
In this section, you generate the "presence" data (
dataPresence) and associated labels vector (
If you are capturing data with an SDR, make sure constant human motion exists in the environment before you create the data. Otherwise, the quality of the captured data could significantly worsen the performance during training and inference. The
captureCSIDatasetfunction displays the current capture number, status, timestamp (
timestampPresence), and CSI visualization.
If you are using the prerecorded dataset, the
loadCSIDatasetfunction generates the
timestampPresenceparameters and visualizes randomly selected samples of the data. The "presence" data in the prerecorded dataset was collected with a person walking constantly in the room for the duration of the capture.
The capture visualization techniques are the same as in step 1. For the "presence" data, the periodogram plot shows focus of the CSI power content in both axes near the low frequency region that corresponds to the center of the image.
if useSDR % Capture CSI with SDR [dataPresence,labelPresence,timestampPresence] = captureCSIDataset(rxsim,"presence"); else % No SDR hardware, load CSI dataset [dataPresence,labelPresence,timestampPresence] = loadCSIDataset("dataset-presence.mat","presence"); end
Dimensions of the presence dataset (numSubcarriers x numPackets x numCaptures): [52 8 250]
Step 3: Create and Train CNN
CNNs are one of the most computationally efficient, accurate, and scalable approaches to problems involving deep learning. Their predominant application is to deep learning tasks based on computer vision . In this section, you define the CNN architecture and divide the data into training (
trainingData) and validation (
validationData) datasets to evaluate the training and final network performance.
If you are using an SDR, you can evaluate the accuracy of the trained CNN using the live capture results. The example does not create a test set if you use an SDR.
If you are using the precaptured data, you can evaluate the performance of the trained CNN using a test set (
To create, train and validate the CNN click Create CNN and Train.
trainingRatio = 0.8; numEpochs = 10; sizeMiniBatch = 8;
trainValTestSplit function to create array datastore objects
trainingData. This process is based on the
trainingRatio parameter. A training dataset (
trainingData), which comprises 80% of the whole dataset, updates the weights of the CNN to fit the general characteristics of the data. This process is based on the training labels contained in the array datastore object. A validation dataset (
validationData), which comprises 10% of the whole dataset, ensures that the model is learning the general behavior of the system rather than memorizing the patterns in it. Finally, a test dataset, which also comprises 10% of the whole dataset, evaluates the performance of the trained CNN using unseen samples of the data. If you are using an SDR, the size of
validationData becomes 20% of the whole dataset, because the testing is based on the human presence status inferences of the trained CNN.
[trainingData,validationData,testData,imgInputSize,numClasses] = trainValTestSplit(useSDR,trainingRatio,dataNoPresence,labelNoPresence,dataPresence,labelPresence);
CSI image input size: [51 7] Number of training images: 400
The example uses CSI periodogram images with size (
rxsim.NumPacketsPerCapture-1) as the training input. Use the Create Simple Deep Learning Neural Network for Classification (Deep Learning Toolbox) example to learn more about the individual layers and their roles in the architecture.
cnn = [ imageInputLayer(imgInputSize,'Normalization','none') convolution2dLayer(3,8,'Padding','same') batchNormalizationLayer reluLayer maxPooling2dLayer(2,'Stride',2) convolution2dLayer(3,16,'Padding','same') batchNormalizationLayer reluLayer maxPooling2dLayer(2,'Stride',2) convolution2dLayer(3,32,'Padding','same') batchNormalizationLayer reluLayer dropoutLayer(0.1) fullyConnectedLayer(numClasses) softmaxLayer classificationLayer]; cnn = layerGraph(cnn); disp(cnn.Layers)
16x1 Layer array with layers: 1 'imageinput' Image Input 51x7x1 images 2 'conv_1' 2-D Convolution 8 3x3 convolutions with stride [1 1] and padding 'same' 3 'batchnorm_1' Batch Normalization Batch normalization 4 'relu_1' ReLU ReLU 5 'maxpool_1' 2-D Max Pooling 2x2 max pooling with stride [2 2] and padding [0 0 0 0] 6 'conv_2' 2-D Convolution 16 3x3 convolutions with stride [1 1] and padding 'same' 7 'batchnorm_2' Batch Normalization Batch normalization 8 'relu_2' ReLU ReLU 9 'maxpool_2' 2-D Max Pooling 2x2 max pooling with stride [2 2] and padding [0 0 0 0] 10 'conv_3' 2-D Convolution 32 3x3 convolutions with stride [1 1] and padding 'same' 11 'batchnorm_3' Batch Normalization Batch normalization 12 'relu_3' ReLU ReLU 13 'dropout' Dropout 10% dropout 14 'fc' Fully Connected 2 fully connected layer 15 'softmax' Softmax softmax 16 'classoutput' Classification Output crossentropyex
% Definition of the training options options = trainingOptions('adam', ... LearnRateSchedule='piecewise', ... InitialLearnRate=0.001, ... % learning rate MaxEpochs=numEpochs, ... MiniBatchSize=sizeMiniBatch, ... ValidationData=validationData, ... Shuffle='every-epoch', ... ExecutionEnvironment='auto',... Verbose=true);
Train the deep network architecture
cnn using the training dataset
trainingData and the training options defined by
% Train the network trainedCNN = trainNetwork(trainingData,cnn,options);
Training on single CPU. |======================================================================================================================| | Epoch | Iteration | Time Elapsed | Mini-batch | Validation | Mini-batch | Validation | Base Learning | | | | (hh:mm:ss) | Accuracy | Accuracy | Loss | Loss | Rate | |======================================================================================================================| | 1 | 1 | 00:00:00 | 37.50% | 84.00% | 0.8120 | 0.6155 | 0.0010 | | 1 | 50 | 00:00:01 | 100.00% | 100.00% | 0.0085 | 0.0150 | 0.0010 | | 2 | 100 | 00:00:02 | 100.00% | 100.00% | 0.0059 | 0.0055 | 0.0010 | | 3 | 150 | 00:00:04 | 100.00% | 100.00% | 0.0134 | 0.0179 | 0.0010 | | 4 | 200 | 00:00:05 | 100.00% | 100.00% | 0.0029 | 0.0024 | 0.0010 | | 5 | 250 | 00:00:06 | 100.00% | 100.00% | 0.0123 | 0.0047 | 0.0010 | | 6 | 300 | 00:00:08 | 100.00% | 100.00% | 0.0012 | 0.0007 | 0.0010 | | 7 | 350 | 00:00:10 | 100.00% | 100.00% | 0.0008 | 0.0005 | 0.0010 | | 8 | 400 | 00:00:13 | 100.00% | 100.00% | 0.0006 | 0.0192 | 0.0010 | | 9 | 450 | 00:00:14 | 100.00% | 100.00% | 0.0001 | 0.0006 | 0.0010 | | 10 | 500 | 00:00:15 | 100.00% | 100.00% | 0.0002 | 0.0063 | 0.0010 | |======================================================================================================================| Training finished: Max epochs completed.
Step 4: Evaluate Performance
In this section, you evaluate the performance of the neural network using training data or live inference with an SDR.
If you are using an SDR, this example uses the
livePresenceDetectionfunction to capture over-the-air beacons and infer human presence ("no-presence" or "presence") using the trained CNN that is for each capture. The function plots the instantaneous raw CSI and CSI periodogram plots, as described in steps 1 and 2, for each capture. The timestamped results for the 10 latest predictions are plotted. By default,
livePresenceDetectionruns for 20 captures. You can set
numCapturesargument to any integer value. For continuous capturing, you can set
Inf. The function returns results in the 1-by-2 cell array
sensingResults.The first element is the prediction results and the second element is the timestamps. Note that
sensingResultswill not be created if
livePresenceDetectionis in the continuous capturing mode.
If you are using the prerecorded data, this example uses the
testPresenceDetectionfunction to generate a confusion matrix in which the rows correspond to the ground truth values ("no-presence" or "presence") and the columns correspond to the CNN predictions ("no-presence" or "presence"). The diagonal elements represent the percentage of samples that are correctly classified. The off-diagonal elements are the percentage of incorrectly classified observations. The sensing accuracy value is calculated using the
testDataand returned in
To run the evaluation, click Evaluate Performance.
if useSDR % Predict movement from the live SDR captures sensingResults = livePresenceDetection(rxsim,trainedCNN,20); else % Predict movement by using the test set sensingResults = testPresenceDetection(testData,trainedCNN); end
 IEEE P802.11 - TASK GROUP BF (WLAN SENSING), https://www.ieee802.org/11/Reports/tgbf_update.htm.
 IEEE 802.11-21/0407r2 - Multi-Band WiFi Fusion for WLAN Sensing, March 2021, https://mentor.ieee.org/802.11/dcn/21/11-21-0407-00-00bf-multi-band-wifi-fusion-for-wlan-sensing.pptx.
 Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278–2324, Nov. 1998, doi: 10.1109/5.726791.