Main Content

signalTimeFrequencyFeatureExtractor

Streamline signal time-frequency feature extraction

Since R2024a

    Description

    Use a signalTimeFrequencyFeatureExtractor object to extract time-frequency-domain features from a signal. You can use the extracted features to train a machine learning model or a deep learning network.

    Creation

    Description

    sFE = signalTimeFrequencyFeatureExtractor creates a signalTimeFrequencyFeatureExtractor object with default property values.

    example

    sFE = signalTimeFrequencyFeatureExtractor(Name=Value) sets property values to the signalTimeFrequencyFeatureExtractor object. For example, signalTimeFrequencyFeatureExtractor(FrameSize=30,FrameOverlapLength=6) creates a signalTimeFrequencyFeatureExtractor object that partitions a signal into overlapping 30-sample frames to extract time-frequency features from each frame.

    Properties

    expand all

    Main Properties

    Number of samples per frame, specified as a positive integer. The object divides the signal into frames of the specified length and extracts features for each frame. If you do not specify FrameSize, or if you specify FrameSize as empty, the object extracts features for the whole signal.

    Data Types: single | double

    Number of samples between the start of frames, specified as a positive integer. The frame rate determines the distance in samples between the starting points of frames. If you specify FrameRate, then you must also specify FrameSize. If you do not specify FrameRate or FrameOverlapLength, then the object assumes FrameRate to be equal to FrameSize. You cannot specify FrameRate and FrameOverlapLength simultaneously.

    Data Types: single | double

    Number of overlapping samples between consecutive frames, specified as a positive integer. FrameOverlapLength must be less than or equal to the frame size. If you specify FrameOverlapLength, then you must also specify FrameSize. You cannot specify FrameOverlapLength and FrameRate simultaneously.

    Data Types: single | double

    Input sample rate, specified as a positive scalar in hertz.

    If you do not specify SampleRate, the extract function of the object assumes the signal sampling rate as Hz.

    Data Types: single | double

    Format of the features generated by the extract function, specified as one of these:

    • "matrix" — Columns correspond to feature values.

    • "table" — Each table variable corresponds to a feature value.

    Data Types: char | string

    Rule to handle incomplete frames, specified as one of these:

    • "drop" — Drop the incomplete frame and do not use it to compute features.

    • "zeropad" — Zero-pad the incomplete frame and use it to compute features.

    This rule applies when the current frame size is less than the specified FrameSize property.

    Data Types: char | string

    Time-frequency analysis method used to extract signal features from, specified as:

    • "spectrogram" — Short-time Fourier transform (stft)

    • "synchrosqueezedspectrogram" — Fourier synchrosqueezed transform (fsst)

    • "emd" — Empirical mode decomposition (emd)

    • "vmd" — Variational mode decomposition (vmd)

    • "scalogram" — Continuous wavelet transform (cwt (Wavelet Toolbox)) magnitude

    • "synchrosqueezedscalogram" — Wavelet synchrosqueezed transform (wsst (Wavelet Toolbox)) magnitude

    • "wavelet" — Maximal overlap discrete wavelet transform (modwt (Wavelet Toolbox))

    • "waveletpacket" — Maximal overlap discrete wavelet packet transform (modwpt (Wavelet Toolbox))

    See Time-Frequency Gallery for more information on time-frequency analysis methods.

    Depending on the value you set in Transform, you can extract the following time-frequency features.

    Features to ExtractSupported Values for Transform
    "spectrogram",
    "synchrosqueezedspectrogram",
    "synchrosqueezedscalogram"
    "emd""vmd""scalogram""wavelet",
    "waveletpacket"
    SpectralKurtosis    
    SpectralSkewness    
    SpectralCrest    
    SpectralFlatness    
    SpectralEntropy    
    TFRidges    
    InstantaneousBandwidth    
    InstantaneousFrequency 
    InstantaneousEnergy  
    MeanEnvelopeEnergy    
    WaveletEntropy    
    TimeSpectrum    
    ScaleSpectrum    

    Note

    • You must have a Wavelet Toolbox™ license to specify Transform as "scalogram", "synchrosqueezedscalogram", "wavelet", or "waveletpacket".

    • You cannot specify FrameSize, FrameRate, or FrameOverlapLength when Transform is "emd" or "vmd".

    Data Types: char | string

    Methods to convert feature vectors to scalar values, specified as a timeFrequencyScalarFeatureOptions object.

    You can specify methods to extract scalar feature values from the Features to Extract. Specify scalarization methods to a feature extractor object by using the ScalarizationMethod name-value argument or the setScalarizationMethods function.

    • If you specify ScalarizationMethod, the signalTimeFrequencyFeatureExtractor object returns the corresponding scalar values for each feature vector.

      To convert a feature vector to scalar feature values:

      • You must enable the feature for extraction by setting the feature name in the signalTimeFrequencyFeatureExtractor object to true.

      • You must specify the desired scalarization methods for each feature name using a cell array of character vectors or a string array and store the information in a timeFrequencyScalarFeatureOptions object.

      After that, the extract function:

      • Extracts the vectors corresponding to each enabled feature.

      • Takes the list of scalarization methods compiled by the object and for each method computes the corresponding scalar value.

      • Concatenates the vector features and the scalar features.

    • If you do not specify ScalarizationMethod, the signalTimeFrequencyFeatureExtractor object does not perform any scalarization.

    For more information about scalarization methods, see Scalarization Methods for Signal Features in Time-Frequency Domain.

    Features to Extract

    You can enable signal features using name-value arguments from the time-frequency signal representation that you specify in Transform.

    Example: signalTimeFrequencyFeatureExtractor(Transform="emd",InstantaneousEnergy=true,MeanEnvelopeEnergy=true)

    Option to extract the spectral kurtosis of the time-frequency signal representation, specified as true or false.

    If you specify SpectralKurtosis as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the spectral kurtosis feature, see spectralKurtosis.

    Data Types: logical

    Option to extract the spectral skewness of the time-frequency signal representation, specified as true or false.

    If you specify SpectralSkewness as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the spectral skewness feature, see spectralSkewness.

    Data Types: logical

    Option to extract the spectral crest of the time-frequency signal representation, specified as true or false.

    If you specify SpectralCrest as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the spectral crest feature, see spectralCrest.

    Data Types: logical

    Option to extract the spectral flatness of the time-frequency signal representation, specified as true or false.

    If you specify SpectralFlatness as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the spectral flatness feature, see spectralFlatness.

    Data Types: logical

    Option to extract the spectral entropy of the time-frequency signal representation, specified as true or false.

    If you specify SpectralEntropy as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the spectral entropy feature, see spectralEntropy.

    Data Types: logical

    Option to extract the time-frequency ridges of the time-frequency signal representation, specified as true or false.

    If you specify TFRidges as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the time-frequency ridge feature, see tfridge.

    Data Types: logical

    Option to extract the instantaneous bandwidth of the time-frequency signal representation, specified as true or false.

    If you specify InstantaneousBandwidth as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the instantaneous bandwidth feature, see instbw.

    Data Types: logical

    Option to extract the instantaneous frequency of the time-frequency signal representation, specified as true or false.

    If you specify InstantaneousFrequency as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the instantaneous frequency feature, see instfreq or hht.

    Data Types: logical

    Option to extract the instantaneous energy of the time-frequency signal representation, specified as true or false.

    If you specify InstantaneousEnergy as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the instantaneous energy feature, see hht.

    Data Types: logical

    Option to extract the mean energy of the upper and lower envelopes for each intrinsic mode function (IMF) of the time-frequency signal representation, specified as true or false.

    If you specify MeanEnvelopeEnergy as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the mean-envelope energy feature, see emd.

    Data Types: logical

    Option to extract the wavelet entropy of the time-frequency signal representation, specified as true or false.

    If you specify WaveletEntropy as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the wavelet entropy feature, see wentropy (Wavelet Toolbox).

    Data Types: logical

    Option to extract the time-averaged wavelet spectrum of the time-frequency signal representation, specified as true or false.

    If you specify TimeSpectrum as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the time spectrum feature, see timeSpectrum (Wavelet Toolbox).

    Data Types: logical

    Option to extract the scale-averaged wavelet spectrum of the time-frequency signal representation, specified as true or false.

    If you specify ScaleSpectrum as true:

    • The signalTimeFrequencyFeatureExtractor object enables this feature for extraction.

    • The extract object function extracts this feature and concatenates it with all the other features that you enable in the signalTimeFrequencyFeatureExtractor object.

    For more information about the scale spectrum feature, see scaleSpectrum (Wavelet Toolbox).

    Data Types: logical

    Object Functions

    extractExtract time-domain, frequency-domain, or time-frequency-domain features
    generateMATLABFunctionCreate MATLAB function compatible with C/C++ code generation
    getExtractorParametersGet current parameter values of feature extractor object
    getScalarizationMethodsGet current scalarization methods of feature extractor object
    setExtractorParametersSet nondefault parameter values for feature extractor object
    setScalarizationMethodsSet scalarization methods for feature extractor object

    Examples

    collapse all

    Extract the spectral kurtosis and instantaneous frequency from the spectrogram of a quadratic-swept chirp signal.

    Generate a chirp with quadratic instantaneous frequency deviation. The chirp is sampled at 2π Hz for 60 seconds. The instantaneous frequency is 0.5 Hz at t = 0 and crosses 2 Hz at t = 60 seconds.

    fs = 2*pi;
    t = 0:1/fs:60;
    x = chirp(t,0.5,60,2,"quadratic");

    Create a signalTimeFrequencyFeatureExtractor object and enable the SpectralKurtosis and InstantaneousFrequency time-frequency-domain features.

    tfFE = signalTimeFrequencyFeatureExtractor( ...
        SpectralKurtosis=true,InstantaneousFrequency=true)
    tfFE = 
      signalTimeFrequencyFeatureExtractor with properties:
    
       Properties
                  FrameSize: []
                  FrameRate: []
                 SampleRate: []
        IncompleteFrameRule: "drop"
              FeatureFormat: "matrix"
                  Transform: "Spectrogram"
        ScalarizationMethod: [1x1 timeFrequencyScalarFeatureOptions]
    
       Enabled Features
         SpectralKurtosis, InstantaneousFrequency
    
       Disabled Features
         SpectralSkewness, SpectralCrest, SpectralFlatness, SpectralEntropy, TFRidges, InstantaneousBandwidth
         MeanEnvelopeEnergy, InstantaneousEnergy, WaveletEntropy, TimeSpectrum, ScaleSpectrum
    
    
       
    

    Extract the features from the chirp signal. Reshape the result into the number of features extracted. Plot the features.

    features = extract(tfFE,x);
    
    featuresRows = reshape(features,[],2);
    stackedplot(featuresRows,"*",...
        DisplayLabels=["Spectral Kurtosis" "Instantaneous Frequency"])
    grid on

    Set up a three-signal signalDatastore object and a signalTimeFrequencyFeatureExtractor object. Extract the spectral skewness and time-frequency ridges to the signals.

    Create a signalDatastore object with three oscillating signals sampled at 3000 Hz for 3 seconds.

    Fs = 3000;
    t = 0:1/Fs:3;
    members = {chirp(t,300,t(end),800); ...
            2*chirp(t,200,t(end),1000,"quadratic",[],"concave"); ...
            vco(sin(2*pi*t),[0.1 0.4]*Fs,Fs)};
    sds = signalDatastore(members,SampleRate=Fs);

    Create a signalTimeFrequencyFeatureExtractor object defining a sample rate. Enable the spectral skewness and time-frequency ridges as features to extract.

    tfFE = signalTimeFrequencyFeatureExtractor(SampleRate=Fs, ...
                  SpectralSkewness=true,TFRidges=true);

    Set up the extractor parameters for a signalTimeFrequencyFeatureExtractor object. Specify the range between Fs/5 and Fs/2.5 to extract the spectral skewness. Set the penalty parameter to zero to extract the time-frequency ridges.

    setExtractorParameters(tfFE,"SpectralSkewness",Range=[Fs/5 Fs/2.5]);
    setExtractorParameters(tfFE,"TFRidges",Penalty=0);

    Extract and plot the time-frequency features to all the signals from a signalDatastore object.

    tiledlayout flow
    signalID = 1;
    while hasdata(sds)
        [data,info] = read(sds);
    
        % Extract features and remove small values.
        [features,infofeatures] = extract(tfFE,data);
        features(abs(features)<1e-4)=0;
    
        % Plot the spectral skewness
        nexttile
        plot(features(infofeatures.SpectralSkewness))
        title("Spectral Skewness")
        subtitle("Signal "+signalID)
        axis tight
        % Plot the time-frequency ridges
        nexttile
        plot(features(infofeatures.TFRidges))
        title("Time-Frequency Ridges")
        subtitle("Signal "+signalID)
        axis tight
    
        signalID = signalID +1;
    end

    Specify scalarization methods to extract the scalar impulse factor and peak value of the instantaneous energy feature for an oscillating signal.

    Set the impulse factor and the peak value as scalarization methods for the instantaneous energy time-frequency domain feature.

    opts = scalarFeatureOptions("timefrequency", ...
        InstantaneousEnergy={'ImpulseFactor';'PeakValue'})
    opts = 
      timeFrequencyScalarFeatureOptions with properties:
    
              SpectralKurtosis: [0x0 string]
              SpectralSkewness: [0x0 string]
                 SpectralCrest: [0x0 string]
              SpectralFlatness: [0x0 string]
               SpectralEntropy: [0x0 string]
                      TFRidges: [0x0 string]
        InstantaneousBandwidth: [0x0 string]
        InstantaneousFrequency: [0x0 string]
           InstantaneousEnergy: [2x1 string]
            MeanEnvelopeEnergy: [0x0 string]
                  TimeSpectrum: [0x0 string]
                WaveletEntropy: [0x0 string]
                 ScaleSpectrum: [0x0 string]
                           All: [0x0 string]
    
    

    Create a signalTimeFrequencyFeatureExtractor object that returns scalar values for the instantaneous energy feature from the variational mode decomposition of a signal.

    tfFE = signalTimeFrequencyFeatureExtractor(Transform="vmd", ...
        InstantaneousEnergy=true,ScalarizationMethod=opts);

    Extract the vector and scalar features for a voltage-controlled oscillating signal.

    fs = 10000;
    t = 0:1/fs:2;
    x = vco(sawtooth(2*pi*t,0.75),[0.1 0.4]*fs,fs);
    [features,info] = extract(tfFE,x);

    Display the scalar feature values.

    % Impulse Factor of the Instantaneous Energy Vector
    features(info.InstantaneousEnergyImpulseFactor)
    ans = 8.7551
    
    % Peak Value of the Instantaneous Energy Vector
    features(info.InstantaneousEnergyPeakValue)
    ans = 1.3385
    

    Extract the time-frequency ridges from a spectrogram using a signalTimeFrequencyFeatureExtractor object.

    Load a data file containing an echolocation pulse batsignal, emitted by a big brown bat (Eptesicus fuscus) and measured with a sample rate DT of 7 microseconds. For more information, see the example Find and Track Ridges Using Reassigned Spectrogram.

    load batsignal

    Create a MATLAB® timetable using the signal and the time information.

    t = (0:length(batsignal)-1)*DT;
    sg = timetable(seconds(t)',batsignal);

    Create an object to extract time-frequency ridges from the spectrogram of a signal.

    fs = 1/DT;
    sTFFE = signalTimeFrequencyFeatureExtractor( ...
        Transform="spectrogram",SampleRate=fs,TFRidges=true);

    Set the leakage, the time resolution, and the overlap percent parameters for the spectrogram analysis method.

    lk = 0.9;
    tRes = 0.00028; % seconds
    oPercent = 85;
    
    setExtractorParameters(sTFFE,"spectrogram", ...
        Leakage=lk,TimeResolution=tRes,OverlapPercent=oPercent);

    Set the number of ridges, number of frequency bins, and penalty parameters for the time-frequency ridges feature.

    nRidges = 3;
    
    setExtractorParameters(sTFFE,"TFRidges", ...
      NumRidges=nRidges,NumFrequencyBins=128,Penalty=0.01);

    Extract and plot the time-frequency ridges.

    tfRidges = reshape(extract(sTFFE,sg),[],nRidges);
    
    tStamps = seconds(tRes/2+(1-oPercent/100)*tRes*(0:length(tfRidges)-1));
    plot(tStamps,tfRidges/1000)
    xlabel("Time")
    ylabel("Frequency (kHz)")

    Extract the instantaneous frequency of a signal using the wavelet synchrosqueezed transform (WSST), the time-averaged wavelet spectrum using the continuous wavelet transform (CWT), and the wavelet entropy using the maximal overlap discrete wavelet transform (MODWT).

    Load an ECG signal corresponding to record 200 of the MIT-BIH Arrhythmia Database [1]. The workspace variable ecgsig contains the signal, and the variable tm contains the sample times. The sample rate is approximately 400 Hz.

    load mit200
    Fs = 400;

    The signal has 10,000 samples. Reshape the signal into a 1000-by-10 matrix. Each column represents one channel in a multichannel signal. Create a signalDatastore object from the matrix.

    ecgsig = reshape(ecgsig,[],10);
    ecgsig = num2cell(ecgsig,1);
    sds = signalDatastore(ecgsig,SampleRate=Fs);

    Set the mean and standard deviation as scalarization methods for the wavelet entropy feature.

    opts = scalarFeatureOptions("timefrequency", ...
        WaveletEntropy={'Mean','StandardDeviation'});

    Create a signalTimeFrequencyFeatureExtractor object that returns:

    • Wavelet entropy and the scalar values specified in opts

    • Instantaneous frequency from a WSST

    • Time-averaged spectrum from a CWT

    You can extract instantaneous frequency from time-frequency signal representations created using methods such as the short-time Fourier transform ("spectrogram"), empirical mode decomposition ("emd"), and the maximal overlap discrete wavelet packet transform ("waveletpacket"). Set the time-frequency analysis method to "synchrosqueezedscalogram". The CWT is the only analysis method you can use to extract the time-averaged wavelet spectrum. The signalTimeFrequencyFeatureExtractor object uses the MODWT, the default analysis method of the wentropy function, to extract the wavelet entropy.

    tfFE = signalTimeFrequencyFeatureExtractor( ...
        SampleRate=Fs, ...
        Transform="synchrosqueezedscalogram", ...
        InstantaneousFrequency=true, ...
        WaveletEntropy=true, ...
        TimeSpectrum=true, ...
        ScalarizationMethod=opts);

    Preallocate arrays to store the extracted features, including the wavelet entropy, of all the signals. To obtain the dimensions, extract the features from the first signal.

    data = read(sds);
    [~,infofeatures] = extract(tfFE,data);
    
    numInstFrq = numel(infofeatures.InstantaneousFrequency);
    numTimeSpect = numel(infofeatures.TimeSpectrum);
    numWaveEnt = numel(infofeatures.WaveletEntropy);
    
    instFrq = zeros(10,numInstFrq);
    timeSpect = zeros(10,numTimeSpect);
    waveEnt = zeros(numWaveEnt,1);
    waveEntMean = zeros(10,1);
    waveEntStd = zeros(10,1);

    Reset the datastore. Extract the features from all the signals.

    reset(sds)
    ctr = 0;
    
    while hasdata(sds)
        ctr = ctr+1;
        data = read(sds);
        [features,infofeatures] = extract(tfFE,data);
        instFrq(ctr,:) = ...
            features(infofeatures.InstantaneousFrequency);
        timeSpect(ctr,:) = ...
            features(infofeatures.TimeSpectrum);
        waveEnt(:,ctr) =  ...
            features(infofeatures.WaveletEntropy);
        waveEntMean(ctr,:) = ...
            features(infofeatures.WaveletEntropyMean);
        waveEntStd(ctr,:) = ...
            features(infofeatures.WaveletEntropyStandardDeviation);
    end

    For each signal, compute the mean and standard deviation directly from the extracted wavelet entropy. Confirm they are equal to the extracted scalar values.

    [mean(waveEnt,1)' waveEntMean]
    ans = 10×2
    
        0.7675    0.7675
        0.7941    0.7941
        0.7533    0.7533
        0.7465    0.7465
        0.8055    0.8055
        0.8141    0.8141
        0.8048    0.8048
        0.7833    0.7833
        0.7999    0.7999
        0.7910    0.7910
    
    
    [std(waveEnt,0,1)' waveEntStd]
    ans = 10×2
    
        0.1714    0.1714
        0.1258    0.1258
        0.2063    0.2063
        0.1945    0.1945
        0.1367    0.1367
        0.1123    0.1123
        0.1235    0.1235
        0.1228    0.1228
        0.1123    0.1123
        0.1168    0.1168
    
    

    Display the time-averaged wavelet spectrum of the signals as a waterfall plot.

    waterfall(timeSpect)
    title("Time-Averaged Wavelet Spectrum")
    xlabel("Frequency Index")
    ylabel("Channel Index")
    zlabel("Power")

    Display the instantaneous frequency of the signals as a waterfall plot.

    waterfall(instFrq)
    view(30,40)
    title("Instantaneous Frequency")
    xlabel("Sample")
    ylabel("Channel Index")
    zlabel("Frequency")

    More About

    expand all

    References

    [1] Moody, G.B., and R.G. Mark. “The Impact of the MIT-BIH Arrhythmia Database.” IEEE Engineering in Medicine and Biology Magazine 20, no. 3 (June 2001): 45–50. https://doi.org/10.1109/51.932724.

    Extended Capabilities

    Version History

    Introduced in R2024a