Preprocessing Data

Data cleaning, smoothing, grouping

Data can require preprocessing techniques to ensure accurate, efficient, or meaningful analysis. Data cleaning refers to methods for finding, removing, and replacing bad or missing data. Detecting local extrema and abrupt changes can help to identify significant data trends. Smoothing and detrending are processes for removing noise and polynomial trends from data, while scaling changes the bounds of the data. Grouping and binning methods identify data characteristics by groups.

Applications

Data Cleaner

Preprocess and organize column-oriented data

Tâches du Live Editor

Clean Missing Data	Find, fill, or remove missing data in the Live Editor
Clean Outlier Data	Find, fill, or remove outliers in the Live Editor
Compute by Group	Summarize, transform, or filter by group in the Live Editor
Find Change Points	Find abrupt changes in data in the Live Editor
Find Local Extrema	Find local maxima and minima in the Live Editor
Normalize Data	Center and scale data in the Live Editor
Smooth Data	Smooth noisy data in the Live Editor
Remove Trends	Remove polynomial trend from data in the Live Editor

Fonctions

développer tout

Missing Data and Outliers

`anymissing`	Determine if any array element is missing
`ismissing`	Find missing values
`rmmissing`	Remove missing entries
`fillmissing`	Fill missing values
`missing`	Create missing values
`standardizeMissing`	Insert standard missing values
`isoutlier`	Find outliers in data
`filloutliers`	Detect and replace outliers in data
`rmoutliers`	Detect and remove outliers in data
`movmad`	Moving median absolute deviation

Detecting Change Points and Local Extrema

`ischange`	Find abrupt changes in data
`islocalmin`	Find local minima
`islocalmax`	Find local maxima

Smoothing Data and Finding Trends

`smoothdata`	Smooth noisy data
`movmean`	Moving mean
`movmedian`	Moving median
`detrend`	Remove polynomial trend
`trenddecomp`	Find trends in data

Normalizing and Scaling Data

`normalize`	Normalize data
`rescale`	Scale range of array elements

Grouping and Binning Data

`discretize`	Group data into bins or categories
`groupcounts`	Number of group elements
`groupfilter`	Filter by group
`groupsummary`	Group summary computations
`grouptransform`	Transform by group
`histcounts`	Histogram bin counts
`histcounts2`	Bivariate histogram bin counts
`findgroups`	Find groups and return group numbers
`splitapply`	Split data into groups and apply function
`rowfun`	Apply function to table or timetable rows
`varfun`	Apply function to table or timetable variables
`accumarray`	Accumulate vector elements

Rubriques

Clean Messy and Missing Data in Tables
This example shows how to find, clean, and delete table rows with missing data.
Detrending Data
Remove linear trends from data.
Grouping Variables To Split Data
You can use grouping variables to categorize data variables.
Split Data into Groups and Calculate Statistics
This example shows how to group data and apply statistics functions to each group.
Split Table Data Variables and Apply Functions
This example shows how to group data variables and apply functions to each group.

Exemples présentés

Clean Messy Data and Locate Extrema Using Live Editor Tasks

Interactively preprocess data with Live Editor Tasks.

Ouvrir le live script

Missing Data in MATLAB

Handle missing values in data sets.

Ouvrir le live script

Data Smoothing and Outlier Detection

Eliminate unwanted noise or behavior in data, and find, fill, and remove outliers.

Ouvrir le live script