Main Content

Getting Started with Anomaly Detection Using Deep Learning

Anomaly detection using deep learning is an increasingly popular approach to automating visual inspection tasks. The goal of anomaly detection is to perform a binary classification of images as normal or anomalous. For example, in the manufacturing industry, anomaly detection can determine which manufactured units meet quality standards and which units are defective.

You can train an anomaly detector using semi-supervised training. A complete semi-supervised workflow consists of training a model on normal image data and determining an anomaly threshold that separates normal images from anomalous images. Anomaly detection techniques generally follow these steps:

  1. Prepare training data and calibration data.

  2. Train a model to produce anomaly scores.

  3. Calibrate the model by determining an anomaly score threshold, and evaluate the performance of the model by classifying metrics and visualizing the classification results.

  4. Classify test images using the trained model and the optimal anomaly threshold.

  5. Deploy the model onto a production line.


An alternative to anomaly detection is anomaly classification. Classification can differentiate between normal images and multiple known anomalous patterns. Classification can provide more insight on the type of anomaly, but it requires a greater amount of training data of the anomaly classes. For an example, see Classify Defects on Wafer Maps Using Deep Learning.

The Computer Vision Toolbox™ Automated Visual Inspection Library offers functions that enable you to train, calibrate, and evaluate anomaly detection networks. You can install the Computer Vision Toolbox Automated Visual Inspection Library from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons. The functionality also requires Deep Learning Toolbox™.

Prepare Training and Calibration Data

You can train an anomaly detector using one-class learning, which means that the network trains on data consisting only of normal images. Despite training on samples only of normal images, the model learns how to distinguish between normal and anomalous images. One-class learning offers many advantages for anomaly detection problems:

  • Representations of anomalies can be scarce.

  • Anomalies can represent expensive or catastrophic outcomes.

  • There can be many types of anomalies, and the types of anomalies can change over the lifetime of the model. Describing what a normal image looks like is often more feasible than providing data that represents all possible anomalies in real world settings.

You can use the Image Labeler app to interactively label images with scene labels. The app returns the ground truth labeled data as a groundTruth object. Convert the ground truth scene labels to an image datastore for training by using the sceneLabelTrainingData function. For more information, see Get Started with the Image Labeler.

In semi-supervised learning, you can tune the performance of the trained model using calibration data. The calibration data set consists of labeled samples of normal and anomalous images. Avoid overfitting the model by using different images in the training and calibration data sets. You can use functions such as partition to split a datastore into separate datastores for training and calibration.

Some techniques, such as Outlier Exposure, add a small number of anomalous images to the training data set.

Train the Model

The Computer Vision Toolbox Automated Visual Inspection Library provides a framework for training a fully convolutional data description (FCDD) anomaly detection network. Start by creating an untrained network using the fcddAnomalyDetector object. Then, train the network by passing the network and the training data to the trainFCDDAnomalyDetector function.

If you want to train an anomaly detection network that uses a different framework, then design a model that accepts images as input and returns scalar valued anomaly scores. The score represents the likelihood that an image is anomalous. Larger scores indicate a higher probability that an image in anomalous samples. Optionally, you can design a network that returns localized anomaly scores in addition to a scalar aggregate score for the image. Localized score information enables you to subjectively understand the network classification decisions.

Calibrate and Evaluate the Model

Calibrate the model by determining an anomaly score threshold that separates the scores of good samples from the scores of anomaly samples. Unfortunately, most data sets cannot be cleanly separated into these two categories, and will have misclassifications in the form of false positives (normal images classified as anomalous) and false negatives (anomalous images classified as normal). By adjusting the threshold, you can tune the false positive and false negative rates to satisfy your operating requirements.

To select a threshold, you can use the anomalyThreshold function. The function supports different optimization methods based on statistical techniques such as receiver operating characteristic (ROC) and precision-recall (PR) curves. The function optionally returns the performance metrics as an rocmetrics (Deep Learning Toolbox) function, which you can use to plot the ROC and PR curves.

Use the evaluateAnomalyDetection function to evaluate the quality of the anomaly detection results using metrics such as the confusion matrix and average precision. The anomalyDetectionMetrics object stores the metrics.

You can visualize the localized anomaly scores to understand the classification results. For example, you can display an anomaly score map as a heatmap overlayed on a calibration or test image using the anomalyMapOverlay function. You can display an interactive figure that displays the classification results and score maps for an entire data set using the viewAnomalyDetectionResults function. Visualizing localized anomaly scores is useful for identifying patterns in false negatives and false positives. The patterns can indicate strategies to further improve the classifier performance.

Perform Classification Using the Model

Classify test images as normal or anomalous by using the classify function of the fcddAnomalyDetector object. You can get the unnormalized anomaly scores for the test images using the predict function.

Deploy the Model

The end goal of visual inspection is often to deploy trained models for use in making business decisions in real-time production systems on assembly lines. The model determines acceptance or rejection given an image. The MATLAB® Coder™ and GPU Coder™ products are effective tools for deploying visual inspection systems to CPU and NVIDIA® GPU targets.

Related Examples

More About