Classify Images from Mobile Device Camera Using Pretrained Network

Set Up Mobile Device

This example shows how to use deep learning to classify images acquired by your mobile device camera.

Install and set up MATLAB^® Mobile™ on your mobile device. Then, sign in to the MathWorks^® Cloud from the MATLAB Mobile Settings. For more information, see Install MATLAB Mobile on Your Device and Sign In to the Cloud.

Start MATLAB Mobile on your device.

Create Connection to Mobile Device Camera

On the Commands screen, create a mobiledev object m.

m = mobiledev

m = 
mobiledev with properties:

                   Connected: 1
            AvailableCameras: {'back' 'front'}
                     Logging: 0
            InitialTimestamp: ''

   AccelerationSensorEnabled: 0
AngularVelocitySensorEnabled: 0
       MagneticSensorEnabled: 0
    OrientationSensorEnabled: 0
       PositionSensorEnabled: 0

Supported functions

The AvailableCameras property indicates that this device has 'back' and 'front' cameras. Create a connection to the 'back' camera.

cam = camera(m,'back')

cam = 
  Camera with properties:

                    Name: 'back'
    AvailableResolutions: {'640x480'  '1280x720'}
               ZoomRange: [1 121.8750]
              Resolution: '640x480'
                    Zoom: 1
                   Flash: 'off'
               Autofocus: 'on'

The camera properties provide information about the image resolution, autofocus, and flash settings.

Load Pretrained Network and Acquire Image

From the Commands screen, load a pretrained GoogLeNet network using Deep Learning Toolbox™.

nnet = googlenet;

Acquire a single image from the camera using the snapshot function with manual shutter mode. After the camera preview opens, you can move your mobile device to capture the desired field of view. For this example, capture an image of the object you want to classify. When you are ready, press the shutter button to acquire the image.

img = snapshot(cam,'manual');

Resize the image to match the input size of the network. The input size for GoogLeNet is 224-by-224. Preview the image in MATLAB Mobile using image.

img = imresize(img,[224,224]);
image(img)

Classify and Display Acquired Image

Classify the object in the acquired image using classify from Deep Learning Toolbox.

label = classify(nnet,img)

label = 

  categorical

     coffee mug 

The object is classified as a coffee mug. Preview the image using the label as the figure title.

image(img)
title(char(label));

Write a Function to Classify an Image

You can write a function in MATLAB Mobile that performs all the previous steps to classify images.

On the Files screen, create a new script in your MATLAB Drive™ folder. Name the file camnet.m. Define the camnet function as follows and save the file.

function value = camnet(cam,nnet)
    img = snapshot(cam,'manual');
    pic = imresize(img,[224,224]);
    value = classify(nnet,pic);
    image(pic)
    title(char(value))
end

On the Commands screen, create the mobiledev object. Then create the camera object.

m = mobiledev;
cam = camera(m,'front');

Load the pretrained GoogLeNet network.

nnet = googlenet;

Call the camnet function.

label = camnet(cam,nnet)

The camera preview opens on your mobile device. Move your mobile device camera to point at the object you want to classify. Press the shutter button to capture the image. After capturing the image, you can view the figure. The figure title shows the predicted label of the object.