Code Generation for Deep Learning on ARM Targets

This example uses:

This example shows how to generate and deploy code for prediction on an ARM®-based device without using a hardware support package.

When you generate code for prediction using the ARM Compute Library and a hardware support package, codegen generates code on the host computer, copies the generated files to the target hardware, and builds the executable on the target hardware. Without a hardware support package, codegen generates code on the host computer. You must run commands to copy the files and build the executable program on the target hardware.

This example uses the packNGo function to package all relevant files into a compressed zip file. Use this example to learn how to deploy the generated code on ARM Neon targets that do not have a hardware support package by using packNGo.

Prerequisites

ARM processor that supports the NEON extension
ARM Compute Library (on the target ARM hardware)
Open Source Computer Vision Library(Open CV)
Environment variables for the compilers and libraries
MATLAB® Coder™
The support package MATLAB Coder Interface for Deep Learning
Deep Learning Toolbox™

The ARM Compute library version that this example uses might not be the latest version that code generation supports. For supported versions of libraries and for information about setting up environment variables, see Prerequisites for Deep Learning with MATLAB Coder (MATLAB Coder).

This example is not supported for MATLAB Online.

squeezenet_predict Function

This example uses the DAG network SqueezeNet to show image classification with the ARM Compute Library. A pretrained SqueezeNet for MATLAB is available in the Deep Learning Toolbox. The squeezenet_predict function loads the SqueezeNet network into a persistent network object. On subsequent calls to the function, the persistent object is reused.

type squeezenet_predict

% Copyright 2018 The MathWorks, Inc.

function out = squeezenet_predict(in) 
%#codegen

% A persistent object mynet is used to load the DAG network object.
% At the first call to this function, the persistent object is constructed and
% set up. When the function is called subsequent times, the same object is reused 
% to call predict on inputs, avoiding reconstructing and reloading the
% network object.

persistent mynet;
if isempty(mynet)
       mynet = coder.loadDeepLearningNetwork('squeezenet','squeezenet');
end

out = mynet.predict(in);

Set Up a Code Generation Configuration Object for a Static Library

When you generate code targeting an ARM-based device and do not use a hardware support package, create a configuration object for a library. Do not create a configuration object for an executable program.

Set up the configuration object for generation of C++ code and generation of code only.

cfg = coder.config('lib');
cfg.TargetLang = 'C++';
cfg.GenCodeOnly = true;

Set Up a Configuration Object for Deep Learning Code Generation

Create a coder.ARMNEONConfig object. Specify the library version and the architecture of the target ARM processor. For example, suppose that the target board is a HiKey/Rock960 board with ARMv8 architecture and ARM Compute Library version 20.02.1.

dlcfg = coder.DeepLearningConfig('arm-compute');
dlcfg.ArmComputeVersion = '20.02.1';
dlcfg.ArmArchitecture = 'armv8';

Attach the Deep Learning Configuration Object to the Code Generation Configuration Object

Set the DeepLearningConfig property of the code generation configuration object to the deep learning configuration object.

cfg.DeepLearningConfig = dlcfg;

Generate Source C++ Code by Using `codegen`

codegen -config cfg squeezenet_predict -args {ones(227, 227, 3, 'single')} -d arm_compute

The code is generated in the arm_compute folder in the current working folder on the host computer.

Generate the Zip File using `packNGo` function

The packNGo function packages all relevant files in a compressed zip file.

zipFileName = 'arm_compute.zip';
bInfo = load(fullfile('arm_compute','buildInfo.mat'));
packNGo(bInfo.buildInfo, {'fileName', zipFileName,'minimalHeaders', false, 'ignoreFileMissing',true});

The code is generated as zip file.

Copy the Generated Zip file to the Target Hardware

Copy the Zip file and extract into folder and remove the Zip file in the hardware

In the following commands, replace:

password with your password
username with your user name
targetname with the name of your device
targetloc with the destination folder for the files

Perform the steps below to copy and extract zip file from Linux.

if isunix, system(['sshpass -p password scp -r '  fullfile(pwd,zipFileName) ' username@targetname:targetloc/']), end
if isunix, system('sshpass -p password ssh username@targetname "if [ -d targetloc/arm_compute ]; then rm -rf targetloc/arm_compute; fi"'), end
if isunix, system(['sshpass -p password ssh username@targetname "unzip targetloc/' zipFileName ' -d targetloc/arm_compute"']), end
if isunix, system(['sshpass -p password ssh username@targetname "rm -rf  targetloc' zipFileName '"']), end

Perform the steps below to copy and extract zip file from Windows.

if ispc, system(['pscp.exe -pw password -r '  fullfile(pwd,zipFileName) ' username@targetname:targetloc/']), end
if ispc, system('plink.exe -l username -pw password targetname "if [ -d targetloc/arm_compute ]; then rm -rf targetloc/arm_compute; fi"'), end
if ispc, system(['plink.exe -l username -pw password targetname "unzip targetloc/' zipFileName ' -d targetloc/arm_compute"']), end
if ispc, system(['plink.exe -l username -pw password targetname "rm -rf  targetloc' zipFileName '"']), end

Copy Example Files to the Target Hardware

Copy these supporting files from the host computer to the target hardware:

Input image, coffeemug.png
Makefile for generating the library, squeezenet_predict_rtw.mk
Makefile for building the executable program, makefile_squeezenet_arm_generic.mk
Synset dictionary, synsetWords.txt

In the following commands, replace:

password with your password
username with your user name
targetname with the name of your device
targetloc with the destination folder for the files

Perform the steps below to copy all the required files when running from Linux

if isunix, system('sshpass -p password scp squeezenet_predict_rtw.mk username@targetname:targetloc/arm_compute/'), end
if isunix, system('sshpass -p password scp coffeemug.png username@targetname:targetloc/arm_compute/'), end
if isunix, system('sshpass -p password scp makefile_squeezenet_arm_generic.mk username@targetname:targetloc/arm_compute/'), end
if isunix, system('sshpass -p password scp synsetWords.txt username@targetname:targetloc/arm_compute/'), end

Perform the steps below to copy all the required files when running from Windows

if ispc, system('pscp.exe -pw password squeezenet_predict_rtw.mk username@targetname:targetloc/arm_compute/'), end
if ispc, system('pscp.exe -pw password coffeemug.png username@targetname:targetloc/arm_compute/'), end
if ispc, system('pscp.exe -pw password makefile_squeezenet_arm_generic.mk username@targetname:targetloc/arm_compute/'), end
if ispc, system('pscp.exe -pw password synsetWords.txt username@targetname:targetloc/arm_compute/'), end

Build the Library on the Target Hardware

To build the library on the target hardware, execute the generated makefile on the ARM hardware.

Make sure that you set the environment variables ARM_COMPUTELIB and LD_LIBRARY_PATH on the target hardware. See Prerequisites for Deep Learning with MATLAB Coder (MATLAB Coder). ARM_ARCH variable is used in Makefile to pass compiler flags based on Arm Architecture. ARM_VER variable is used in Makefile to compile the code based on Arm Compute Version. Replace the hardware credentials and paths in similar to above steps.

Perform the below steps to build the library from Linux.

if isunix, system('sshpass -p password scp main_squeezenet_arm_generic.cpp username@targetname:targetloc/arm_compute/'), end
if isunix, system(['sshpass -p password ssh username@targetname "make -C targetloc/arm_compute/ -f squeezenet_predict_rtw.mk ARM_ARCH=' dlcfg.ArmArchitecture ' ARM_VER=' dlcfg.ArmComputeVersion ' "']), end

Perform the below steps to build the library from windows.

if ispc, system('pscp.exe -pw password main_squeezenet_arm_generic.cpp username@targetname:targetloc/arm_compute/'), end
if ispc, system(['plink.exe -l username -pw password targetname "make -C targetloc/arm_compute/ -f squeezenet_predict_rtw.mk ARM_ARCH=' dlcfg.ArmArchitecture ' ARM_VER=' dlcfg.ArmComputeVersion ' "']), end

Create Executable from the Library on the Target Hardware

Build the library with the source main wrapper file to create the executable. main_squeezenet_arm_generic.cpp is the C++ main wrapper file which invokes squeezenet_predict function to create the executable.

Run the below command to create the executable from Linux.

if isunix, system('sshpass -p password ssh username@targetname "make -C targetloc/arm_compute/ -f makefile_squeezenet_arm_generic.mk targetDirName=targetloc/arm_compute"'), end

Run the below command to create the executable from Windows.

if ispc, system('plink.exe -l username -pw password targetname "make -C targetloc/arm_compute/ -f makefile_squeezenet_arm_generic.mk targetDirName=targetloc/arm_compute"'), end

Run the Executable on the Target Hardware

Run the executable from Linux using below command.

if isunix, system('sshpass -p password ssh username@targetname "cd targetloc/arm_compute/; ./squeezenet coffeemug.png"'), end

Run the executable from Windows using below command.

if ispc, system('plink.exe -l username -pw password targetname "cd targetloc/arm_compute/; ./squeezenet coffeemug.png"'), end

Top 5 Predictions:
-----------------------------
88.299% coffee mug
7.309% cup
1.098% candle
0.634% paper towel
0.591% water jug