Question about different processing time when using the API "Predict"

Dear MATLAB team,

I have developed an application by using the App Designer. My app includes a deep learning network. When the app calls the API "predict" to process the same input data again and again, the processing time is diferent and has a large difference. They are,

历时 0.200100 秒。

历时 0.213935 秒。

历时 0.304718 秒。

历时 0.230328 秒。

历时 0.266616 秒。

历时 0.222978 秒。

历时 0.198209 秒。

历时 0.159689 秒。

历时 0.159675 秒。

历时 0.183159 秒。

历时 0.185328 秒。

历时 0.216026 秒。

历时 0.186724 秒。

历时 1.337411 秒*****。

历时 0.206957 秒。

历时 0.207161 秒。

历时 0.189404 秒。

历时 0.219939 秒。

历时 0.351349 秒。

历时 0.493345 秒。

The maximal time is 1.337411 s.

The different processing time makes the application cannot shows predicted results at a constant speed. So, why is the processing time different? And, how can I keep the processing time the same?

Thank you!

Answer 1

[Manas Shivakumar]

Hello Wenliang zhu,

I understand that you are curious about the substantial difference in processing times when using the ‘predict’ API with the same input on different executions.

There could be multiple reasons for this behaviour. Here are some possible explanations:

• System Load: Simultaneous process or tasks can consume CPU or memory resources, leading to variations in processing time.
• Hardware limitation: Factors such as CPU speed, available memory and disk I/O speed can impact the performance of your application.
• Caching effects: MATLAB’s deep learning toolbox often utilizes caching mechanisms to optimize performance. Subsequent calls may or may not benefit from cached data.
• Data dependencies: Variations in data retrieval or availability from external data sources or dependencies can impact processing time due to network and/or disk I/O delays.

To achieve more consistent processing time, consider the following measures:

• Code Optimizations: Minimize unnecessary computations and optimize data loading and preprocessing steps.
• Hardware acceleration: if the system supports it, utilize GPU acceleration for improved performance.
• Monitor system performance: Keep track of system resource utilization during the execution of your application to identify any resource-intensive processes or bottlenecks.
• Batch processing: Consider batching multiple inputs together and processing them as a group to reduce overhead.

Additionally, you can also use MATLAB’s profiling tools to analyse and optimise your code. Here is the link to the documentation: https://in.mathworks.com/help/matlab/matlab_prog/profiling-for-improving-performance.html

By implementing these measures, you can aim for a more consistent processing time for the ‘predict’ API.

Hope this information helps!