Identify Objects Acoustically with a Neural Network

Live Script exploring how to classify objects using their acoustic signatures with a neural network.

This Live Script explores how to classify objects with a neural network based on the real audio recordings of the sounds the objects make when excited. The VGGish pre-trained artificial neural network is shown to accurately classify common objects with very different acoustic signatures and even nearly identical objects such as individual wine glasses in a set.

The script follows the example Investigate Audio Classifications Using Deep Learning Interpretability Techniques. It includes code to snip a long audio recording of irregular banging sounds into frames appropriate for network learning. It uses de-silencing with a sound selection process designed to isolate intrinsic from circumstantial acoustic signatures. It illustrates how shifting the frequency acoustic recordings (using a two-line frequency-shifting trick) to ranges distinct to the network assists in classifying them with a network which has insufficient frequency resolution.

This script may interest students and instructors of physics and related fields who are new to neural networks. It is designed to function as an easy hands-on lab. Sample audio files and instructions to make similar recordings are included. Provided 'Background Information' may help demystify sound generation and complex neural networks. 'Try this' suggestions, coding 'Challenges,' hyperlinks, and references are included for further exploration. Additional educational Live Scripts by the author are available here.