Acoustic segmentation by using neural networks

The project task presents the process of acoustic segmentation by using neural networks in case of musical instruments. We achieved 95% accuracy by pre-processing and preparing data for network training. We wrote the program in Python and ran it in Keras framework. In the project, we were interested in recognizing different types of sounds in a video, or in the corresponding audio recording. Recognizing specific sounds is complex, so we limited ourselves to sound recordings without the presence of noise. The recordings were made by recording various instruments that a network must recognize. The project focused on the design, fabrication and analysis of a neural network that served to perform acoustic segmentation. Namely, the acoustic segmentation system must detect the sound segments in the audio recordings so that each captures only a specific sound.  

 

In the project, we first used a program for pre-processing of the acoustic signal, which removes the silence in the recordings, because this further helps to train the network faster. When preprocessing, it is also necessary to know what data we will use. We used several different procedures to prepare the data. If we look at the signal in the time domain, the signal does not visually convey anything special, but this changes if the audio recording is displayed in different form, therefore, we used the Fourier transform, or fast-fourier transform (FFT). A spectrogram is a visual representation of the frequency spectrum of an audio signal, as a function of time. The spectrogram can be generated using an optical spectrometer, a series of band-pass filters, or using a Fourier transform. The spectrogram is important in our project for the presentation of the images we used in training the CNN neural network.