Run batch analysis
This tutorial introduces the application of the module
batch_processing in analyzing a large amount of audio recordings. This tutorial contains three sections:
Generating a concatenated spectrogram from multiple recordings
Applying model-based source separation to detect target sounds
Performing adaptive and semi-supervised source separation to assess acoustic diversity
In order to build a robust source separation model, we often need to use a large amount of annotated sounds for model training. The module
batch_processing allows to extract annotated clips from multiple recordings and subsequently generates a concatenated spectrogram.
At first, create a folder and copy a set of annotated recordings to the folder. Enter the folder path in
folder and the extended file name of the associated txt files containing annotations (generated by using Raven software) in
annotation_extension. For example, if our recording files are named using the structure of Location_Date_Time.wav and txt files are Location_Date_Time.Table.1.selections.txt, please enter
Then, define spectrogram parameters and set
batch_processing.params_spectrogram. By doing this,
batch_processing will combine all the recordings from the folder and generate a concatenated spectrogram according to the associated annotations. If only a portion of recordings in the folder is needed, we can specify the beginning file number (start from 1) in
start and the number of recordings in
batch_processing, a concatenated spectrogram is saved in
batch_processing.spectrogram, and we can plot the concatenated spectrogram by using
from soundscape_IR.soundscape_viewer import batch_processing from soundscape_IR.soundscape_viewer import matrix_operation from soundscape_IR.soundscape_viewer import source_separation # Assign the folder path contains annotated recordings path='./data/batch/sec1_2/' # Define parameters batch = batch_processing(folder=path, file_extension='.wav', annotation_extension='.Table.1.selections.txt') batch.params_spectrogram(FFT_size=1024, f_range = [0,30000], environment='wat', prewhiten_percent=20, time_resolution=0.05, padding=0.5, folder_combine=True) batch.run(start=1, num_file=2) # Plot concatenated spectrogram matrix_operation().plot_lts(batch.spectrogram, batch.f, fig_width=18, fig_height=6, lts=False) # Train a source separation model using PC-NMF model=source_separation(feature_length=10, basis_num=20) model.learn_feature(input_data=batch.spectrogram, f=batch.f, method='PCNMF') # Plot the basis functions of each source model.plot_nmf(plot_type='W', source=1) model.plot_nmf(plot_type='W', source=2)
Source separation and spectrogram detection
This section demonstrates the use of a source separation model in
batch_processing to detect target sounds from a set of unannotated recordings.
Here, we set
folder_combine=False to sequentially process recordings. In addition, we add
batch_processing.params_spectrogram_detection to import the model and to run an energy detector. In
batch_processing.params_spectrogram_detection, we can specify multiple sources (e.g.,
source=[1,2]) to separately detect regions of interests from different sources.
Detection results will be saved as txt files (with source number after the original filename), using the same annotation format prepared via Raven software. The plotting function of detection results can be deactivated by setting
from soundscape_IR.soundscape_viewer import batch_processing # Assign the folder path of unannotated recordings and the path for saving detection results (in txt format) path='./data/batch/sec1_2/' save_path = './data/txt' # Define parameters batch = batch_processing(folder=path, file_extension='.wav') batch.params_spectrogram(FFT_size=1024, environment='wat', prewhiten_percent=20, time_resolution=0.05, f_range=[0, 30000]) batch.params_separation(model) batch.params_spectrogram_detection(source=1, threshold=6, smooth=1, pad_size=0, path=save_path, minimum_interval=0.5, show_result=True) batch.run(start=3, num_file=2)
Adaptive and semi-supervised source separation
The application of source separation in
batch_processing also enables the use of adpated and semi-supervised learning methods in extracting acoustic features associated with target sounds displaying varied features and unknown sounds.
At first, load a source separation model previously trained. Then, define
batch_processing.params_separation to enable adpated and semi-supervised source separation. Note that we can assign multiple values in
adaptive_alpha for different sound sources. In this example, we allow basis functions associated with deer calls to adapt from the variations among different calling individuals.
save_basis=True to save the basis fuctions learned in source separation procedures into a mat file.
from soundscape_IR.soundscape_viewer import source_separation from soundscape_IR.soundscape_viewer import batch_processing # Load a source separation model model=source_separation() model.load_model('./data/model/KT08_linear_fl24_bn10.mat') # Assign the folder_id of unannotated recordings and the path for saving basis functions (in mat format) path='./data/batch/sec3/' save_path='./data/mat' # Define parameters batch = batch_processing(folder=path, file_extension='.wav') batch.params_spectrogram(FFT_size=512, time_resolution=0.1, window_overlap=0.5, prewhiten_percent=25, f_range=[0,8000]) batch.params_separation(model, adaptive_alpha=[0,0.2], additional_basis = 0, save_basis=True, path=save_path) batch.run() # Load the basis functions from a specific mat file load_basis=source_separation() load_basis.load_model('./data/mat/KT08_20171118_123000.mat', model_check=False) # Plot the basis functions of each source for i in range(load_basis.source_num): load_basis.plot_nmf(plot_type = 'W', source = i+1)
Aftering performing adapted or semi-supervised source separation, we can use
batch_processing to load all basis functions for visualizing acoustic diversity.
Here, we use
batch_processing.params_load_result to load a large number of mat files. Please enter the filename information of the mat files so that
batch_processing can extract the recording date and time from each mat file. For example, if the file name is ‘KT08_20171118_123000.mat’, please enter
dateformat='yyyymmdd_HHMMSS'. After the loading procedure, a source separation model contains all the basis functions is saved in
batch_processing.model, and we can use
model.plot_nmf() to plot basis functions or extract basis functions from
from soundscape_IR.soundscape_viewer import batch_processing # Assign the folder path of saved mat files mat_path='./data/mat' # Define parameters batch = batch_processing(folder=mat_path, file_extension='.mat') batch.params_load_result(data_type='basis', initial='KT08_', dateformat='yyyymmdd_HHMMSS') batch.run() # Plot the entire set of basis functions idx = batch.model.W_cluster == 1 batch.model.W = batch.model.W[:,idx][:,::7] batch.model.plot_nmf(plot_type='W')
With adapted source separation, we can obtain a set of basis functions associated with different sound types. We choose one feature from the adapted basis functions in each mat file and project these basis functions on an one-dimensional axis using Uniform Manifold Approximation and Projection (UMAP). The UMAP coordinate summarizes the spectral variations of the adapted basis functions. By sorting the adapted basis functions according to their UMAP scores, the basis functions at the left part are mostly characterized by burst-pulses, and the basis functions at the right part are tonal sounds with a few harmonics.
import umap import numpy as np # Performing UMAP analysis on the adapted basis functions model = umap.UMAP(n_neighbors=3, n_components = 1) data_umap = model.fit_transform(batch.model.W.T) # Display the spectral variations of the adapted basis functions idx = np.argsort(data_umap[:,0]) batch.model.W = batch.model.W[:,idx] batch.model.plot_nmf(plot_type='W')