Batch analysis

class soundscape_IR.soundscape_viewer.batch_processing.batch_processing(folder=[], folder_id=[], file_extension='.wav', annotation_extension=None)[source]

Batch processing a large amount of acoustic data.

This class integrates a set of methods supported by soundscape_IR to automatically process a large amount of soundscape recordings. At first, copy all the WAVE files collected from one recorder (presumably made at the same location) to a folder. Then, define analysis procedures and associated parameters. Finally, run the program to initiate analysis procedures.

There are three types of outputs:

Spectrogram detection results are saved in text files (using the format of Raven software).
Feature extraction results are saved in mat files.
Basis functions learned in source separation procedures are saved in mat files.

Parameters

folderstr

Folder path for analysis.

folder_idstr

The folder ID of Google Drive folder for analysis.

See https://ploi.io/documentation/database/where-do-i-get-google-drive-folder-id for the detial of folder ID.

file_extension{‘.wav’, ‘.WAV’, ‘.mat’, ‘.txt’}, default = ‘.wav’

Data format for analysis.

annotation_extensionstr, default = None

Extended file name of the associated txt files contain annotations (generated by using Raven software).

For example, if our recording files are named using the structure of Location_Date_Time.wav and txt files are Location_Date_Time.Table.1.selections.txt, please set annotation_extension='.Table.1.selections.txt'

Examples

Combine manual annotations of multiple recordings to generate a concatenated spectrogram for model training.

>>> from soundscape_IR.soundscape_viewer import batch_processing
>>> batch = batch_processing(folder='./data/', annotation_extension='.Table.1.selections.txt')
>>> batch.params_spectrogram(FFT_size=1024, prewhiten_percent=50, time_resolution=0.1, padding=0.5, folder_combine=True)
>>> batch.run()
>>>
>>> from soundscape_IR.soundscape_viewer import matrix_operation
>>> matrix_operation().plot_lts(batch.spectrogram, batch.f, lts=False)

Apply an energy detector to identify regions of interest.

>>> from soundscape_IR.soundscape_viewer import batch_processing
>>> batch = batch_processing(folder='./data/')
>>> batch.params_spectrogram(FFT_size=1024, environment='wat', prewhiten_percent=50, time_resolution=0.1)
>>> batch.params_spectrogram_detection(threshold=6, smooth=1, minimum_interval=0.5, path='./data/txt')
>>> batch.run()

Run adaptive and semi-supervised source separation.

>>> from soundscape_IR.soundscape_viewer import source_separation
>>> model=source_separation(filename='model.mat')
>>>
>>> from soundscape_IR.soundscape_viewer import batch_processing
>>> batch = batch_processing(folder='./data/)
>>> batch.params_spectrogram(FFT_size=512, time_resolution=0.1, window_overlap=0.5, prewhiten_percent=25, f_range=[0,8000])
>>> batch.params_separation(model, adaptive_alpha=[0,0.2], additional_basis = 2, save_basis=True, path='./data/mat/')
>>> batch.run()

Methods

`params_adaptive_prewhiten`([eps, smooth, ...])	Define parameters for adaptive prewhitening.
`params_feature_extraction`([source, ...])	Define feature extraction parameters.
`params_load_result`([data_type, initial, ...])	Define parameters for loading analysis results.
`params_lts_maker`([source, time_resolution, ...])	Define parameters for making a long-term spectrogram.
`params_separation`(model[, iter, ...])	Define parameters for the prediction phase of source separation.
`params_spectrogram`([channel, offset_read, ...])	Define spectrogram parameters.
`params_spectrogram_detection`([source, ...])	Define parameters for spectrogram-based sound detection.
`params_tonal_detection`([source, ...])	Define parameters for tonal sound extraction.
`run`([start, num_file])	Run batch processing procedures.

params_adaptive_prewhiten(eps=0.1, smooth=1, continuous_adaptive=True)[source]

Define parameters for adaptive prewhitening.

Apply this method to prewhien spectrograms via time weighted running average of noise spectrum. See Lin et al. (2013) for details.

Parameters

epsfloat, default = 0.1

Ratio to update the weighted running average of power spectral density in each time bin (i).

Running average of noise is estimated independently in each frequency bin (f).

\[N = (1-eps)*P_{f,i} + eps*P_{f,i+1}\]

smoothfloat ≥ 0, default = 1

Standard deviation of Gaussian kernel for smoothing the spectrogram data.

See sigma in scipy.ndimage.gaussian_filter for details.

continuous_adaptiveboolean, default = True

Set to True when audio recordings are collected in time series.

Noise estimated from the i_th file will be passed to i+1_th file.

References

1: Lin, T.-H., Chou, L.-S., Akamatsu, T., Chan, H.-C., & Chen, C.-F. (2013) An automatic detection algorithm for extracting the representative frequency of cetacean tonal sounds. Journal of the Acoustical Society of America, 134: 2477-2485. https://doi.org/10.1121/1.4816572

params_feature_extraction(source=0, energy_percentile=None, interval_range=[1, 500], lts_maker=False, folder_id=[], path='./')[source]

Define feature extraction parameters.

See the method feature_extraction of soundscape_IR.soundscape_viewer.utility.spectrogram_detection for details.

Define the folder_id or path to save feature extraction results as mat files.

params_load_result(data_type='basis', initial=[], dateformat='yyyymmdd_HHMMSS', year_initial=0)[source]

Define parameters for loading analysis results.

Currently, only support mat files containing basis funcitons and feature extraction results, and text files containing detection results. Please choose data_type from {‘basis’, ‘feature’, ‘detection’}.

See soundscape_IR.soundscape_viewer.soundscape_viewer.lts_maker for details.

Parameters

data_type{‘basis’, ‘feature’, ‘detection’}, default = ‘basis’

Examples

Use batch_processing to combine the entire set of newly learned basis functions.

>>> from soundscape_IR.soundscape_viewer import batch_processing
>>> batch = batch_processing(folder='./data/mat/', file_extension='.mat')
>>> batch.params_load_result(data_type='basis', initial='Location_', dateformat='yyyymmdd_HHMMSS')
>>> batch.run()
>>>
>>> # Plot the basis functions learned by using semi-supervised source separation
>>> batch.model.plot_nmf(plot_type='W', source=3)

Attributes

model: See soundscape_IR.soundscape_viewer.source_separation for details.
detection_result: See soundscape_IR.soundscape_viewer.spectrogram_detection for details.
PI, PI_result, f, spectral_result: See soundscape_IR.soundscape_viewer.spectrogram_detection.feature_extraction for details.

params_lts_maker(source=1, time_resolution=[], dateformat='yyyymmdd_HHMMSS', initial=[], year_initial=0, filename='Separation_LTS.mat', folder_id=[], path='./')[source]

Define parameters for making a long-term spectrogram.

See soundscape_IR.soundscape_viewer.soundscape_viewer.lts_maker for details.

Enter the filename information to extract the recording date and time from each audio file. For example, if the file name is ‘KT08_20171118_123000.wav’, please enter initial='KT08_' and dateformat='yyyymmdd_HHMMSS'.

If the file name does not contain information of century (when dateformat='yymmdd_HHMMSS'), specify century in year_initial. For example, the recording ‘KT08_171118_123000.wav’ was made in 2017, then set year_initial=2000.

Define the folder_id or path to save the long-term spectrogram as a mat file.

params_separation(model, iter=50, adaptive_alpha=0, additional_basis=0, save_basis=False, folder_id=[], path='./')[source]

Define parameters for the prediction phase of source separation.

See soundscape_IR.soundscape_viewer.source_separation.source_separation for details.

Set save_basis to True for saving basis functions learned in the prediction phase (when adaptive or semi-supervised source separation are enabled). Define the folder_id or path to save basis functions as mat files.

params_spectrogram(channel=1, offset_read=0, FFT_size=512, environment='wat', time_resolution=None, window_overlap=0, f_range=[], prewhiten_percent=0, padding=0, folder_combine=False, mel_comp=0, sensitivity=0)[source]

Define spectrogram parameters.

See soundscape_IR.soundscape_viewer.utility.audio_visualization for details.

params_spectrogram_detection(source=0, threshold=6, smooth=0, minimum_interval=0, minimum_duration=None, maximum_duration=None, pad_size=0, folder_id=[], path='./', show_result=False)[source]

Define parameters for spectrogram-based sound detection.

See soundscape_IR.soundscape_viewer.utility.spectrogram_detection for details.

Define the folder_id or path to save detection results as txt files. Set show_result to True for directly viewing the detection results (WARNING: this may generate a large number of figures).

params_tonal_detection(source=0, tonal_threshold=0.5, smooth=1.5, threshold=3, folder_id=[], path='./')[source]

Define parameters for tonal sound extraction.

See soundscape_IR.soundscape_viewer.utility.tonal_detection for details.

Define the folder_id or path to save detection results as txt files.

run(start=1, num_file=None)[source]

Run batch processing procedures.

Parameters

startint ≥ 1, default = 1: The file number to begin batch analysis.
num_fileNone or int ≥ 1, default = None: The number of files after start for batch analysis.