Data processing¶
Attention
Starting with version 0.5.0, the data processing interface Process
is deprecated. Attempts to use Process
will still
work at the moment but will no longer be available in the future.
See version 0.4.1 documentation
if you’re working with that echopype release.
Functionality¶
EK60 and AZFP narrowband echosounders:
- Calibration and echo-integration to obtain volume backscattering strength (Sv) from power data.
- Simple noise removal by removing data points (set to
NaN
) below an adaptively estimated noise floor [1]. - Binning and averaging to obtain mean volume backscattering strength (MVBS) from the calibrated data.
- Compute mean volume backscattering strength (MVBS) based
on either the number of pings and sample intervals
(the
range_bin
dimension in the dataset) or a specified ping time interval and range interval in physics units (seconds and meters, respectively).
EK80 and EA640 broadband echosounders:
Calibration based on pulse compression output in the form of average over frequency (alpha).
Attention
This feature is still under development. We found inconsistencies among pulse compression outputs from EchoView, Matlab Echolab, and the echopype implementation, see #308. In addition, currently there are issues with calibrating files containing both broadband and narrowband (the “CW mode”) data, see #310.
The same noise removal and MVBS computation functionality available to the narrowband echosounders.
The steps for performing these analyses are summarized below:
Calibration:
import echopype as ep nc_path = './converted_files/file.nc' # path to a converted nc file echodata = ep.open_converted(nc_path) # create an EchoData object ds_Sv = ep.calibrate.compute_Sv(echodata) # obtain a dataset containing Sv, range, and # the calibration and environmental parameters
Reduce data by computing MVBS:
# Reduce data based on physical units ds_MVBS = ep.preprocess.compute_MVBS( ds_Sv, # calibrated Sv dataset range_meter_bin=20, # bin size to average along range in meters ping_time_bin='20S' # bin size to average along ping_time in seconds ) # Reduce data based on sample number ds_MVBS = ep.preprocess.compute_MVBS_index_binning( ds_Sv, # calibrated Sv dataset range_bin_num=30, # number of sample bins to average along the range_bin dimensionm ping_num=5 # number of pings to average )
Noise removal:
# Remove noise ds_Sv_clean = ep.preprocess.remove_noise( # obtain a denoised Sv dataset ds_Sv, # calibrated Sv dataset range_bin_num=30, # number of samples along the range_bin dimension for estimating noise ping_num=5, # number of pings for estimating noise )
The functions in the calibrate
subpackage take in an EchoData
object,
which is essentially a container for multiple xarray Dataset
instances,
and return a single xarray Dataset
containing the calibrated backscatter
quantities and the samples’ corresponding range in meters.
The input and output of all functions in the preprocess
subpackage are xarray Dataset
instances, with the input being a Dataset
containing Sv
and range
generated from calibration.
The calibrate
and preprocess
functions do not save the calculation results to disk,
but the returned xarray Dataset
can be saved using native xarray methods
such as to_netcdf
and to_zarr
.
For example, to save the Sv and MVBS results to disk:
ds_Sv.to_netcdf('file_Sv.nc')
ds_MVBS.to_netcdf('file_MVBS.nc')
Note
Echopype’s data processing functionality is being developed actively. Be sure to check back here often!
Environmental parameters¶
Environmental parameters, including temperature, salinity and pressure, are critical in biological interpretation of ocean sonar data. They influence:
- Transducer calibration, through seawater absorption. This influence is frequency-dependent, and the higher the frequency the more sensitive the calibration is to the environmental parameters.
- Sound speed, which impacts the conversion from temporal resolution (of each data sample) to spatial resolution, i.e. the sonar observation range changes with sound speed.
By default, echopype uses the following for calibration:
- EK60 and EK80: Environmental parameters saved with the raw data files. For EK60, instrument operators may enter temperature and salinity values into the Simrad EK60 software’s Environment dialog and the Simrad software will calculate sound speed and sound absorption; alternatively, sound speed may be entered directly. Only sound speed and sound absorption are saved into the raw file.
- AZFP: Salinity and pressure provided by the user, and temperature recorded at the instrument.
Seawater sound absorption and sound speed may be recalculated with echopype if
more accurate in-situ environmental parameters are available.
To update these values, pass the environmental parameters
as a dictionary while calling ep.calibrate.compute_Sv()
:
env_params = {
'temperature': 8, # temperature in degree Celsius
'salinity': 30, # salinity in PSU
'pressure': 50, # pressure in dbar
}
ds_Sv = ep.calibrate.compute_Sv(echodata, env_params=env_params)
These values will be used in calculating sound speed, sound absorption, and the thickness of each sonar sample, which is used in calculating the range. The updated values can be retrieved with:
ds_Sv['sound_absorption'] # absorption in [dB/m]
ds_Sv['sound_speed'] # sound speed in [m/s]
ds_Sv['range'] # range for each sonar sample in [m]
For EK60 and EK80 data, echopype updates the sound speed using the formula from Mackenzie (1981) [2] and seawater absorption using the formula from Ainslie and McColm (1981) [3].
For AZFP data, echopype updates the sound speed and seawater absorption using the formulae provided by the manufacturer ASL Environmental Sciences.
Calibration parameters¶
Calibration here refers to the calibration of transducers on an echosounder, which finds the mapping between the voltage signal recorded by the echosounder and the actual (physical) acoustic pressure received at the transducer. This mapping is critical in deriving biological quantities from acoustic measurements, such as estimating biomass. More detail about the calibration procedure can be found in [4].
Echopype by default uses calibration parameters stored in the converted files along with the backscatter measurements and other metadata parsed from the raw data file. However, since careful calibration is often done separately from the data collection phase of the field work, accurate calibration parameters are often supplied in the post-processing stage. Currently echopype allows users to overwrite the following calibration parameters:
- EK60 and EK80:
sa_correction
,gain_correction
, andequivalent_beam_angle
- AZFP:
EL
,DS
,TVR
,VTX
,Sv_offset
, andequivalent_beam_angle
As an example, to reset the equivalent beam angle for all frequencies,
specify cal_params
while calling the calibration functions:
import xarray as xr
equivalent_beam_angle = xr.DataArray( # set all channels at once
[-17.47, -20.77, -21.13, -20.4, -30],
dims=['frequency'],
coords=[echodata.beam.frequency]
)
cal_params = {
'equivalent_beam_angle': equivalent_beam_angle
}
ds_Sv = ep.calibrate.compute_Sv(echodata, cal_params=cal_params)
To reset the equivalent beam angle for 18 kHz only, one can do:
# set value for 18 kHz only
echodata.beam.equivalent_beam_angle.loc[dict(frequency=18000)] = 18.02
References¶
[1] | De Robertis A, Higginbottoms I. (2007) A post-processing technique to estimate the signal-to-noise ratio and remove echosounder background noise. ICES J. Mar. Sci. 64(6): 1282–1291. |
[2] | Mackenzie K. (1981) Nine‐term equation for sound speed in the oceans. J. Acoust. Soc. Am. 70(3): 806-812 |
[3] | Ainslie MA, McColm JG. (1998) A simplified formula for viscous and chemical absorption in sea water. J. Acoust. Soc. Am. 103(3): 1671-1672 |
[4] | Demer DA, Berger L, Bernasconi M, Bethke E, Boswell K, Chu D, Domokos R, et al. (2015) Calibration of acoustic instruments. ICES Cooperative Research Report No. 1. 133 pp. |