API reference#

API components that most users will interact with.

Attention

In echopype versions prior to 0.5.0, the API in this page focused on the convert and process subpackages. See the 0.4.1 API page if you’re using a previous release. That workflow is being deprecated.

Content

EchoData class#

EchoData is an object that handles interfacing raw converted data. It is used for calibration and other processing.

Classes#

EchoData([converted_raw_path, ...])

Echo data model class for handling raw converted data, including multiple files associated with the same data set.

Open raw and converted files#

echopype.open_raw(raw_file: PathHint, sonar_model: SonarModelsHint, xml_path: Optional[PathHint] = None, convert_params: Optional[Dict[str, str]] = None, storage_options: Optional[Dict[str, str]] = None, use_swap: bool = False, max_mb: int = 100) → Optional[echopype.echodata.echodata.EchoData]#

Create an EchoData object containing parsed data from a single raw data file.

The EchoData object can be used for adding metadata and ancillary data as well as to serialize the parsed data to zarr or netcdf.

Parameters
raw_filestr

path to raw data file

sonar_modelstr

model of the sonar instrument

  • EK60: Kongsberg Simrad EK60 echosounder

  • ES70: Kongsberg Simrad ES70 echosounder

  • EK80: Kongsberg Simrad EK80 echosounder

  • EA640: Kongsberg EA640 echosounder

  • AZFP: ASL Environmental Sciences AZFP echosounder

  • AD2CP: Nortek Signature series ADCP (tested with Signature 500 and Signature 1000)

xml_pathstr

path to XML config file used by AZFP

convert_paramsdict

parameters (metadata) that may not exist in the raw file and need to be added to the converted file

storage_optionsdict

options for cloud storage

use_swap: bool

If True, variables with a large memory footprint will be written to a temporary zarr store at ~/.echopype/temp_output/parsed2zarr_temp_files

max_mbint

The maximum data chunk size in Megabytes (MB), when offloading variables with a large memory footprint to a temporary zarr store

Returns
EchoData object
Raises
ValueError

If sonar_model is None or sonar_model given is unsupported.

FileNotFoundError

If raw_file is None.

TypeError

If raw_file input is neither str or pathlib.Path type.

Notes

use_swap=True is only available for the following echosounders: EK60, ES70, EK80, ES80, EA640. Additionally, this feature is currently in beta.

echopype.open_converted(converted_raw_path: PathHint, storage_options: Dict[str, str] = None, **kwargs)#

Create an EchoData object from a single converted netcdf or zarr file.

Parameters
converted_raw_pathstr

path to converted data file

storage_optionsdict

options for cloud storage

kwargsdict

optional keyword arguments to be passed into xr.open_dataset

Returns
EchoData object

Combine EchoData objects#

echopype.combine_echodata(echodatas: Optional[List[echopype.echodata.echodata.EchoData]] = None, zarr_path: Optional[Union[str, pathlib.Path]] = None, overwrite: bool = False, storage_options: Dict[str, Any] = {}, client: Optional[distributed.client.Client] = None, channel_selection: Optional[Union[List, Dict[str, list]]] = None, consolidated: bool = True) → echopype.echodata.echodata.EchoData#

Combines multiple EchoData objects into a single EchoData object. This is accomplished by writing each element of echodatas in parallel (using Dask) to the zarr store specified by zarr_path.

Parameters
echodataslist of EchoData object

The list of EchoData objects to be combined

zarr_path: str or pathlib.Path, optional

The full save path to the final combined zarr store

overwrite: bool

If True, will overwrite the zarr store specified by zarr_path if it already exists, otherwise an error will be returned if the file already exists.

storage_options: dict

Any additional parameters for the storage backend (ignored for local paths)

client: dask.distributed.Client, optional

An initialized Dask distributed client

channel_selection: list of str or dict, optional

Specifies what channels should be selected for an EchoData group with a channel dimension (before combination).

  • if a list is provided, then each EchoData group with a channel dimension

will only contain the channels in the provided list - if a dictionary is provided, the dictionary should have keys specifying only beam groups (e.g. “Sonar/Beam_group1”) and values as a list of channel names to select within that beam group. The rest of the EchoData groups with a channel dimension will have their selected channels chosen automatically.

consolidated: bool

Flag to consolidate zarr metadata. Defaults to True

Returns
EchoData

A lazy loaded EchoData object obtained from zarr_path, with all data from the input EchoData objects combined.

Raises
ValueError

If the provided zarr path does not point to a zarr file

TypeError

If the a provided zarr_path input is not of type string or pathlib.Path

RuntimeError

If zarr_path already exists and overwrite=False

TypeError

If a list of EchoData objects are not provided

ValueError

If any EchoData object’s sonar_model is None

ValueError

If any EchoData object does not have a file path

ValueError

If the provided EchoData objects have the same filenames

RuntimeError

If the first time value of each EchoData group is not less than the first time value of the subsequent corresponding EchoData group, with respect to the order in echodatas

RuntimeError

If the same EchoData groups in echodatas do not have the same number of channels and the same name for each of these channels.

RuntimeError

If any of the following attribute checks are not met amongst the combined EchoData groups:

  • the keys are not the same

  • the values are not identical

  • the keys date_created or conversion_time do not have the same types

RuntimeError

If any EchoData group has a channel dimension value with a duplicate value.

RuntimeError

If channel_selection=None and the channel dimensions are not the same across the same group under each object in echodatas.

NotImplementedError

If channel_selection is a list and the listed channels are not contained in the EchoData group across all objects in echodatas.

Notes

  • EchoData objects are combined by appending their groups individually to a zarr store.

  • All attributes (besides attributes whose values are arrays) from all groups before the combination will be stored in the Provenance group.

  • The instance attributes source_file and converted_raw_path of the combined EchoData object will be copied from the first EchoData object in the given list.

  • If no zarr_path is provided, the combined zarr file will be '~/.echopype/temp_output/combined_echodata.zarr'.

  • If no client is provided, then a client with a local scheduler will be used. The created scheduler and client will be shutdown once computation has finished.

  • For each run of this function, we print our the client dashboard link.

Examples

Combine lazy loaded EchoData objects:

>>> ed1 = echopype.open_converted("file1.zarr")
>>> ed2 = echopype.open_converted("file2.zarr")
>>> combined = echopype.combine_echodata(echodatas=[ed1, ed2],
>>>                                      zarr_path="path/to/combined.zarr",
>>>                                      storage_options=my_storage_options)

Combine in-memory EchoData objects:

>>> ed1 = echopype.open_raw(raw_file="EK60_file1.raw", sonar_model="EK60")
>>> ed2 = echopype.open_raw(raw_file="EK60_file2.raw", sonar_model="EK60")
>>> combined = echopype.combine_echodata(echodatas=[ed1, ed2],
>>>                                      zarr_path="path/to/combined.zarr",
>>>                                      storage_options=my_storage_options)

Data processing subpackages#

calibrate#

Functions#

compute_Sv(echodata, **kwargs)

Compute volume backscattering strength (Sv) from raw data.

compute_TS(echodata, **kwargs)

Compute target strength (TS) from raw data.

preprocess#

Functions#

estimate_noise(*args, **kwargs)

remove_noise(*args, **kwargs)

compute_MVBS(*args, **kwargs)

compute_MVBS_index_binning(*args, **kwargs)

qc#

Functions#

coerce_increasing_time(ds[, time_name, ...])

Coerce a time coordinate so that it always flows forward.

exist_reversed_time(ds, time_name)

Test for occurrence of time reversal in specified datetime coordinate variable.

mask#

Functions#

frequency_differencing(source_Sv[, ...])

Create a mask based on the differences of Sv values using a pair of frequencies.

apply_mask(source_ds, mask[, var_name, ...])

Applies the provided mask(s) to the Sv variable var_name in the provided Dataset source_ds.

Utilities#

Utilities for calculating seawater acoustic properties.

Functions#

calc_absorption(frequency[, temperature, ...])

Calculate sea water absorption in units [dB/m].

calc_sound_speed([temperature, salinity, ...])

Calculate sound speed in [m/s].

Visualization subpackage#

Visualization module to quickly plot raw, Sv, and MVBS dataset.

NOTE: To use this subpackage. `Matplotlib` and `cmocean` package must be installed.

echopype.visualize.create_echogram(data: Union[echopype.echodata.echodata.EchoData, xarray.core.dataset.Dataset], channel: Optional[Union[str, List[str]]] = None, frequency: Optional[Union[str, List[str]]] = None, get_range: Optional[bool] = None, range_kwargs: dict = {}, water_level: Optional[Union[int, float, xarray.core.dataarray.DataArray, bool]] = None, **kwargs) → List[Union[xarray.plot.facetgrid.FacetGrid, matplotlib.collections.QuadMesh]]#

Create an Echogram from an EchoData object or Sv and MVBS Dataset.

Parameters
dataEchoData or xr.Dataset

Echodata or Xarray Dataset to be plotted

channelstr or list of str, optional

The channel to be plotted. Otherwise all channels will be plotted.

frequencyint, float, or list of float or ints, optional

The frequency to be plotted. If not specified, all frequency will be plotted.

get_rangebool, optional

Flag as to whether range (echo_range) should be computed or not, by default it will just plot range_sample` as the yaxis.

Note that for data that is “Sv” xarray dataset, get_range defaults to True.

range_kwargsdict

Keyword arguments dictionary for computing range (echo_range). Keys are env_params, waveform_mode, and encode_mode.

water_levelint, float, xr.DataArray, or bool, optional

Water level data array for platform water level correction. Note that auto addition of water level can be performed when data is an EchoData object by setting this argument to True. Currently because the water level information is not available as part of the Sv dataset, a warning is issued when water_level=True in this case and no correction is performed. This behavior will change in the future when the default content of Sv dataset is updated to include this information.

**kwargs: optional

Additional keyword arguments for xarray plot pcolormesh.

Notes

The EK80 echosounder can be configured to transmit either broadband (waveform_mode="BB") or narrowband (waveform_mode="CW") signals. When transmitting in broadband mode, the returned echoes are encoded as complex samples (encode_mode="complex"). When transmitting in narrowband mode, the returned echoes can be encoded either as complex samples (encode_mode="complex") or as power/angle combinations (encode_mode="power") in a format similar to those recorded by EK60 echosounders.