Development roadmap

Dependency

Echopype depends on many libraries in the scientific Python ecosystem, and therefore need to keep up with their updates. The two big ticket items that we hope to resolve soon are:

• Upgrade to use Numpy version 2

• Upgrade to use Zarr version 3

See the requirements.txt file for the current pinned versions. We aim to remove the specification of maximum version (e.g., zarr<3) whenever possible.

Data conversion and standardization

Echopype currently support converting files from a few echosounder models to netCDF or Zarr files following a modified version of the ICES SONAR-netCDF4 convention. As the core data representation stabilizes, the next steps are to:

• Enhance adherence to community conventions of metadata and processed data, such as:

  • ICES AcMeta

  • The new Gridded group introduced in SONAR-netCDF4 v2.0

  • Australia IMOS SOOP-BA conventions

• Add support for data from other echosounder models, including:

  • Simrad EK/BI500 (unfinished PR)

  • Biosonics DT4 files

Rule-based algorithms

We plan to add more common rule-based (i.e. non-ML) echosounder data analysis algorithms into Echopype. The high priority items are:

• Full support for broadband processing (in the calibrate subpackage)

  • currently calibrate.compute_Sv supports generating band-averaged Sv for broadband data

• Noise removal (in the clean subpackage)

  • currently clean contains a handful of noise removal functions from Ryan et al. [RDKK15].

  • there are many others that can be useful, including a more efficient algorithm for detecting transient noise

• Regridding (in the commongrid subpackage)

  • currently commongrid contains functions produce MVBS and NASC

  • need a function to regrid Sv with integrated output preserved

  • need a function to regrid a mask to a different grid

• Bottom detection (in the mask subpackage)

• Swarm or school detection (in the mask subpackage)

• Single target detection (in the mask subpackage)

• Calibration and other utility functions

  • standard target calibration

  • updated estimates of sound speed and absorption coefficients

  • water column profile-based (rather than water column average-based) Sv and TS computation

Note

Echopype is designed to be used as a programmatic API and not for manual editing. For interactive visualization, check out Echoshader.

Data processing levels

Parallel to Echopype, the Echostack team is also working on defining a set of “data processing levels” for echosounder data: see the proposal in Echolevels. Clearly defined data processing levels are conducive to broader data usage. Many Echopype functions generate prototype data provenance and processing level information as data attributes. However, both the proposed data processing levels and the implementation require revision. Please chime in with any input or questions via GitHub issues!

Echostack: Companion developments

Echopype focuses on data standardization, aggregations, and processing for building efficient and scalable data workflow. To address other needs in integrative analysis of echosounder data, check out the following companion libraries in the Echostack:

• Echoregions: Interface with echogram interpretation masks from physics-based or data-driven methods

• Echoshader: Interactive visualization widgets leveraging the HoloViz suite of tools

• Echopop: Incorporate trawl biological data and scattering models for biomass estimation, currently focused on Pacific hake

• Echodataflow: Orchestrate workflow on the cloud or local platforms

• Echolevels: Proposed specifications of echosounder data processing levels