Development roadmap

Dependency

Echopype depends on many libraries in the scientific Python ecosystem and therefore needs to keep up with their evolution. Recent milestones include:

• Migration to pyproject.toml-based packaging

• Support for NumPy 2

• Support for Zarr 3

Project metadata and dependencies are managed through pyproject.toml. We aim to keep dependency constraints as permissive as possible while maintaining compatibility across supported Python versions.

Data conversion and standardization

Echopype currently support converting files from a few echosounder models to netCDF or Zarr files following a modified version of the ICES SONAR-netCDF4 convention. As the core data representation stabilizes, the next steps are to:

• Enhance adherence to community conventions of metadata and processed data, such as:

  • ICES AcMeta

  • The new Gridded group introduced in SONAR-netCDF4 v2.0

  • Australia IMOS SOOP-BA conventions

• Add support for data from other echosounder models, including:

  • Simrad EK/BI500 (unfinished PR)

  • Biosonics DT4 files

Rule-based algorithms

We plan to add more common rule-based (i.e. non-ML) echosounder data analysis algorithms into Echopype. The high priority items are:

• Expanded support for broadband processing (in the calibrate subpackage)

  • broadband Sv processing is currently supported to compute band-averaged Sv from broadband data

  • ongoing development includes TS spectrum, Sv spectrum, and related broadband analysis workflows

• Noise removal (in the clean subpackage)

  • currently clean contains a handful of noise removal functions from .

  • there are many others that can be useful, including a more efficient algorithm for detecting transient noise

• Regridding (in the commongrid subpackage)

  • currently commongrid contains functions produce MVBS and NASC

  • need a function to regrid Sv with integrated output preserved

  • need a function to regrid a mask to a different grid

• Bottom detection (in the mask subpackage)

• Swarm or school detection (in the mask subpackage)

• Single target detection (in the mask subpackage)

• Calibration and other utility functions

  • standard target calibration

  • updated estimates of sound speed and absorption coefficients

  • water column profile-based (rather than water column average-based) Sv and TS computation

Note

Echopype is designed to be used as a programmatic API and not for manual editing. For interactive visualization, check out Echoshader.

Data processing levels

Parallel to Echopype, the Echostack team is also working on defining a set of “data processing levels” for echosounder data: see the proposal in Echolevels. Clearly defined data processing levels are conducive to broader data usage. Many Echopype functions generate prototype data provenance and processing level information as data attributes. However, both the proposed data processing levels and the implementation require revision. Please chime in with any input or questions via GitHub issues!

Echostack: Companion developments

Echopype focuses on data standardization, aggregations, and processing for building efficient and scalable data workflow. To address other needs in integrative analysis of echosounder data, check out the following companion libraries in the Echostack:

• Echoregions: Interface with echogram interpretation masks from physics-based or data-driven methods

• Echoshader: Interactive visualization widgets leveraging the HoloViz suite of tools

• Echopop: Incorporate trawl biological data and scattering models for biomass estimation, currently focused on Pacific hake

• Echodataflow: Orchestrate workflow on the cloud or local platforms

• Echolevels: Proposed specifications of echosounder data processing levels