Skip to content
deisa-dask v0.7.1 is here! 🎉 Discover what’s new
deisa-dask v0.6.3 : Fixing Per-Array MPI Communicators Creation Order

deisa-dask v0.6.3 : Fixing Per-Array MPI Communicators Creation Order

July 29, 2026·bmartin
bmartin

deisa-dask 0.6.3: Improved Reliability, Better MPI Support, and New Performance Benchmarks

We’re pleased to announce deisa-dask 0.6.3, a release focused on improving robustness and developer experience. While this release does not introduce major API changes, it significantly strengthens the runtime by improving callback execution, fixing several concurrency issues, expanding MPI compatibility, and introducing new benchmarking tools.

More reliable callback execution

One of the primary goals of this release was to make callback execution more predictable.

execute_callbacks() now waits for completion

In previous versions, execute_callbacks() could return before all registered callbacks had finished executing. This made it difficult to reason about application state after callback execution, especially when callbacks performed asynchronous work.

With 0.6.3, execute_callbacks() now blocks until every callback has completed. This provides a much clearer execution model: once the function returns, all callbacks associated with the current data have finished.

For users, this means fewer synchronization concerns and more deterministic behavior when integrating deisa-dask into larger workflows.

Detecting invalid timestep ordering

Callbacks now verify that incoming iterations are monotonically increasing.

If an older timestep is accidentally delivered after a newer one, deisa-dask raises an exception instead of silently accepting inconsistent data.

This additional validation helps detect application bugs early and prevents sliding windows from entering inconsistent states.

Improved internal robustness

Several internal synchronization issues have been resolved in this release.

A concurrency issue affecting the internal window management has been fixed, making callback execution more reliable under concurrent workloads. In addition, callback metadata now consistently reports the correct iteration associated with each received array, eliminating cases where iteration information could become inconsistent.

Although these improvements are largely invisible to users, they increase the reliability of long-running simulations and streaming workflows.

Better MPI compatibility

deisa-dask now continuously tests its MPI backend against both OpenMPI and MPICH in continuous integration.

Supporting multiple MPI implementations is important because behavior can differ subtly between libraries. Running the test suite against both implementations helps ensure that new features remain portable across HPC systems without requiring users to modify their applications.

Measuring callback latency

This release introduces a new benchmark that measures one of the most important performance metrics for in-situ analysis: the time between a simulation sending data and a callback beginning execution.

Rather than measuring only communication throughput, this benchmark captures the end-to-end latency experienced by applications:

Bridge.send()
        │
        ▼
 Communication
        │
        ▼
 Data reconstruction
        │
        ▼
 Callback execution begins

This provides a more representative view of the responsiveness of the complete deisa-dask pipeline and establishes a foundation for tracking latency improvements in future releases.

Additional benchmark scripts have also been added for studying client connection overhead and communication latency under different configurations.

A cleaner development workflow

This release also improves the development experience.

Code formatting and style are now standardized using Ruff, with automated formatting and linting integrated into the continuous integration pipeline. This helps keep the codebase consistent while catching common issues before they reach the main branch.

Looking ahead

Although 0.6.3 is primarily a maintenance release, it strengthens many of the core components of deisa-dask:

  • callback execution is now deterministic,
  • iteration ordering is validated automatically,
  • concurrency issues have been resolved,
  • MPI compatibility is tested across multiple implementations,
  • and new benchmarking tools make it easier to evaluate end-to-end performance.

Together, these improvements make deisa-dask a more reliable platform for developing in-situ analytics workflows while providing a stronger foundation for future performance and scalability improvements.

Full Changelog: https://github.com/deisa-project/deisa-dask/compare/v0.6.2...v0.6.3

Last updated on