Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove copy of state into mpi window #735

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

remove copy of state into mpi window #735

wants to merge 6 commits into from

Conversation

hkershaw-brown
Copy link
Member

Description:

Puts the %copies array into the mpi window for get_state (forward operator)
Removes the copy of the %copies(1:real_ens_members, :) to the window.
This reduces the per-code memory usage (removes a %copies(1:real_ens_members, 1:my_num_vars)
~1/2 for the state per memory core usage.

The offset is calculated from %num_copies, the get_state is only the real ensemble members

ink

I think this is a good time to remove the cray window module.

Fixes issue

fixes #718

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Documentation changes needed?

  • My change requires a change to the documentation.
    • I have updated the documentation accordingly.

Tests

There is a developer test included. Note this needs fortran-testanything code to run.
Bitwase lorenz_96 mpi, mpif08, nompi

Checklist for merging

  • Updated changelog entry
  • Documentation updated
  • Update conf.py

Checklist for release

  • Merge into main
  • Create release from the main branch with appropriate tag
  • Delete feature-branch

Testing Datasets

  • Dataset needed for testing available upon request
  • Dataset download instructions included
  • No dataset needed

using external fortran-testanything
model_mod.f90 is in the work directory. This is a lazy move to avoid fiddling with EXTRA in quickbuild.sh

for issue #718
mpi version with all of %copies in the window
mpif08 version with all of %copies in the window
null_mpi_utilities_mod.f90 version with matchin arguments to the mpi version (does nothing)

fixes #718
hkershaw-brown and others added 2 commits September 11, 2024 10:00
The cray window mod was originally added because not all compilers/
mpi implemenations (mpi 2.0) supported mpi windows without using
cray pointers. The cray_win_mod has not been compiled into dart for
several years, and support for mpi windows has improved (mpi 3.0)
@hkershaw-brown
Copy link
Member Author

@hkershaw-brown check PMO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: Reduce memory usage for forward operators MPI window
1 participant