Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiler: Support for C-level MPI_Allreduce #2344

Merged
merged 5 commits into from
May 31, 2024
Merged

compiler: Support for C-level MPI_Allreduce #2344

merged 5 commits into from
May 31, 2024

Conversation

FabioLuporini
Copy link
Contributor

No description provided.

Copy link

codecov bot commented Apr 4, 2024

Codecov Report

Attention: Patch coverage is 93.80952% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 86.86%. Comparing base (602d5e6) to head (35aea7c).

Files Patch % Lines
devito/core/gpu.py 33.33% 4 Missing ⚠️
devito/mpi/reduction_scheme.py 87.50% 3 Missing ⚠️
devito/operator/operator.py 62.50% 2 Missing and 1 partial ⚠️
devito/ir/clusters/algorithms.py 95.34% 2 Missing ⚠️
devito/ir/support/basic.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2344      +/-   ##
==========================================
+ Coverage   86.81%   86.86%   +0.04%     
==========================================
  Files         233      234       +1     
  Lines       43759    43886     +127     
  Branches     8076     8102      +26     
==========================================
+ Hits        37990    38122     +132     
+ Misses       5061     5059       -2     
+ Partials      708      705       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@mloubout mloubout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overallm the reduction part looks good, but we need to think carefully about the sparse subfunc because the current changes I think just make our setup worse and harder to fix later.
Can discuss if we wanna proceed but tag an "urgent" issue on it

[dv.Inc(s, p), dv.Eq(mr.n[0], s)],
name='sum')
op.apply(**kwargs)
op = dv.Operator([dv.Eq(s, 0.0)] + eqns +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpicking: can probalby wrap (since everywhere in this file) the s=0, eqs, s+=expr, n[0]=s into an make_reduction(eqns, expr)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do in my next PR

__all__ = ['DistReduce']


class DistReduce(sympy.Function, Reconstructable):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need sympy.Function ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well it needs to be a sympy object and the most logical one is a Function in my opinion

__rkwargs__ = ('op', 'grid', 'ispace')

def __new__(cls, var, op=None, grid=None, ispace=None, **kwargs):
obj = sympy.Function.__new__(cls, var, **kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

op.__new__ ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no op is a devito internal type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(yes, we should start adding types...)

mapper = {
OpInc: 'MPI_SUM',
OpMax: 'MPI_MAX',
OpMin: 'MPI_MIN',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing OpProd: MPI_PROD

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coming from ... ? we don't support reductions that aren't inc/max/min IIRC

"or iterable (e.g., list, np.ndarray)" % key)
if d in key.dimensions and not self.alias:
# From a reconstruction which leaves `dimensions` intact
return key
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future reference, this is very problematic (both if and else case) because now key's parent does not match the sparsefunction that uses it as a subfunction so can lead to very problematic runtime args/post_process so we may need to completely avoid this rebuild and always create a new subfunc using the key's data (need a '_local' initalizer

devito/builtins/utils.py Show resolved Hide resolved
@@ -453,7 +453,7 @@ def test_sum_sparse(self):

def test_min_max_sparse(self):
"""
Test that mmin/mmax work on SparseFunction
Test that mmin/mmax work on SparseFunction.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"SparseFunctions"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will fix in my next PR

devito/builtins/utils.py Show resolved Hide resolved
devito/ir/clusters/algorithms.py Show resolved Hide resolved
def _rcompile_wrapper(cls, **kwargs0):
def wrapper(expressions, **kwargs1):
return rcompile(expressions, {**kwargs0, **kwargs1})
def _rcompile_wrapper(cls, **kwargs):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kwargs0 was recently changed as well right, now back to it, ok

@FabioLuporini FabioLuporini merged commit 10963f3 into master May 31, 2024
30 of 31 checks passed
@FabioLuporini FabioLuporini deleted the allreduce branch May 31, 2024 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants