Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] use installed C++ RMM in python build #588

Merged
merged 11 commits into from
Oct 16, 2020

Conversation

germasch
Copy link
Contributor

@germasch germasch commented Oct 1, 2020

This PR switches the build of the python rmm to use the installed C++ RMM.

For the most part, this is just a cleanup, since the installed header-only RMM is the same as the one in the source tree.

But combined with PR #580, it will address issues like #584, where there is no spdlog preinstalled on the system. Currently, cmake will pull in its own spdlog in this case, but that spdlog won't be available to build the python modules. #580 will install spdlog together with RMM in this case, and with this PR, the python rmm build will use it.

Also, on a side note, for all I can tell, right now the build of the python modules will not use the downloaded Thrust 1.10.0, either, but whatever Thrust will be found on the Python side (ie., probably the one from the CUDA toolkit).

For all I can tell it's not actually recognized by setuptools, nor actually
used, and in any case, there is no more actual library.
Various existing hardcoded paths were not actually used anymore,
this is generally cleaner, and importantly, if the C++ build pulled in
external dependencies that were not locally installed, this will ensure
that these are used in the python build.
@germasch germasch requested a review from a team as a code owner October 1, 2020 00:30
@GPUtester
Copy link
Contributor

Can one of the admins verify this patch?

1 similar comment
@GPUtester
Copy link
Contributor

Can one of the admins verify this patch?

@harrism
Copy link
Member

harrism commented Oct 1, 2020

ok to test

@GPUtester
Copy link
Contributor

Please update the changelog in order to start CI tests.

View the gpuCI docs here.

@harrism
Copy link
Member

harrism commented Oct 1, 2020

@germasch just informing you since you are a prolific external contributor to RMM now. We are nearing the end of the 0.16 dev cycle: Schedule here. We are entering the burndown phase tomorrow. In a nutshell, burndown means "no new PRs". If you think something should get into 0.16, please open a PR today, unless it's a significant change, in which case it should probably wait until 0.17. Tomorrow branch-0.17 will be created so you can target it with new PRs. See https://docs.rapids.ai/releases/process/

@kkraus14 kkraus14 added 2 - In Progress Currently a work in progress CMake Python Related to RMM Python API labels Oct 1, 2020
@kkraus14
Copy link
Contributor

kkraus14 commented Oct 1, 2020

For what it's worth, touching the setup.py in this way is an area that doesn't have coverage in CI so I'd prefer to be extra cautious and not have this target 0.16 right before release.

Assigning to 0.17 for this reason and will update the branch once created.

@germasch
Copy link
Contributor Author

germasch commented Oct 1, 2020

I agree that this should be checked over carefully. But I also think the changes are small enough that it is possible for an experienced python person to review this and be pretty confident that nothing unexpected will happen (the maybe most complex part, getting INSTALL_PREFIX into setup.py follows how it's already done for CUDA_HOME).

I don't really care myself (I'm only using the C++ side of RMM), but it's worth mentioning that it fixes an actual issue (#584), which I had reproduced here. I didn't put a "fixes xxx" github comment to that effect since it's only true after my other cmake PR is merged.

@germasch
Copy link
Contributor Author

germasch commented Oct 1, 2020

Or maybe I can't get it past the style checker, anyway :(

When I ran black (version 20.8b1) locally, it reformatted setup.py and rmm.py, the latter wasn't even touched by this PR. In any case, your CI black is still not happy with what my black did...

@germasch
Copy link
Contributor Author

germasch commented Oct 2, 2020

Got the formatting thing worked out. Since #580 is merged, fixes #584.

@germasch germasch changed the title [WIP] use installed C++ RMM in python build [REVIEW] use installed C++ RMM in python build Oct 2, 2020
@kkraus14 kkraus14 changed the base branch from branch-0.16 to branch-0.17 October 14, 2020 04:35
@kkraus14 kkraus14 requested review from a team as code owners October 14, 2020 04:35
Copy link
Contributor

@kkraus14 kkraus14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't done an in depth review yet but needed to block merging

python/setup.py Outdated Show resolved Hide resolved
@kkraus14 kkraus14 added 3 - Ready for review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Oct 15, 2020
@kkraus14 kkraus14 added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for review Ready for review by team labels Oct 16, 2020
@kkraus14 kkraus14 merged commit 4ec890f into rapidsai:branch-0.17 Oct 16, 2020
@germasch germasch deleted the pr/python-build branch October 16, 2020 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge CMake Python Related to RMM Python API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants