replace numpy.distutils with CMake and scikit-build-core build backend
#87
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As setuptools cannot natively call f2py, we call it ourselves by using a custom build extension.
as a attempt to fix the spin up environment error below: Error response from daemon: Head "https://registry-1.docker.io/v2/library/ubuntu/manifests/bionic": received unexpected HTTP status: 503 Service Unavailable
as `np.NaN` was removed in the NumPy 2.0 release.
|
Thank you @rtburns-jpl for the PR! The circle CI testing passed, but my local testing did not, reporting that the
conda create --name test_pysolid --yes
conda activate test_pysolid
(test_pysolid) yunjunz:~/tools>$ mamba install -c conda-forge fortran-compiler --file PySolid/requirements.txt --file PySolid/tests/requirements.txt
(test_pysolid) yunjunz:~/tools>$ python -m pip install -e PySolid
Obtaining file:///Users/yunjunz/tools/PySolid
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in ./mambaforge/envs/test_pysolid/lib/python3.12/site-packages (from pysolid==0.3.2.post7) (2.0.0)
Requirement already satisfied: scipy in ./mambaforge/envs/test_pysolid/lib/python3.12/site-packages (from pysolid==0.3.2.post7) (1.13.1)
Building wheels for collected packages: pysolid
Building editable for pysolid (pyproject.toml) ... done
Created wheel for pysolid: filename=pysolid-0.3.2.post7-0.editable-cp312-cp312-macosx_10_13_x86_64.whl size=17136 sha256=6527d51b6adcdcdea448a3c076fc28b75ffb09e329ab55fb992631adeaeb05b7
Stored in directory: /private/var/folders/16/9yhnnw8d10b2dq1dr09bx9r00000gn/T/pip-ephem-wheel-cache-launa6oe/wheels/59/5f/40/845da050940776ea855c4a700d9a4b894d994a1494b07043cb
Successfully built pysolid
Installing collected packages: pysolid
Successfully installed pysolid-0.3.2.post7
(test_pysolid) yunjunz:~/tools>$ python PySolid/tests/grid.py
--------------------------------------------------
/Users/yunjunz/tools/PySolid/tests/grid.py
Traceback (most recent call last):
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/pysolid/grid.py", line 45, in calc_solid_earth_tides_grid
from pysolid.solid import solid_grid
ModuleNotFoundError: No module named 'pysolid.solid'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/yunjunz/tools/PySolid/tests/grid.py", line 33, in <module>
tide_u) = pysolid.calc_solid_earth_tides_grid(dt_obj, atr, verbose=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/pysolid/grid.py", line 50, in calc_solid_earth_tides_grid
raise ImportError(msg)
ImportError: Cannot import name 'solid' from 'pysolid'!
Maybe solid.for is NOT compiled yet.
Check instruction at: https://github.com/insarlab/PySolid.
(test_pysolid) yunjunz:~/tools>$ python PySolid/tests/point.py
--------------------------------------------------
/Users/yunjunz/tools/PySolid/tests/point.py
PYSOLID: calculate solid Earth tides in east/north/up direction
PYSOLID: lot/lon: 34.0/-118.0 degree
PYSOLID: start UTC: 2020-11-01T04:00:00
PYSOLID: end UTC: 2020-12-31T02:00:00
PYSOLID: time step: 60 seconds
Traceback (most recent call last):
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/pysolid/point.py", line 163, in calc_solid_earth_tides_point_per_day
from pysolid.solid import solid_point
ModuleNotFoundError: No module named 'pysolid.solid'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/yunjunz/tools/PySolid/tests/point.py", line 28, in <module>
tide_u) = pysolid.calc_solid_earth_tides_point(lat, lon, dt_obj0, dt_obj1, verbose=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/pysolid/point.py", line 115, in calc_solid_earth_tides_point
tide_ui) = calc_solid_earth_tides_point_per_day(lat, lon,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/pysolid/point.py", line 168, in calc_solid_earth_tides_point_per_day
raise ImportError(msg)
ImportError: Cannot import name 'solid' from 'pysolid'!
Maybe solid.for is NOT compiled yet.
Check instruction at: https://github.com/insarlab/PySolid. |
|
Ok that took a while to sort out but I think I've got it figured out now. At first I couldn't reproduce your issue because I was installing to a local prefix and all the files appeared to be there. I ultimately decided to switch over to scikit-build since the hacks I'd need to keep setuptools working would probably not be worth the hassle. I had trouble with regular scikit-build, but apparently now there is a "new-new" method, scikit-build-core, which had its fortran example code last modified just last week. So based on that example, I've got something that seems to work with both setup.py and pip venvs. Let me know if the latest push works for you, and if anything needs clarification. |
as it's not needed anymore
yunjunz
changed the title
numpy.distutils with CMake and scikit-build-core build backend
yunjunz
approved these changes
Jun 22, 2024
There was a problem hiding this comment.
This works, both tests on my laptop and circle CI passed. Thank you so much @rtburns-jpl!
One last clarification on the documentation, is it still possible for users/developers to manually compile the fortran code? If so, what would be the recommended procedure? I have tried the current option 3 and it did not work, as shown below. Otherwise, we could just remove this part of the instruction to avoid potential confusion.
rge(gla,glo,u,v,w,x,y,z)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Constructing wrapper function "rot1"...
rot1(theta,x,y,z,u,v,w)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Constructing wrapper function "rot3"...
rot3(theta,x,y,z,u,v,w)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Constructing wrapper function "setjd0"...
setjd0(iyr,imo,idy)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Constructing wrapper function "civjts"...
civjts(iyr,imo,idy,ihr,imn,sec,tsec)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Constructing wrapper function "jtsciv"...
jtsciv(tsec,iyr,imo,idy,ihr,imn,sec)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Constructing wrapper function "civmjd"...
civmjd(iyr,imo,idy,ihr,imn,sec,mjd,fmjd)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Constructing wrapper function "mjdciv"...
mjdciv(mjd,fmjd,iyr,imo,idy,ihr,imn,sec)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Creating wrapper for Fortran function "utc2ttt"("utc2ttt")...
Constructing wrapper function "utc2ttt"...
utc2ttt = utc2ttt(tutc)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Creating wrapper for Fortran function "gps2ttt"("gps2ttt")...
Constructing wrapper function "gps2ttt"...
gps2ttt = gps2ttt(tgps)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Creating wrapper for Fortran function "utc2tai"("utc2tai")...
Constructing wrapper function "utc2tai"...
utc2tai = utc2tai(tutc)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Creating wrapper for Fortran function "getutcmtai"("getutcmtai")...
Constructing wrapper function "getutcmtai"...
getutcmtai = getutcmtai(tsec)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Creating wrapper for Fortran function "tai2tt"("tai2tt")...
Constructing wrapper function "tai2tt"...
tai2tt = tai2tt(ttai)
Generating possibly empty wrappers"
Maybe empty "solid-f2pywrappers.f"
Creating wrapper for Fortran function "gps2tai"("gps2tai")...
Constructing wrapper function "gps2tai"...
gps2tai = gps2tai(tgps)
Constructing COMMON block support for "stuff"...
rad,pi,pi2
Constructing COMMON block support for "comgrs"...
a,e2
Constructing COMMON block support for "limitflag"...
leapflag
Constructing COMMON block support for "mjdoff"...
mjd0
Wrote C/API module "solid" to file "./solidmodule.c"
Fortran 77 wrappers are saved to "./solid-f2pywrappers.f"
Traceback (most recent call last):
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/bin/f2py", line 8, in <module>
sys.exit(main())
^^^^^^
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 765, in main
run_compile()
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 737, in run_compile
builder.compile()
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 178, in compile
self.run_meson(self.build_dir)
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 171, in run_meson
self._run_subprocess_command(setup_command, build_dir)
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 167, in _run_subprocess_command
subprocess.run(command, cwd=cwd, check=True)
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Users/yunjunz/tools/mambaforge/envs/test_pysolid/lib/python3.12/subprocess.py", line 1955, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'meson'
|
Yeah, the manual installation still works for me. I think you're just missing meson, but you can install it from conda-forge ( |
|
It looks like @yunjunz beat me to the feedback/review, but I just wanted to say this looks great! Thanks @rtburns-jpl |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As setuptools cannot natively call f2py, we call it ourselves by using a custom build extension.
Closes #78
(This seemed like the simplest way to do things - keep using setuptools, and have it manually call f2py under the hood.)