metadata missing from pypi

[dimbleby]

$ curl -s https://pypi.org/pypi/dulwich/json | jq ".info.requires_dist"
null

This metadata would be useful to package managers, allowing them to figure out dependencies before they have to go and download the wheel.

pypi/warehouse#6231 (comment) says that this happens if the sdist is uploaded before the wheels - which, checking upload timestamps, does indeed seem to be how things happen on dulwich releases.

I'm not entirely sure how the sdist gets uploaded for this project, it doesn't seem to be in the github workflow.

twine upload dist/* would work fine, apparently, but the code in this repository seems only to upload the wheels - is there some other process by which the sdist is uploaded?

Anyway, if you could arrange to upload the wheels first then I expect that would populate the metadata.

[jelmer]

Right now, I manually upload the dist tarball as part of the release process (using https://github.com/jelmer/releaser). One option would be to move that into a GitHub action (along with the building of the wheels) but that would mean losing the PGP signature on the source tarball, which would be a step back.

I think I'd prefer to wait for the proper fix - pypa/setuptools#1805 - unless there is a workaround without downsides.

[dimbleby]

Manually uploading the sdist isn't the problem, if you want to do that, but if the releaser workflow - which I am not familiar with - allowed you to do that after github had released the wheels then that would be great.

I suspect that waiting for pypa/setuptools#1805 is likely to mean waiting forever.

Still, this isn't the biggest deal in the world. If it's for some reason hard for you to upload the sdist after the wheels - well, so be it.

[jelmer]

Releaser builds and uploads the source tarball first (possibly aborting if pypi rejects it), and then pushes the changes to git.

I could push to git first and then upload the source tarball with pgp signature once the wheels have uploaded, but that would lengthen the process of doing a release from about 5 minutes to at least half an hour (as it takes a while for the wheels to build).

Since the consequences here are just that pip installations become slightly slower and since the maintainers of pip and setuptools are the same group, the fix should be really be there - rather than as a workaround in each of the thousands of packages that upload binary wheels.

I'll close this, but open to less intrusive workarounds - and I'd encourage people to report this in the setuptools bug if it affects them.

[dimbleby]

One option would be to move that into a GitHub action ... but that would mean losing the PGP signature on the source tarball

I don't think this has to be true - you could provide your identity as a secret to the pipeline, and sign from within the action.

(But I am not sufficiently motivated to figure out the details, and probably you're not either!)

[jelmer]

I don't want to upload my PGP (sub)key to GitHub, that's a significant step back in personal security. I could generate a separate key, but that means another key to manage the lifecycle of.

[jelmer]

Looks like we could reverse course on this, since pypi has dropped support for PGP: https://blog.pypi.org/posts/2023-05-23-removing-pgp/

Happy to accept a PR to move the source upload to a github action as well, if anybody is interested in working on that.