Build Python polars wheels with PGO

[messense]

Problem description

It seems that PGO is a success for pydantic-core, I wonder whether it will speedup py-polars.

[ritchie46]

Yes, I'd be very interested in that. I think including the db-benchmark and the tpch benchmark code in the guided data seem to me to be interesting candidates.

Tests would be easiest. Not any experience with setting it up though, so any help on this is appreciated.

[messense]

@ritchie46 Is there any easy way to compare two benchmark results? I'd like to confirm that it actually improves performance before putting too much effort.

[ritchie46]

The easiest start are the TPCH benchmarks: https://github.com/pola-rs/tpch/tree/main/polars_queries

The repo has a Makefile that creates the dataset. Running this provides a timing table and some plot utils. Now that I think of it, that's also good PGO input.

[zamazan4ik]

I think you also could be interested in my recent benchmark regarding PGO to different kinds of software (including a lot of databases and database libraries (like RocksDB)) - https://github.com/zamazan4ik/awesome-pgo .

[jmakov]

From some of the blog posts it looks like +15% can be done. That's the difference a CPU makes from one generation to another.

[FilipAndersson245]

Any update on this? PGO is a very interesting way to improve performance with the only cost is deployment and build complexity.
I can recommend looking at https://github.com/Kobzol/cargo-pgo for inspiration on how to run it.