Datasets or benchmarks to add to Polaris

[cwognum]

Hi everyone,

Polaris is built to host any relevant dataset or benchmark in ML-based drug discovery. The best way to help us at this stage of development is to identify and upload to the Hub such high-quality datasets or benchmarks. This will not only add value to the Hub by hosting more relevant content, but will also inform our feature roadmap.

This GH discussion is meant to collect, share and triage ideas for benchmarks or datasets to add next. I will make sure to regularly update the overview down below.

If Polaris grows, it might make sense to move this to GH issues with a dedicated GH board.

---

How to add a dataset or benchmark?

If you're interested in adding a dataset or benchmark to the Polaris Hub, here are some pointers to get started:

1. Before you start, make sure that you have the rights to redistribute the dataset or benchmark through Polaris. Polaris supports a subset of CC licenses and the MIT license.
2. If your dataset is small in size and has no complex dtypes such as 3D structures of proteins or conformers, you can save all of it efficiently in a Pandas DataFrame. In this case, these tutorials are a great way to get started: Data Models and Data Curation
3. If your data types do get more complex or if the datasets get large (>10GB), you will want to create a Zarr dataset. In this case, these tutorials are a great way to get started: Zarr Datasets and Dataset Factory.
4. Finally, since dataset curation can involve subjective decisions, it's important to be transparent about how a dataset or benchmark has been created. To this end, we've started the Polaris Recipes repository, and we invite you to create a PR there that can be run to reproduce the same dataset or benchmark.

Let us know if you get stuck on any of the above! Our aim is to make this process as seamless as possible and your feedback is invaluable in improving our documentation.

---

Datasets or benchmarks to add

Last updated: July 17, 2024

Completed

• Biogen ADMET dataset - Thanks @zhu0619
• PKIS-1 (Kinase Panel) - Thanks @zhu0619
• PKIS-2 (Kinase Panel) - Thanks @zhu0619
• Protein Gym - Thanks to @danieldritter

In progress

• ADME@NCATS - In progress by @stwhitfield
• D3R - In progress by @stwhitfield
• AstraZeneca ADME - In progress by @stwhitfield
• MoleculeACE - In progress by @derekvantilborg
• CHORISO - In progress by @doncamilom
• BEND - In progress by @frederikkemarin
• MARCEL - In progress by @zhu0619
• Novartis Surrogate CYPP450 dataset - In progress by @zhu0619
• PDE10A - In progress by @dessygil

Backlog

• ExCAPE-DB

[dessygil]

@timkucera I was wondering if I could take a swing at adding the datasets on ProteinShake?

[LFhase]

May I ask how to create and upload a customized dataset? It keeps responding " Client error '401 Unauthorized' for URL"

[cwognum]

Hey @LFhase, thanks for contributing to Polaris!

Have you logged in to Polaris yet? If so, could you try overwriting your login details using:

polaris login --overwrite

If this doesn't work, please file a bug report (there's an issue template for this in this repo) and I'll investigate!

[LFhase]

Hi @cwognum , thank you very much for your prompt reply! I have tried the login command and the issue remains with another error report. The details are given in the bug report #145 . Thank you very much for helping look into it!