-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is there matbench benchmark result for Wrenformer? #72
Comments
I believe @hrushikesh-s is currently working on submitting Wrenformer to Matbench. Maybe he can teel you more. In case you haven't seen there are some preliminary results for various Wrenformer hyperparameter settings plotted in #44. |
@hongshuh Also, what are you planning on using Wrenformer for? If discovery, these results might interest you: https://matbench-discovery.materialsproject.org/preprint#results. |
Yea, I am also following the discorvery benchmark, It seems to handle the task as a regression problem by predicting the energy above hull, rather than treating it as a classification task of identifying whether a material is stable or not. I am a bit puzzled by this approach, since the aim seems to be the identification of stable materials, which would intuitively seem to be a classification task. |
That's right.
I have some preliminary results which suggest doing direct classification does not improve over regression. But I think that's definitely something that could be investigated further. If you want to check how well a Wrenformer stability classifier performs compared to the Wrenformer regressor, that would be a very welcome contribution to MBD! |
This section from Bartel et al. 2021 is also relevant here:
|
Thanks! Maybe the regression values provide more information to the model than just "stable" or "unstable" labels. |
I saw in the commit history that you have conducted some experiments in matbench benchmark, It's a very good idea and model, but I may not have enough computational resources to run it, I would like to know if you have final resuts?
The text was updated successfully, but these errors were encountered: