Skip to content

Feat/add folmsbee conformer benchmark#429

Open
lwehrhan wants to merge 4 commits intoddmms:mainfrom
lwehrhan:feat/add-folmsbee-conformer-benchmark
Open

Feat/add folmsbee conformer benchmark#429
lwehrhan wants to merge 4 commits intoddmms:mainfrom
lwehrhan:feat/add-folmsbee-conformer-benchmark

Conversation

@lwehrhan
Copy link

@lwehrhan lwehrhan commented Mar 16, 2026

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

Summary

The Folmsbee dataset of low-energy conformers of drug-like molecules. The differences in energy are smaller compared to the Wiggle500 dataset and it features a greater number of molecules. The highest available level of theory for energy evaluations to be used as ground-truth is DLPNO-CCSD(T). This is a test for moving the benchmarks of mlip-audit into this repository. I have included an analysis script for this benchmark, however would like to kindly ask for assistance with building and harmonizing the Dash layout.

Linked issue

Resolves #427

Progress

  • Calculations
  • Analysis
  • Application
  • Documentation

Testing

New decorators/callbacks

@joehart2001
Copy link
Collaborator

joehart2001 commented Mar 18, 2026

Hi @lwehrhan, thank you for your PR and its looking great overall! A few things:

  1. would you be able to share the data file so i can uplaod it to our s3 bucket so i can test the calc and analysis is running as expected?
  2. i have pushed the app and also the metrics.yml, would you be able to check over this metrics file to make sure its correct?

Once we've got the data file uploaded, i think we can make a few changes to the calc script for consistency with similar benchmarks, but i think the changes will be minor.

Just a note, make sure you to fetch any changes i've made before working locally, otherwise your next push may overwrite my changes.

thanks!

# Add D3 calculator for this test
calc = model.add_d3_calculator(calc)

data_path = Path(__file__).parent / "data" / "folmsbee_dataset.json"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
data_path = Path(__file__).parent / "data" / "folmsbee_dataset.json"
data_path = (
download_s3_data(
filename="Folmsbee.zip",
key="inputs/conformers/Folmsbee/Folmsbee.zip",
)
/ "Folmsbee"
)

import pytest
from tqdm import tqdm

from ml_peg.models.get_models import load_models
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from ml_peg.models.get_models import load_models
from ml_peg.calcs.utils.utils import download_s3_data
from ml_peg.models.get_models import load_models

@joehart2001 joehart2001 added the new benchmark Proposals and suggestions for new benchmarks label Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new benchmark Proposals and suggestions for new benchmarks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: new benchmark with folmsbee conformer dataset

3 participants