Skip to content

Commit 733a459

Browse files
Add skill evaluation dataset for cuopt-lp-milp-api-python (#1172)
## Summary Initial skill evaluation dataset for `cuopt-lp-milp-api-python` at `skills/cuopt-lp-milp-api-python/evals/evals.json`. 99 entries adapted from the [microsoft/OptiGuide](https://github.com/microsoft/OptiGuide) IndustryOR corpus (MIT, attribution in `evals/SOURCES.md`). - `ground_truth` is the numeric optimal value; rubric requires exact match to the precision shown (no tolerance) - `expected_behavior` is generic across all entries — does not pre-categorize as LP vs MILP - Each entry has a `source` field referencing the dataset row for traceability QP eval set is out of scope (the corpus has no genuine QP problems) and will follow in a separate PR. Authors: - Ramakrishnap (https://github.com/rgsl888prabhu) Approvers: - Ishika Roy (https://github.com/Iroy30) URL: #1172
1 parent a4de253 commit 733a459

2 files changed

Lines changed: 1131 additions & 0 deletions

File tree

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Sources
2+
3+
Eval prompts in `evals.json` for the `cuopt-lp-milp-api-python` skill are
4+
adapted from the **OptiGuide / OptiMind IndustryOR** dataset:
5+
6+
- Repository: [microsoft/OptiGuide](https://github.com/microsoft/OptiGuide)
7+
- File: [`optimind/data/optimind_cleaned_classified_industryor.csv`](https://github.com/microsoft/OptiGuide/blob/main/optimind/data/optimind_cleaned_classified_industryor.csv)
8+
- License: MIT (Copyright (c) Microsoft Corporation)
9+
10+
Each entry's `source` field references the original row index. Problem
11+
statements are quoted verbatim; ground-truth values are the dataset's
12+
optimal objective values.
13+
14+
## License
15+
16+
The MIT license under which the source dataset is distributed:
17+
18+
```
19+
MIT License
20+
21+
Copyright (c) Microsoft Corporation.
22+
23+
Permission is hereby granted, free of charge, to any person obtaining a copy
24+
of this software and associated documentation files (the "Software"), to deal
25+
in the Software without restriction, including without limitation the rights
26+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
27+
copies of the Software, and to permit persons to whom the Software is
28+
furnished to do so, subject to the following conditions:
29+
30+
The above copyright notice and this permission notice shall be included in all
31+
copies or substantial portions of the Software.
32+
33+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
34+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
35+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
36+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
37+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
38+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
39+
SOFTWARE
40+
```

0 commit comments

Comments
 (0)