Fix benchmark script #32635
Fix benchmark script #32635
Conversation
| for metric in metrics: | ||
| keys = metric.split(".") | ||
| value = report | ||
| value = report.to_dict() |
There was a problem hiding this comment.
v0.2.0 doesn't need to_dict, but 0.4.0 needs this.
There was a problem hiding this comment.
v0.2.0 of what package?
There was a problem hiding this comment.
sorry, it's optimum-benchmark
There was a problem hiding this comment.
The issue is specifically from this __post_init__ in v0.2.0 that was incomplete ; only casts the config dict to the appropriate dataclass:
https://github.com/huggingface/optimum-benchmark/blob/v0.2.0/optimum_benchmark/base.py#L25-L29
which was fixed/completed in:
https://github.com/huggingface/optimum-benchmark/blob/v0.3.0/optimum_benchmark/benchmark/base.py#L24-L33
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
amyeroberts
left a comment
amyeroberts
left a comment
There was a problem hiding this comment.
Thanks for fixing @ydshieh!
Change looks OK - but I'm guess it's not backwards compatible if the user has v0.2.0? Is optimum-benchmark version pinned anywhere in the library?
|
Thanks. I update the min. version. |
* fix * >= 0.3.0 --------- Co-authored-by: ydshieh <[email protected]>
What does this PR do?