-
-
Notifications
You must be signed in to change notification settings - Fork 666
Closed
Labels
Description
🚀 Feature
Hi there!
It might be useful to group some metrics together to have less code and more coherence. My use case was using some ignite metrics in HuggingFace Trainer API which expects a single callable for computing metrics. Having this feature, that could be easily realizable with Ignite. MetricGroup could be something like this:
from typing import Any, Dict
from ignite.metrics import Metric
class MetricGroup(Metric):
_state_dict_all_req_keys = ('metrics',)
def __init__(self, metrics:Dict[str, Metric]):
self.metrics = metrics
super(MetricGroup, self).__init__()
def reset(self):
for m in self.metrics.values():
m.reset()
def update(self, output):
for m in self.metrics.values():
m.update(m._output_transform(output))
def compute(self) -> Dict[str, Any]:
return {k: m.compute() for k,m in self.metrics.items()}Usage:
metric = MetricGroup({'acc': Accuracy(), 'perplexity': Perplexity()})
metric.attach(engine) # or sth in HF Trainer API