Skip to content

To introduce MetricGroup class #3264

@sadra-barikbin

Description

@sadra-barikbin

🚀 Feature

Hi there!
It might be useful to group some metrics together to have less code and more coherence. My use case was using some ignite metrics in HuggingFace Trainer API which expects a single callable for computing metrics. Having this feature, that could be easily realizable with Ignite. MetricGroup could be something like this:

from typing import Any, Dict
from ignite.metrics import Metric


class MetricGroup(Metric):
  _state_dict_all_req_keys = ('metrics',)

  def __init__(self, metrics:Dict[str, Metric]):
    self.metrics = metrics
    super(MetricGroup, self).__init__()

  def reset(self):
    for m in self.metrics.values():
      m.reset()

  def update(self, output):
    for m in self.metrics.values():
      m.update(m._output_transform(output))

  def compute(self) -> Dict[str, Any]:
    return {k: m.compute() for k,m in self.metrics.items()}

Usage:

metric = MetricGroup({'acc': Accuracy(), 'perplexity': Perplexity()})
metric.attach(engine) # or sth in HF Trainer API

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions