Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions tests/snappi_tests/intf_utils/intf_accept_metrics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# This file defines the interfaces that snappi tests accept external metrics.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All common label names are missing too, e.g.: PortId, QueueId, PSUId....

otherwise it will be very hard to create unified dashboard, because each tests could use its own names, and causing problems in filters.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the definitions of the metric names and meta are missing in the file, we need to get them defined and show a unified format. this will be used for crafting the dashboards.


# Metrics data are organized into the hierarchies below
# ResourceMetrics
# ├── ResourceID
# └── ScopeMetrics
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need the level of ScopeMetrics.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought is
Resource level: all metrics from one test run
Scope level: all metrics belonging to one device
Metric level: all metrics belonging to one category
I might be wrong. Let's discuss this topic tomorrow.

# ├── ScopeID
# └── Metric
Copy link
Copy Markdown
Collaborator

@r12f r12f Dec 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create a generic Metric class that represents a single metric, which contains:

  • description/labels: Name, Description, unit, ....
  • Value: single layer is good enough with inheritance.
  • Reporter: Reference to MetricsReporter. Register itself to Reporter when created, so Reporter can gather all metrics after everything is changed.
class Metric...:
    def __init__(name, ...., reporter):
        reporter.add_metric(self)
        ....

class GaugeMetric(Metric):
    def __init__(name, ...., reporter):
        super.__init__(...)
        self.value = 0

    def set(v):
        self.value = v
....


reporter = MetricReporterFactory(...).build()
port_rx = GaugeMetric(...., reporter)

port_rx.set(123)
reporter.report(time)

Copy link
Copy Markdown
Collaborator

@r12f r12f Dec 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hence, ultimately the final code for people to use would be:

metrics = {
   "PortRx" = GaugeMetric(......, reporter)
   ....
}

for r in csv:
    for c in r:
        metric[c.title].set(c.value)

reporter.report(time)

# ├── Name
# ├── Description
# ├── Unit
# ├── metadata
# └── data
# └── Gauge
#
# A ResourceMetrics has its ID and a list of ScopeMetrics objects.
# A ScopeMetrics has its ID and a list of Metric objects.
# A Metric has several attributes and data. So far we only have Gauge type data.
# A Gauge has a list of NumberDataPoint objects.
# A NumberDataPoint has its label, value, flags and the timestamp at which the data was collected.
# +---------------------+
# | DataPoint 1 |
# | +---------+ +-----+ |
# +-----+ | |timestamp| |label| |
# | 1 |-->| +---------+ +-----+ |
# +-----+ | |
# | . | | +-----+ +-----+ |
# | . | | |value| |flags| |
# | . | | +-----+ +-----+ |
# | . | +---------------------+
# | . | .
# | . | .
# | . | .
# | . | +---------------------+
# | . | | DataPoint M |
# +-----+ | +---------+ +-----+ |
# | M |-->| |timestamp| |label| |
# +-----+ | +---------+ +-----+ |
# | |
# | +-----+ +-----+ |
# | |value| |flags| |
# | +-----+ +-----+ |
# +---------------------+

from typing import List, Dict, Union

class NumberDataPoint:
def __init__(self, time_unix_nano: int, label: List[Dict[str, str]], value: Union[int, float], flags: int = None):
self.time_unix_nano = time_unix_nano # UNIX Epoch time in nanoseconds
self.label = label # The key of key-value pairs in dictionaries
self.value = value # Metric value (can be double or integer)
self.flags = flags # Optional flags

def __repr__(self):
return (f"NumberDataPoint(label={self.label}, "
f"time_unix_nano={self.time_unix_nano}, value={self.value}, flags={self.flags})")


class Gauge:
def __init__(self):
self.data_points = [] # List of NumberDataPoint objects

def add_data_point(self, data_point):
self.data_points.append(data_point)

def __repr__(self):
return f"Gauge(data_points={self.data_points})"


class Metric:
def __init__(self, name, description, unit, data_points, metadata=None):
self.name = name # Metric name
self.description = description # Metric description
self.unit = unit # Metric unit (e.g., seconds, bytes)
self.data = data # Can be Gauge only
self.metadata = metadata or {} # Default to an empty dictionary if None

def __repr__(self):
return (f"Metric(name={self.name}, description={self.description}, "
f"unit={self.unit}, data={self.data})")


# a ScopeMetrics object's ID is device_id
class ScopeMetrics:
def __init__(self, device_id):
self.device_id = device_id
self.metrics = []

def add_metric(self, metric):
self.metrics.append(metric)

def __repr__(self):
return f"ScopeMetrics(scope={self.scope}, metrics={self.metrics})"


# a ResourceMetrics object's ID is test_run_id
class ResourceMetrics:
def __init__(self, test_run_id, os_version):
self.test_run_id = test_run_id
self.os_version = os_version
self.scope_metrics = []

def add_scope_metrics(self, scope_metric):
self.scope_metrics.append(scope_metric)

def __repr__(self):
return f"ResourceMetrics(resource={self.resource}, scope_metrics={self.scope_metrics})"