Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions tests/snappi_tests/intf_utils/intf_accept_metrics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# This file defines the interfaces that snappi tests accept external metrics.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All common label names are missing too, e.g.: PortId, QueueId, PSUId....

otherwise it will be very hard to create unified dashboard, because each tests could use its own names, and causing problems in filters.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the definitions of the metric names and meta are missing in the file, we need to get them defined and show a unified format. this will be used for crafting the dashboards.

import logging
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file it not part of intf_utils, because it is not related to interface.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sm-xu this comment is missing.

import json
import datetime
import time

from typing import List, Dict, Union
from intf_report_metrics import MetricReporterFactory OtelMetricReporter

class Metric:
def __init__(self,
name,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing type hints

description,
unit,
timestamp,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following fields is common for entire tests, so it can be move into the reporter as common metadata:

  • testbed_name
  • os_version
  • testcase_name
  • test_run_id

The following fields are common for all metrics in a single report action, so it can be lifted into the reporter's report function parameters:

  • timestamp

The following fields are not clear on its purpose, we need to rename it to make it clear:

  • component_id

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe the timestamp here means the test_start_time?

testbed_name,
os_version,
testcase_name,
test_run_id,
device_id,
component_id,
reporter: MetricReporterFactory,
metadata = None):
"""
Args:
name (str): metric name (e.g., psu power, sensor temperature, port stats, etc.)
description (str): brief description of the metric
unit (str): metric unit (e.g., seconds, bytes)
timestamp (int): UNIX Epoch time in nanosecond, when the metric is collected
testbed_name (str): testbed name
os_version (str): switch OS version
testcase_name (str): test case name
test_run_id (str): ID of the test run
device_id (str): switch device ID
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be lifted up to reporter, since it is common to all

component_id (str): ID of the component (e.g., psu, sensor, port, etc.)
reporter(obj): object of MetricReporterFactory
metadata (str): e.g. serial number, model number, etc. Default to an empty dictionary if None
Returns:
N/A
"""
self.name = name
self.description = description
self.unit = unit
self.timestamp = timestamp
self.testbed_name = testbed_name
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all these fields can be moved to reporter, since it is shared by everyone.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestMetrics itself can be removed, once we add the per metric class.

self.os_version = os_version
self.testcase_name = testcase_name
self.test_run_id = test_run_id
self.device_id = device_id
self.component_id = component_id
self.reporter = reporter.create_metrics_reporter()
self.metadata = metadata or {}

def __repr__(self):
return (f"Metric(name={self.name!r}, description={self.description!r}, "
f"unit={self.unit!r}, timestamp={self.timestamp!r}, "
f"testbed_name={self.testbed_name!r}, os_version={self.os_version!r}, "
f"testcase_name={self.testcase_name!r}, test_run_id={self.test_run_id!r}, "
f"device_id={self.device_id!r}, component_id={self.component_id!r}, "
f"reporter={self.reporter!r}), metadata={self.metadata!r})")


class GaugeMetric(Metric):
def __init__(self,
name,
description,
unit,
timestamp,
testbed_name,
os_version,
testcase_name,
test_run_id,
device_id,
component_id,
reporter: MetricReporterFactory,
metadata = None,
metrics: Dict[str, Union[int, str, float]] = None):
# Initialize the base class
super().__init__(name, description, unit, timestamp, testbed_name, os_version,
testcase_name, test_run_id, device_id, component_id, reporter, metadata, metrics)

# Additional fields for GaugeMetric
self.metrics = metrics or {}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

each Metric should only represent a single metric. If we are trying to create something that holds all metrics, it should be 1 layer above, say MetricCollections / MetricList / Metrics or whatever.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the purpose of this field is not too clear...


def add_metrics(self, new_metrics: Dict[str, Union[int, str, float]]):
# Add new elements to the metrics dictionary.
# new_metrics: Dictionary containing new key-value pairs to append.
self.metrics.update(new_metrics)

def __repr__(self):
return (f"ExtendedMetric(name={self.name!r}, "
f"description={self.description!r}, "
f"unit={self.unit!r}, "
f"timestamp={self.timestamp!r}, "
f"testbed_name={self.testbed_name!r}, "
f"os_version={self.os_version!r}, "
f"testcase_name={self.testcase_name!r}, "
f"test_run_id={self.test_run_id!r}, "
f"device_id={self.device_id!r}, "
f"component_id={self.component_id!r}, "
f"component_id={self.reporter!r}, "
f"metadata={self.metadata!r}, "
f"metrics={self.metrics!r})")
36 changes: 36 additions & 0 deletions tests/snappi_tests/intf_utils/intf_report_metrics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import logging
import json
import datetime
import time

from typing import List, Dict, Union
from intf_accept_metrics import Metric GaugeMetric

class MetricReporterFactory:
def __init__(self, connection):
# Temporary code initializing the MetricReporterFactory with a database connection
# will be replaced with OpenTelemetry connection
self.connection = connection
self.reporter = None

def create_metrics_reporter(self):
self.reporter = OtelMetricReporter(self.connection)
return self.reporter

class OtelMetricReporter:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reporter should not be limited to Otel.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not addressed

def __init__(self, connection):
# Temporary code initializing the OtelMetricReporter
# will be replaced with OpenTelemetry connection
self.connection = connection
self.metrics = []

def register_metric(self, metrics):
self.metrics.append(metrics)

def report(self, timestamp):
# Temporary code to report metrics
print(f"Reporting metrics at {timestamp}")
for metric in self.metrics:
print(metric)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will be great to create a new abstracted function for us to override.