Redesign and implement the telemetry framework (sonic-net#20387)

What is the motivation for this PR?
The current telemetry framework has many serious problems:

Hard to import, because it is outside of the tests folder
No implementation, hard to try out
Hard to use and extend, because the interface is too primitive
No tests and can break easily
How did you do it?
This PR add a few things to improve the current telemetry framework:

Add new design doc with example code provided in the doc.
Implement both TS (timeseries) and DB (database) reporter, so we can try the feature easily.
How did you verify/test it?
Run all tests locally and passed.

Any platform specific information?
N/A.

Fix TS reporters.

Fix TSRepoter.

Fix TS reporter.

minor fix.

Fix example.

Reverse relationship of metrics and reporter to support more types of metrics in future.

Remove bydesign not working test.

Fix example.

Update tests.

Update TS reporter and DB repoter.

Update minor things.

Remove no required code.

Revert unexpected change.

Rename and update doc.

* Get os version and job id (sonic-net#1)

Get os version and elastictest job id from correct place.

* Revert OS version change.

* Remove duplicated mock reporter fixture.

---------

Co-authored-by: Yutong Zhang <[email protected]>
Signed-off-by: Lakshmi Yarramaneni <[email protected]>

Author: 2 people

docs/tests/telemetry.md

@@ -0,0 +1,3 @@
+# Telemetry Utils
+
+Please refer to [SONiC Mgmt Test Telemetry Framework](../../../docs/tests/telemetry.md) design doc.

test_reporting/telemetry/README.md

@@ -0,0 +1,79 @@
+"""
+SONiC Mgmt Test Telemetry Framework
+
+"""
+
+# Base classes
+from .base import Reporter, Metric, MetricCollection, MetricDefinition
+
+# Metric types
+from .metrics import GaugeMetric, HistogramMetric
+
+# Reporters
+from .reporters import TSReporter, DBReporter
+
+# Device metric collections
+from .metrics.device import (
+    DevicePortMetrics, DevicePSUMetrics, DeviceQueueMetrics,
+    DeviceTemperatureMetrics, DeviceFanMetrics
+)
+
+# Constants and labels
+from .constants import (
+    # Common Labels
+    METRIC_LABEL_DEVICE_ID, METRIC_LABEL_DEVICE_PORT_ID, METRIC_LABEL_DEVICE_PSU_ID,
+    METRIC_LABEL_DEVICE_QUEUE_ID, METRIC_LABEL_DEVICE_SENSOR_ID, METRIC_LABEL_DEVICE_FAN_ID,
+
+    # Port Metrics
+    METRIC_NAME_PORT_RX_BPS, METRIC_NAME_PORT_TX_BPS, METRIC_NAME_PORT_RX_UTIL, METRIC_NAME_PORT_TX_UTIL,
+
+    # PSU Metrics
+    METRIC_NAME_PSU_VOLTAGE, METRIC_NAME_PSU_CURRENT, METRIC_NAME_PSU_POWER,
+
+    # BGP Metrics
+    METRIC_NAME_BGP_CONVERGENCE_TIME_PORT_RESTART,
+
+    # Units
+    UNIT_SECONDS, UNIT_BYTES_PER_SECOND, UNIT_PERCENT, UNIT_COUNT
+)
+
+# Pytest fixtures (imported for convenience, but should be used via conftest.py)
+from .fixtures import ts_reporter, db_reporter
+
+# Version information
+__version__ = "1.0.0"
+
+# Public API - define what gets imported with "from common.telemetry import *"
+__all__ = [
+    # Base classes
+    'Reporter', 'Metric', 'MetricCollection', 'MetricDefinition',
+
+    # Metric types
+    'GaugeMetric', 'HistogramMetric',
+
+    # Reporters
+    'TSReporter', 'DBReporter',
+
+    # Device metrics
+    'DevicePortMetrics', 'DevicePSUMetrics', 'DeviceQueueMetrics',
+    'DeviceTemperatureMetrics', 'DeviceFanMetrics',
+
+    # Essential constants
+    'METRIC_LABEL_DEVICE_ID', 'METRIC_LABEL_DEVICE_PORT_ID', 'METRIC_LABEL_DEVICE_PSU_ID',
+    'METRIC_LABEL_DEVICE_QUEUE_ID', 'METRIC_LABEL_DEVICE_SENSOR_ID', 'METRIC_LABEL_DEVICE_FAN_ID',
+
+    # Port metric names
+    'METRIC_NAME_PORT_RX_BPS', 'METRIC_NAME_PORT_TX_BPS', 'METRIC_NAME_PORT_RX_UTIL', 'METRIC_NAME_PORT_TX_UTIL',
+
+    # PSU metric names
+    'METRIC_NAME_PSU_VOLTAGE', 'METRIC_NAME_PSU_CURRENT', 'METRIC_NAME_PSU_POWER',
+
+    # BGP metric names
+    'METRIC_NAME_BGP_CONVERGENCE_TIME_PORT_RESTART',
+
+    # Common units
+    'UNIT_SECONDS', 'UNIT_PERCENT', 'UNIT_COUNT', 'UNIT_BYTES_PER_SECOND',
+
+    # Pytest fixtures
+    'ts_reporter', 'db_reporter'
+]