Add loganalyzer for sairedis for advanced reboot tests

[shi-su]

Description of PR

Summary:
Fixes # (issue)

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Approach

What is the motivation for this PR?

How did you do it?

Add loganalyzer for sairedis for advanced reboot tests to gather information about the time the device enters different states in fast/warm reboot.

How did you verify/test it?

Tested locally and check the time for each state:

{
    "Services": {
        "bgp": {
            "timestamp": {
                "Stopping": "Apr 23 21:58:52.955303", 
                "Starting": "Apr 23 22:00:12.362430", 
                "Stopped": "Apr 23 21:58:59.164886", 
                "Started": "Apr 23 22:00:14.106322"
            }, 
            "start_time": 1.743892, 
            "stop_time": 6.209583, 
            "reboot_time": 74.941436
        }, 
        "radv": {
            "timestamp": {
                "Stopping": "Apr 23 21:58:52.206894", 
                "Starting": "Apr 23 22:00:17.860428", 
                "Stopped": "Apr 23 21:58:52.919687", 
                "Started": "Apr 23 22:00:19.533290"
            }, 
            "start_time": 1.672862, 
            "Stopping count": 2, 
            "stop_time": 0.712793, 
            "reboot_time": 86.613603
        }, 
        "longest_downtime": 150.668403, 
        "total_downtime": "N/A", 
        "teamd": {
            "timestamp": {
                "Stopping": "Apr 23 21:59:12.966736", 
                "Starting": "Apr 23 22:00:15.975674", 
                "Stopped": "Apr 23 21:59:19.739232", 
                "Started": "Apr 23 22:00:17.385416"
            }, 
            "start_time": 1.409742, 
            "stop_time": 6.772496, 
            "reboot_time": 57.646184
        }, 
        "gbsyncd": {
            "timestamp": {
                "Stopping": "Apr 23 21:59:34.710807", 
                "Starting": "Apr 23 22:00:15.972795", 
                "Stopped": "Apr 23 21:59:35.435963", 
                "Started": "Apr 23 22:00:17.588242"
            }, 
            "start_time": 1.615447, 
            "stop_time": 0.725156, 
            "reboot_time": 42.152279
        }, 
        "syncd": {
            "timestamp": {
                "Stopping": "Apr 23 21:59:19.778560", 
                "Starting": "Apr 23 22:00:15.976602", 
                "Stopped": "Apr 23 21:59:31.518727", 
                "Started": "Apr 23 22:00:17.784502"
            }, 
            "start_time": 1.8079, 
            "stop_time": 11.740167, 
            "reboot_time": 46.265775
        }, 
        "swss": {
            "timestamp": {
                "Stopping": "Apr 23 21:59:09.799595", 
                "Starting": "Apr 23 22:00:14.143509", 
                "Stopped": "Apr 23 21:59:10.910408", 
                "Started": "Apr 23 22:00:15.937556"
            }, 
            "start_time": 1.794047, 
            "stop_time": 1.110813, 
            "reboot_time": 65.027148
        }, 
        "reboot_time": 149.513459
    }, 
    "sairedis_state": {
        "default_route_set": {
            "timestamp": {
                "Started": "Apr 23 22:00:41.501188"
            }
        }, 
        "sai_switch_create": {
            "timestamp": {
                "Started": "Apr 23 22:00:27.838804", 
                "Stopped": "Apr 23 22:00:27.840236"
            }, 
            "time": 0.001432
        }
    }
}

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[vaibhavhd]

It would be great if we can generalize the LogAnalyzer to support argument based log type grepping.
For example, here analyze_sairedis is added. But, in future this can grow to many different flags.
To prevent that, if we the caller can specify what files need to be analyzed, this would be useful for several other usecases.

[vaibhavhd]

Looking good. Some refactoring suggestions added. Suggest to test another unrelated test which uses log_analyzer, to make sure that there is no regression.

[vaibhavhd]

Lgtm, thanks for this useful enhancement!