Upgrading hsflowd version from 2.0.45 to 2.0.51-26 to address for [sf…

[Gokulnath-Raja]

Why I did it

Below error log is seen sometimes when services startup.
ERR sflow#hsflowd: device Loopback0 Get SIOCGIFFLAGS failed : No such device

How I did it

latest changes to hsflowd will avoid this race (e.g. the release tagged as v2.0.51-26). The new start up sequence waits first for the redis connections to come up, then waits for the redis to indicate SYSTEM_READY|SYSTEM_STATE Status=Up, and only then does it read the interfaces from Linux.

How to verify it

Post upgrade to latest version of hsflowd, manually verifying the sflow functionalities such as counter/flow samples.
UT is inprogres.

Which release branch to backport (provide reason below if selected)

• 201811
• 201911
• 202006
• 202012
• 202106
• 202111
• 202205
• 202211

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

[dgsudharsan]

@Gokulnath-Raja Please ensure all the existing sonic-mgmt Sflow tests pass with this upgrade.

[dgsudharsan]

@Gokulnath-Raja Please also sync to latest master to resolve the conflicts.

[Gokulnath-Raja]

@Gokulnath-Raja Please also sync to latest master to resolve the conflicts.

Done

[Gokulnath-Raja]

@Gokulnath-Raja Please ensure all the existing sonic-mgmt Sflow tests pass with this upgrade.

Validated sFlow functionality attached sample logs
hsflow_update.txt

[dgsudharsan]

@Gokulnath-Raja Thanks. Did we qualify with sonic-mgmt sflow test suite? It would cover different test scenarios and not basic sanity test.

[Gokulnath-Raja]

@Gokulnath-Raja Thanks. Did we qualify with sonic-mgmt sflow test suite? It would cover different test scenarios and not basic sanity test.

Could not find any sonic-mgmt sflow test suite... We have done following testing,

1. Configured sFlow on multiple interface, checked the counter samples on different rate
2. Configured sFlow on multiple interface, checked the flow samples on different rate with different traffic pattern custom payload of L2 and L3 traffic
3. Flapped interface
4. Enable and disable globally to check the sFlow functionality

[dgsudharsan]

@Gokulnath-Raja Thanks. Did we qualify with sonic-mgmt sflow test suite? It would cover different test scenarios and not basic sanity test.

Could not find any sonic-mgmt sflow test suite... We have done following testing,

Hi @Gokulnath-Raja We do have sonic-mgmt test in sflow https://github.com/sonic-net/sonic-mgmt/blob/master/tests/sflow/test_sflow.py
I feel its important to run it before doing an upgrade. Can you please make sure we run this test with the new hsflow version?

[dgsudharsan]

Hi @Gokulnath-Raja Any update here?

[Gokulnath-Raja]

Hi @Gokulnath-Raja Any update here?

We are working on building test topology and test server to run the mgmt test_sflow, will keep you updated.

[mohanapriya-meganathan]

Hi @Gokulnath-Raja Any update here?

We are working on building test topology and test server to run the mgmt test_sflow, will keep you updated.

@dgsudharsan , We ran the script and found one failure. When we are deleting the agent-id, hsflowd is referring for the eth0 interface and expecting samples in that interface. (This is with the updated hsflowd , When you start hsflowd with no explicit agent-id configured, it will run an "election" to decide which IP address to adopt.). We are debugging further.

[dgsudharsan]

@Gokulnath-Raja Please compile with flag ENABLE_SFLOW_DROPMON=y and make sure compilation passes. Recent sflow upgrade to 2.0.45-1 broke the build and I have notified the PR owners about it.

[Gokulnath-Raja]

@Gokulnath-Raja Please compile with flag ENABLE_SFLOW_DROPMON=y and make sure compilation passes. Recent sflow upgrade to 2.0.45-1 broke the build and I have notified the PR owners about it.

@dgsudharsan getting below dropman patch issues
Cloning into 'host-sflow'...
/sonic/src/sflow/hsflowd/host-sflow /sonic/src/sflow/hsflowd
Switched to a new branch 'sflow'
error: patch failed: src/Linux/Makefile:34
error: src/Linux/Makefile: patch does not apply
error: patch failed: src/Linux/hsflowd.c:1877
error: src/Linux/hsflowd.c: patch does not apply
stg import: Diff does not apply cleanly
make[1]: *** [Makefile:9: /sonic/target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb] Error 2
make[1]: Leaving directory '/sonic/src/sflow/hsflowd'
[ FAIL LOG END ] [ target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb ]
make: *** [slave.mk:685: target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb] Error 1
make[1]: *** [Makefile.work:597: target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb] Error 2

[Gokulnath-Raja]

@Gokulnath-Raja Please compile with flag ENABLE_SFLOW_DROPMON=y and make sure compilation passes. Recent sflow upgrade to 2.0.45-1 broke the build and I have notified the PR owners about it.

@dgsudharsan getting below dropman patch issues Cloning into 'host-sflow'... /sonic/src/sflow/hsflowd/host-sflow /sonic/src/sflow/hsflowd Switched to a new branch 'sflow' error: patch failed: src/Linux/Makefile:34 error: src/Linux/Makefile: patch does not apply error: patch failed: src/Linux/hsflowd.c:1877 error: src/Linux/hsflowd.c: patch does not apply stg import: Diff does not apply cleanly make[1]: *** [Makefile:9: /sonic/target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb] Error 2 make[1]: Leaving directory '/sonic/src/sflow/hsflowd' [ FAIL LOG END ] [ target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb ] make: *** [slave.mk:685: target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb] Error 1 make[1]: *** [Makefile.work:597: target/debs/bullseye/hsflowd_2.0.51-26_amd64.deb] Error 2

@dgsudharsan did following changes and successfully compile with flag ENABLE_SFLOW_DROPMON=y
please let me know should i update the merge request with below changes

diff --git a/src/sflow/hsflowd/patch/dropmon/0001-sflow-enabled-drop-monitor-support-for-SONiC.patch b/src/sflow/hsflowd/patch/dropmon/0001-sflow-enabled-drop-monitor-support-for-SONiC.patch
index 7a3561fd7..06573ce62 100644
--- a/src/sflow/hsflowd/patch/dropmon/0001-sflow-enabled-drop-monitor-support-for-SONiC.patch
+++ b/src/sflow/hsflowd/patch/dropmon/0001-sflow-enabled-drop-monitor-support-for-SONiC.patch
@@ -5,39 +5,25 @@ Subject: [PATCH] [sflow] enabled drop monitor support for SONiC

 Signed-off-by: Vadym Hlushko <[email protected]>
 ---
- src/Linux/Makefile                   | 2 +-
  src/Linux/hsflowd.c                  | 8 ++++++++
  src/Linux/scripts/hsflowd.conf.sonic | 2 ++
- 3 files changed, 11 insertions(+), 1 deletion(-)
+ 2 files changed, 10 insertions(+)

-diff --git a/src/Linux/Makefile b/src/Linux/Makefile
-index 8128cf2..cf538e7 100644
---- a/src/Linux/Makefile
-+++ b/src/Linux/Makefile
-@@ -34,7 +34,7 @@ FEATURES_DENT= DENT PSAMPLE SYSTEMD DROPMON
- FEATURES_EOS= EAPI
- FEATURES_OS10= OS10 DBUS SYSTEMD
- FEATURES_OPX= OPX DBUS SYSTEMD
--FEATURES_SONIC= SONIC PSAMPLE DOCKER
-+FEATURES_SONIC= SONIC PSAMPLE DOCKER DROPMON
- FEATURES_XEN= XEN OVS
- FEATURES_HOST= NFLOG PCAP TCP DOCKER KVM OVS DBUS SYSTEMD
-
 diff --git a/src/Linux/hsflowd.c b/src/Linux/hsflowd.c
 index 5d94e79..25031d1 100644
 --- a/src/Linux/hsflowd.c
 +++ b/src/Linux/hsflowd.c
-@@ -1877,6 +1877,14 @@ extern "C" {
+@@ -2074,6 +2074,14 @@ extern "C" {
      sp->psample.ingress = YES;
-     sp->psample.egress = NO;
-     sp->psample.group = 1;
+     sp->psample.egress = YES;
+     sp->psample.group = 1; // Ingress PSAMPLE group number. Expects egress on (group+1).
 +    // drop-monitor support
 +    myLog(LOG_INFO, "drop-monitor support for SONiC");
 +    sp->dropmon.dropmon = YES;
-+    sp->dropmon.group = 1;
 +    sp->dropmon.start = NO;
 +    sp->dropmon.limit = 1000;
 +    sp->dropmon.sw = NO;
++    sp->dropmon.rn = NO;
 +    sp->dropmon.hw = YES;
  #endif /* HSP_LOAD_SONIC */

@@ -51,7 +37,7 @@ index e675730..fb52a54 100644
    # sonic {} loaded automatically
    # psample {} loaded automatically
 +  # packet drop monitoring:
-+  dropmon { group=1 start=on limit=1000 sw=off hw=on }
++  dropmon { start=on limit=1000 sw=off hw=on }
    # ====== detect new interfaces ======
    refreshAdaptors=60
    # ======  Agent IP selection ======

[dgsudharsan]

@Gokulnath-Raja Please check this PR #15948. Once its merged, you can resync your PR and verify if it compiles fine.

[dgsudharsan]

@Gokulnath-Raja Can you please resync your changes to latest master as #15948 is merged?

[dgsudharsan]

@Gokulnath-Raja Can you please update the status of sonic-mgmt sflow tests?

[Gokulnath-Raja]

We ran the script and found one failure. When we are deleting the agent-id, hsflowd is referring for the eth0 interface and expecting samples in that interface. (This is with the updated hsflowd , When you start hsflowd with no explicit agent-id configured, it will run an "election" to decide which IP address to adopt.).

@dgsudharsan we found the RCA and fixed the same... Running tests now will update the PR

[Gokulnath-Raja]

We ran the script and found one failure. When we are deleting the agent-id, hsflowd is referring for the eth0 interface and expecting samples in that interface. (This is with the updated hsflowd , When you start hsflowd with no explicit agent-id configured, it will run an "election" to decide which IP address to adopt.).

@dgsudharsan we found the RCA and fixed the same... Running tests now will update the PR

@dgsudharsan we have raise PR for sonic-mgmt test for sflow sonic-net/sonic-mgmt#9766 kindly approve the same

[mssonicbld]

@Gokulnath-Raja PR conflicts with 202211 branch

[mssonicbld]

Cherry-pick PR to 202305: #16615