Skip to content

[SNMP]: Add dependency in SNMP service to start after syncd#8860

Closed
SuvarnaMeenakshi wants to merge 1 commit intosonic-net:201811from
SuvarnaMeenakshi:snmp_service
Closed

[SNMP]: Add dependency in SNMP service to start after syncd#8860
SuvarnaMeenakshi wants to merge 1 commit intosonic-net:201811from
SuvarnaMeenakshi:snmp_service

Conversation

@SuvarnaMeenakshi
Copy link
Contributor

service. SNMP tries to access COUNTERS_DB, to avoid errors
while accessing DB, SNMP should start after syncd.

Signed-off-by: Suvarna Meenakshi sumeenak@microsoft.com

Why I did it

When SNMP Interface QoS counters is queried around the same time as syncd/swss restarts, the below error is seen:

ERR snmp#snmp-subagent [ax_interface] ERROR: MIBUpdater.start() caught an unexpected exception during update_data()#012Traceback (most recent call last):#012  File "/usr/local/lib/python3.6/dist-packages/ax_interface/mib.py", line 40, in start#012    self.reinit_data()#012  File "/usr/local/lib/python3.6/dist-packages/sonic_ax_impl/mibs/vendor/cisco/ciscoPfcExtMIB.py", line 40, in reinit_data#012    self.update_data()#012  File "/usr/local/lib/python3.6/dist-packages/sonic_ax_impl/mibs/vendor/cisco/ciscoPfcExtMIB.py", line 49, in update_data#012    for sai_id in self.if_id_map}#012  File "/usr/local/lib/python3.6/dist-packages/sonic_ax_impl/mibs/vendor/cisco/ciscoPfcExtMIB.py", line 49, in <dictcomp>#012    for sai_id in self.if_id_map}#012  File "/usr/local/lib/python3.6/dist-packages/swsssdk/interface.py", line 38, in wrapped#012    ret_data = f(inst, db_name, *args, **kwargs)#012  File "/usr/local/lib/python3.6/dist-packages/swsssdk/interface.py", line 324, in get_all#012    raise UnavailableDataError(message, _hash)#012swsssdk.exceptions.UnavailableDataError: Key 'b'COUNTERS:oid:0x1000000000029'' unavailable in database 'COUNTERS_DB'
ERR syncd#syncd: _brcm_sai_cosq_stat_get:1340 cosq stat get failed with error Invalid parameter (0xfffffffc).
DEBUG syncd#syncd: brcm_sai_get_queue_stats:884 cosq stat get failed with error -5 for port 57 qid 10
NOTICE syncd#syncd: :- setQueueCounterList: Queue oid:0x3902150000000b does not has supported counters
INFO snmp#supervisord: snmp-subagent ERROR:ax_interface:MIBUpdater.start() caught an unexpected exception during update_data()
INFO snmp#supervisord: snmp-subagent Traceback (most recent call last):
INFO snmp#supervisord: snmp-subagent   File "/usr/local/lib/python3.6/dist-packages/ax_interface/mib.py", line 40, in start
INFO snmp#supervisord: snmp-subagent     self.reinit_data()
INFO snmp#supervisord: snmp-subagent   File "/usr/local/lib/python3.6/dist-packages/sonic_ax_impl/mibs/vendor/cisco/ciscoPfcExtMIB.py", line 40, in reinit_data
INFO snmp#supervisord: snmp-subagent     self.update_data()
INFO snmp#supervisord: snmp-subagent   File "/usr/local/lib/python3.6/dist-packages/sonic_ax_impl/mibs/vendor/cisco/ciscoPfcExtMIB.py", line 49, in update_data
INFO snmp#supervisord: snmp-subagent     for sai_id in self.if_id_map}
INFO snmp#supervisord: snmp-subagent   File "/usr/local/lib/python3.6/dist-packages/sonic_ax_impl/mibs/vendor/cisco/ciscoPfcExtMIB.py", line 49, in <dictcomp>
INFO snmp#supervisord: snmp-subagent     for sai_id in self.if_id_map}
INFO snmp#supervisord: snmp-subagent   File "/usr/local/lib/python3.6/dist-packages/swsssdk/interface.py", line 38, in wrapped
INFO snmp#supervisord: snmp-subagent     ret_data = f(inst, db_name, *args, **kwargs)
INFO snmp#supervisord: snmp-subagent   File "/usr/local/lib/python3.6/dist-packages/swsssdk/interface.py", line 324, in get_all
INFO snmp#supervisord: snmp-subagent     raise UnavailableDataError(message, _hash)
INFO snmp#supervisord: snmp-subagent swsssdk.exceptions.UnavailableDataError: Key 'b'COUNTERS:oid:0x1000000000029'' unavailable in database 'COUNTERS_DB'

This error comes only once after service restarts.
This happens because upon config reload, the order of services that comes up is:
swss
snmp
syncd
When SNMP comes up before syncd and SNMP query for QoS counters is done in parallel, then the above error is seen.

How I did it

Add dependency so that SNMP comes up after syncd comes up, which will ensure that COUNTERS_DB is populated when SNMP tries to access it.

How to verify it

Run the image after this fix, do a config reload, perform continuous query of qos counters using:
snmpwalk -v2c -c 1.3.6.1.4.1.9.9.580.1.5.5
continuous query should not generate any error in syslog.
snmp service should come up after swss and syncd services.
Checked "sudo systemctl status to check start time of each service.

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

service. SNMP tries to access COUNTERS_DB, to avoid errors
while accessing DB, SNMP should start after syncd.

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@qiluo-msft
Copy link
Collaborator

It's kind of partial backport of #3656.
I prefer not to merge until we have a stronger reason, not just a snmpagent error message.

vivekrnv added a commit to vivekrnv/sonic-buildimage that referenced this pull request Oct 21, 2021
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
@SuvarnaMeenakshi
Copy link
Contributor Author

Closing this as we do not have a strong reason for this fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants