[sonic-py-swsssdk][sonic-snmpagent][sonic-utilities] Update submodules

[jleveque]

Submodule updates include the following commits. Also adding sonic-swsscommon build dependencies where necessary.

• src/sonic-py-swsssdk 1ea30d2...9d9f0c6 (1):

  [SonicV2Connector] make decode_responses=True as default, so python2 application need no code change when migrated to python3(test pr 2 #93)

• src/sonic-snmpagent 6e4a796...6b94ec3 (4):

• src/sonic-snmpagent 6e4a796...57e54d9 (3):

  Replace swsssdk.SonicV2Connector with swsscommon.SonicV2Connector (SWIG wrapper of C++ implementation) in production code (platform/cavium : Download debian packages #162)
  Interact with Redis by str instead of bytes, migrate to SonicV2Connector with decode_responses=True ([baseimage] Install package without starting service #171)
  Add a test case for LLDP_LOC_CHASSIS.lldp_loc_man_addr has only one IPv6 address (install latest intel igb driver 5.3.5.4 in the base image #167)
  [LLDP]: Update init_db to load global database config (add quagga teamd targets into docker-sonic-p4 #166)

• src/sonic-utilities acfa824...b693cf6 (11):

  Remove SKU create pytest output directory before execution of the script (Update MLNX SAI pointer #1226)
  [show][techsupport][multi-ASIC] Add support to collect tech support on multi ASIC platform ([Ingrasys] Update qsfp port SI values on S9200-64X #1193)
  [show] Fix show ip bgp sum ([Ingrasys] Update platform configuration files #1194)
  [sonic_sku_create] Move tests from sonic-utilities-tests/ folder to tests/ folder (Added psuutil support for S6100/Z9100 Dell Platforms #1222)
  Replace swsssdk.SonicV2Connector with swsscommon.SonicV2Connector (SWIG wrapper of C++ implementation) in production code ([swss]: Fix missing comma in the ports.json.j2 file #1217)
  Copy missing values from INIT_CFG to config_db as part of db_migration task (Derive varlog size based on total flash size #1209)
  [connect][clear] Support --devicename option for connect/clear line commands ([TACACS+]: Add configDB enforcer for TACACS+ #1214)
  [consutil][show] Remove root need from show line command (Added PSU util for Mellanox platforms, updated sonic-utilities submodule #1218)
  [Mellanox] SKU creator Tool ([Arista7260cx3] fancontrol configuration file #1163)
  SONiC installer - fix string formatting during image type check ([platform] Accton AS7712-32X. Update for sensors and sfputil. #1197)
  [show/fgnhg.py] Fix whitespace issues (Install more debs for python3 in sonic-slave #1211)

[lguohan]

looks like test failure, please check.

[jleveque]

@qiluo-msft: It looks like a problem in swsssdk. Can you please take a look?

22:51:24  src/sonic_ax_impl/mibs/ieee802_1ab.py:167: in reinit_data
22:51:24      self.oid_name_map = Namespace.get_sync_d_from_all_namespace(mibs.init_sync_d_interface_tables, self.db_conn)
22:51:24  src/sonic_ax_impl/mibs/__init__.py:584: in get_sync_d_from_all_namespace
22:51:24      ns_tuple = per_namespace_func(db_conn)
22:51:24  src/sonic_ax_impl/mibs/__init__.py:230: in init_sync_d_interface_tables
22:51:24      if_name_map_util, if_id_map_util = port_util.get_interface_oid_map(db_conn)
22:51:24  /usr/local/lib/python3.7/dist-packages/swsssdk/port_util.py:78: in get_interface_oid_map
22:51:24      if_id_map = {sai_oid: if_name for if_name, sai_oid in if_name_map.items()
22:51:24  /usr/local/lib/python3.7/dist-packages/swsssdk/port_util.py:80: in <dictcomp>
22:51:24      if get_index_func(if_name) is not None}
22:51:24  /usr/local/lib/python3.7/dist-packages/swsssdk/port_util.py:56: in get_index_from_str
22:51:24      match = re.match(pattern, if_name)
22:51:24  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
22:51:24  
22:51:24  pattern = '^Ethernet(\\d+)$', string = b'Ethernet0', flags = 0
22:51:24  
22:51:24      def match(pattern, string, flags=0):
22:51:24          """Try to apply the pattern at the start of the string, returning
22:51:24          a Match object, or None if no match was found."""
22:51:24  >       return _compile(pattern, flags).match(string)
22:51:24  E       TypeError: cannot use a string pattern on a bytes-like object

[qiluo-msft]

Could you also advance sonic-snmpagent/sonic-utilities? Recent code should fixed it.

[jleveque]

@madhanmellanox: When updating the sonic-utilities submodule here, tests.sku_create_test.TestSkuCreate is failing the check builds. Can you please investigate?

22:26:09  =================================== FAILURES ===================================
22:26:09  _________________________ TestSkuCreate.test_no_param __________________________
22:26:09  
22:26:09  self = <tests.sku_create_test.TestSkuCreate object at 0x7f23b0844c90>
22:26:09  
22:26:09      def test_no_param(self):
22:26:09          my_command = sku_create_script + " -f "  + sku_def_file  + " -d " + input_path
22:26:09      
22:26:09          #Test case execution without stdout
22:26:09  >       result = subprocess.check_output(my_command,stderr=subprocess.STDOUT,shell=True)
22:26:09  
22:26:09  tests/sku_create_test.py:55: 
22:26:09  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
22:26:09  
22:26:09  popenargs = ('/sonic/src/sonic-utilities/scripts/sonic_sku_create.py -f /sonic/src/sonic-utilities/tests/sku_create_input/Mellanox-SN2700-D48C8.xml -d /sonic/src/sonic-utilities/tests/sku_create_input',)
22:26:09  kwargs = {'shell': True, 'stderr': -2}
22:26:09  process = <subprocess.Popen object at 0x7f23b0844f10>
22:26:09  output = 'SKU directory: /sonic/src/sonic-utilities/tests/sku_create_input/Mellanox-SN2700-D48C8_NEW/ already exists\n Please use -r flag to remove the SKU dir first\n'
22:26:09  unused_err = None, retcode = 1
22:26:09  cmd = '/sonic/src/sonic-utilities/scripts/sonic_sku_create.py -f /sonic/src/sonic-utilities/tests/sku_create_input/Mellanox-SN2700-D48C8.xml -d /sonic/src/sonic-utilities/tests/sku_create_input'
22:26:09  
22:26:09      def check_output(*popenargs, **kwargs):
22:26:09          r"""Run command with arguments and return its output as a byte string.
22:26:09      
22:26:09          If the exit code was non-zero it raises a CalledProcessError.  The
22:26:09          CalledProcessError object will have the return code in the returncode
22:26:09          attribute and output in the output attribute.
22:26:09      
22:26:09          The arguments are the same as for the Popen constructor.  Example:
22:26:09      
22:26:09          >>> check_output(["ls", "-l", "/dev/null"])
22:26:09          'crw-rw-rw- 1 root root 1, 3 Oct 18  2007 /dev/null\n'
22:26:09      
22:26:09          The stdout argument is not allowed as it is used internally.
22:26:09          To capture standard error in the result, use stderr=STDOUT.
22:26:09      
22:26:09          >>> check_output(["/bin/sh", "-c",
22:26:09          ...               "ls -l non_existent_file ; exit 0"],
22:26:09          ...              stderr=STDOUT)
22:26:09          'ls: non_existent_file: No such file or directory\n'
22:26:09          """
22:26:09          if 'stdout' in kwargs:
22:26:09              raise ValueError('stdout argument not allowed, it will be overridden.')
22:26:09          process = Popen(stdout=PIPE, *popenargs, **kwargs)
22:26:09          output, unused_err = process.communicate()
22:26:09          retcode = process.poll()
22:26:09          if retcode:
22:26:09              cmd = kwargs.get("args")
22:26:09              if cmd is None:
22:26:09                  cmd = popenargs[0]
22:26:09  >           raise CalledProcessError(retcode, cmd, output=output)
22:26:09  E           CalledProcessError: Command '/sonic/src/sonic-utilities/scripts/sonic_sku_create.py -f /sonic/src/sonic-utilities/tests/sku_create_input/Mellanox-SN2700-D48C8.xml -d /sonic/src/sonic-utilities/tests/sku_create_input' returned non-zero exit status 1
22:26:09  
22:26:09  /usr/lib/python2.7/subprocess.py:223: CalledProcessError

[madhanmellanox]

@jleveque Looks like the workspace is not created afresh so the /sonic/src/sonic-utilities/tests/sku_create_input/Mellanox-SN2700-D48C8_NEW/ is already created during previous run of this test script and it has to be removed first. Since this is test Python script to test Sku Creator tool, I can remove this folder first, if it already exists.

[jleveque]

@madhanmellanox: I think it makes sense for the test script to first remove the new SKU directory, if it exists, before running the tests. Can you please open a PR to take care of this?

[madhanmellanox]

@jleveque I pulled a new sonic-utilities repo and trying to address the "folder already exists issue" while running sku creator tool pytest. But, when I try to run the pytest myself and verify I am getting the following error. Is it because of tests folder instead of sonic-utilities-tests folder. I have never seen this before. It is reporting sonic-py-common cannot be imported. Do you see this issue also?

madhan@arc-build-server:/builds2/madhan/skuremoveoutdir1108/sonic-utilities$ sudo pytest -v sku_create_test.py -s
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 338, in _importconftest
mod = conftestpath.pyimport()
File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line 668, in pyimport
import(modname)
File "/usr/local/lib/python2.7/dist-packages/pytest/assertion/rewrite.py", line 207, in load_module
py.builtin.exec(co, mod.dict)
File "/usr/local/lib/python2.7/dist-packages/py/builtin.py", line 221, in exec
exec2(obj, globals, locals)
File "", line 7, in exec2
File "/builds2/madhan/skuremoveoutdir1108/sonic-utilities/tests/conftest.py", line 9, in
import mock_tables.dbconnector
File "/builds2/madhan/skuremoveoutdir1108/sonic-utilities/tests/mock_tables/dbconnector.py", line 9, in
from sonic_py_common import multi_asic
ImportError: No module named sonic_py_common
ERROR: could not load /builds2/madhan/skuremoveoutdir1108/sonic-utilities/tests/conftest.py

[jleveque]

@madhanmellanox: I do not see this issue. Is your build environment set up properly? sonic_py_common is a dependency, and will get build and installed in the build environment (like the slave container) when you attempt to build the sonic-utilities target.

[jleveque]

Retest this please

[jleveque]

Retest mellanox please

[jleveque]

@qiluo-msft: vsimage build is failing. Sanity check reports snmp-subagent is exited. Can you please take a look?

'exited_critical_process': [u'snmp-subagent']

[jleveque]

Retest mellanox please