Added gbsyncd infra for multi-ASIC, multi-PHY mode

[shyam77git]

• External PHY is managed via gearbox (gbsybcd docker container) in SONiC
• Enhanced 'External PHY management' from SONiC's single-ASIC environment to multi-ASIC
• Enhanced gbsyncd docker container from single-Namespace to multi-Namespace mode
• Added gbsyncd.service.j2 on per_namespace basis
• Each namepace/ASIC now has its unique gbsyncd<ASIC#> docker container with its own Gearbox table, redis-DB

Signed-off-by: Shyam Kumar [email protected]

Why I did it

Added gbsyncd.service.j2 on per_namespace basis.
Presently (in current codebase), only one instance of gbsyncd docker container is spawned so it works for/in LC in single-ASIC mode. However, for multi-ASIC mode, where there is a unique/separate Namespace for each ASIC, gbsyncd docker container too has to uniquely spawn in each Namespace.

How I did it

Created gbsyncd.service.j2 under per_namespace.

How to verify it

Key points:

1. There is one gbsyncd per namespace uniquely identified via gbsyncd@<asic#
2. Each gbsyncd/gearbox has its own (unique) set of following config files:
   • gearbox/gbsyncd files: gearbox_config.json, context_config.json, phy_config_1.json
     Note: represents unique file for each PHY belonging to particular ASIC
   • NPU/ASIC files: port_config.ini, silicon_one.json

Validated this on Cisco's 36x400G LC which has 3 ASICs.
Ensured that unique gbsyncd (docker container) gets spawned in each ASIC Namespace.
Also checked that all required config files - gearbox related etc. - are placed under respective ASIC namespace.

root@localhost:/usr/share/sonic/device# docker ps | grep gbsyncd

effef3fd0000 docker-gbsyncd-cisco:latest "/usr/local/bin/supe…" 3 weeks ago Up 3 hours gbsyncd1

e3502dc0b08f docker-gbsyncd-cisco:latest "/usr/local/bin/supe…" 3 weeks ago Up 3 hours gbsyncd2

3191c2a51b43 docker-gbsyncd-cisco:latest "/usr/local/bin/supe…" 3 weeks ago Up 3 hours gbsyncd0
root@localhost:/usr/share/sonic/device#

root@localhost:/usr/share/sonic/device# docker exec -it gbsyncd1 bash
root@localhost:/#
root@localhost:/#
root@localhost:/# ls -ltr /usr/share/sonic/hwsku/
total 1113
-rw-r--r-- 1 root root 61 Feb 20 2017 sai.profile
-rw-r--r-- 1 root root 9237 Feb 20 2017 phy9_config_1.json
-rw-r--r-- 1 root root 9237 Feb 20 2017 phy8_config_1.json
-rw-r--r-- 1 root root 9237 Feb 20 2017 phy11_config_1.json
-rw-r--r-- 1 root root 9237 Feb 20 2017 phy10_config_1.json
-rw-r--r-- 1 root root 0 Feb 20 2017 pai.profile
-rw-r--r-- 1 root root 4233 Feb 20 2017 gearbox_config.json
-rw-r--r-- 1 root root 1634 Feb 20 2017 context_config.json
-rw-r--r-- 1 root root 8699 Jul 21 20:02 port_config.ini
-rw-r--r-- 1 root root 1052867 Jul 21 20:28 silicon_one.json
-rw-r--r-- 1 root root 9237 Jul 21 20:35 phy6_config_1.json
-rw-r--r-- 1 root root 9237 Jul 21 20:36 phy7_config_1.json
root@localhost:/#

Please refer to detail logs in this attachment:
Following is the output from Cisco's 36x400G LC loaded with gbsyncd multi-namespace changes:
UT-logs-gbsyncd-config-files-per-namespace.txt
It contains:

• Directory structure of config files on this LC which has 3 ASICs (NPUs).
  • Each ASIC has 6 (External) PHYs on it.
• Output of gearbox_config.json, context_config.json and phy12_config_1.json from ASIC2

Which release branch to backport (provide reason below if selected)

• 201811
• 201911
• 202006
• 202012
• 202106

Description for the changelog

Enhanced 'External PHY management' (gearbox) from SONiC's single-ASIC environment to multi-ASIC.

A picture of a cute animal (not mandatory but encouraged)

[abdosi]

does this condition hold true for all asic ?

[shyam77git]

Yes.
one can confirm the same from UT logs attached (which shows that required 3 gbsyncd@<asic#> got spawned for LC with 3 ASICs)

[abdosi]

@SuvarnaMeenakshi @judyjoseph

[judyjoseph]

Can you add any changes made to these files gearbox_config.json/context_config.json file in PR description as reference.

[shyam77git]

Can you add any changes made to these files gearbox_config.json/context_config.json file in PR description as reference.

@judyjoseph
Added following to the Description section:
Following is the output from Cisco's 36x400G LC loaded with gbsyncd multi-namespace changes:
Please refer to detail logs in this attachment:
UT-logs-gbsyncd-config-files-per-namespace.txt
It contains:

• Directory structure of config files on this LC which has 3 ASICs (NPUs).
  • Each ASIC has 6 (External) PHYs on it.
• Output of gearbox_config.json, context_config.json and phy12_config_1.json from ASIC2

Also added following Key notes to Description section:

1. There is one gbsyncd per namespace uniquely identified via gbsyncd@<asic#)
2. Each gbsyncd/gearbox has its own (unique) set of following config files:
   • gearbox/gbsyncd files: gearbox_config.json, context_config.json, phy_config_1.json
     Note: represents unique file for each PHY belonging to particular ASIC
   • NPU/ASIC files: port_config.ini, silicon_one.json

[judyjoseph]

One observation, may not be related to this PR, I see the gearbox/phy_config's in the hwsku directory also, are this needed ?

ls -ltr /usr/share/sonic/hwsku/
-rw-r--r-- 1 root root 0 Feb 20 2017 skip_auto_cfg_gen
-rw-r--r-- 1 root root 114201 Feb 20 2017 silicon_one.json
-rw-r--r-- 1 root root 61 Feb 20 2017 sai.profile
-rw-r--r-- 1 root root 6891 Feb 20 2017 port_config.ini
-rw-r--r-- 1 root root 9237 Feb 20 2017 phy5_config_1.json
-rw-r--r-- 1 root root 9235 Feb 20 2017 phy4_config_1.json
-rw-r--r-- 1 root root 9231 Feb 20 2017 phy3_config_1.json
-rw-r--r-- 1 root root 9203 Feb 20 2017 phy2_config_1.json
-rw-r--r-- 1 root root 9203 Feb 20 2017 phy1_config_1.json
-rw-r--r-- 1 root root 9193 Feb 20 2017 phy0_config_1.json
-rw-r--r-- 1 root root 0 Feb 20 2017 pai.profile
-rw-r--r-- 1 root root 4148 Feb 20 2017 gearbox_config.json
-rw-r--r-- 1 root root 1628 Feb 20 2017 context_config.json
drwxr-xr-x 1 root root 4096 Jul 21 21:25 2/
drwxr-xr-x 1 root root 4096 Jul 25 23:31 0/
drwxr-xr-x 1 root root 4096 Jul 25 23:35 1/