Distributed VOQ HLD by skeesara-nokia · Pull Request #639 · sonic-net/SONiC

skeesara-nokia · 2020-06-30T16:28:30Z

This is a design document proposal for Distributed VOQ from the Nokia-SONiC team

ghost · 2020-06-30T16:28:42Z

All CLA requirements met.

Updated text reflects discussion on the 07/23/2020 call on Option-2. Additional discussion still pending

…oposal.

doc/voq/voq_hld.md

Updated for changes in database object names and example configurations

doc/voq/voq_hld.md

As per the VOQ HLDs, internal networking between the linecards and supervisor is required within a chassis. Allocating 127.X/16 subnets for private communication within a chassis is a good candidate. It doesn't require any external IP allocation as well as ensure that the traffic will not leave the chassis. References: sonic-net/SONiC#622 sonic-net/SONiC#639 **- How I did it** Changed the `interfaces.j2` file to add `127.0.0.1/16` as the `lo` ip address. Then once the interface is up, the post-up command removes the `127.0.0.1/8` ip address. The order in which the netmask change is made matters for `127.0.0.1` to be reachable at all times. **- How to verify it** ``` root@sonic:~# ip address show dev lo 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/16 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever ``` Co-authored-by: Baptiste Covolato <[email protected]>

Name changes from global_app_db to chassis_app_db

global_app_db name change to chassis_app_db

doc/voq/voq_hld.md

rlhui · 2020-09-27T04:46:34Z

Are all action items discussed in chassis subgroup for this HLD closed/updated in doc if needed? E.g. "AI: Update Switch_ID can be computed using the SKU information. If the config_db is automatically created using the chassis map, Switch_ID can be auto-generated as well"?

eswaranb · 2020-09-28T05:18:58Z

Are all action items discussed in chassis subgroup for this HLD closed/updated in doc if needed? E.g. "AI: Update Switch_ID can be computed using the SKU information. If the config_db is automatically created using the chassis map, Switch_ID can be auto-generated as well"?

The switch_id is provided as part of minigraph of the linecards under the DEVICE_METADATA section. I don't think our current proposal is to auto-generate switch_ID as part of Sonic.

Reversed the order of section 2.2 and 2.3 compared to the previous version.

This PR includes implementation of System ports initialization & orchestration, Interface & Neighbor Synchronization using GLOBAL_APP_DB, Inband Interface Configuration. HLD: sonic-net/SONiC#639 Dependency PR(sonic-swss-common): sonic-net/sonic-swss-common#380 Co-authored-by: vedganes <[email protected]>

…ynchronization... (#380) This PR includes implementation of System ports initialization & orchestration, Interface & Neighbor Synchronization using GLOBAL_APP_DB, Inband Interface Configuration. HLD: sonic-net/SONiC#639 Signed-off-by: vedganes <[email protected]>

…ynchronization... (#1431) This PR includes implementation of System ports initialization & orchestration, Interface & Neighbor Synchronization using GLOBAL_APP_DB, Inband Interface Configuration. HLD: sonic-net/SONiC#639 Dependency PR's (sonic-swss-common): sonic-net/sonic-swss-common#380 (sonic-sairedis): sonic-net/sonic-sairedis#657 * [voq] Redis instance for global app db * [voq] System port init and orchestration * [voq] Router interface for system ports, global app db sync * [voq] Inband interface config * [voq] System port neighbors Kernel programming of static neigh and full mask static route for system neighbors is moved to nbrmgr since orchagent should not own kernel objects to avoid complication in warmboot reconciliation The orchagent programs only SAI object for the system neighbors and signals the completion of SAI programming by setting mac address in SYSTE_NEIGH table of STATE_DB. Nbrmgr subscribes to these state entries to program kernel entries corresponding to the system neighs * [swss]VS tests for VOQ system ports The following tests are added as part of virtual chassis tests - VOQ switch objects verification - System port object syncing with chassis app db by checking presence of system port RIFs in chassis app db in supervisor card - Creation of system port RIF record creation in ASIC_DB Following changes are done for the above tests: - SYSTEM_PORT table is added in default_config.json of line cards. These are loaded as part of config_db loading - Core and Port index mapping file is added in line card directories. - use dvs_database.py lib to connect to redis databases instead of using raw connection via swsscommon DBConnectors. - call dvs apis to get access to databases for the local redis instances. - Default handling for speed of the system port is removed in addSystemPorts() - Comment added to clarify types of rif-s synced to chassis app db - vs tests: connection to chassis app dp uses chassis app db id defined in dvs lib instead of using the id from swsscommon Using the chassis app db id from swsscommon instead of chassis app db id from dvd lib . This is to avoid python lgtm alert and test failure. Co-authored-by: vedganes <[email protected]> Co-authored-by: vganesan-nokia <[email protected]>

As per the VOQ HLDs, internal networking between the linecards and supervisor is required within a chassis. Allocating 127.X/16 subnets for private communication within a chassis is a good candidate. It doesn't require any external IP allocation as well as ensure that the traffic will not leave the chassis. References: sonic-net/SONiC#622 sonic-net/SONiC#639 **- How I did it** Changed the `interfaces.j2` file to add `127.0.0.1/16` as the `lo` ip address. Then once the interface is up, the post-up command removes the `127.0.0.1/8` ip address. The order in which the netmask change is made matters for `127.0.0.1` to be reachable at all times. **- How to verify it** ``` root@sonic:~# ip address show dev lo 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/16 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever ``` Co-authored-by: Baptiste Covolato <[email protected]>

…ynchronization... (sonic-net#1431) This PR includes implementation of System ports initialization & orchestration, Interface & Neighbor Synchronization using GLOBAL_APP_DB, Inband Interface Configuration. HLD: sonic-net/SONiC#639 Dependency PR's (sonic-swss-common): sonic-net/sonic-swss-common#380 (sonic-sairedis): sonic-net/sonic-sairedis#657 * [voq] Redis instance for global app db * [voq] System port init and orchestration * [voq] Router interface for system ports, global app db sync * [voq] Inband interface config * [voq] System port neighbors Kernel programming of static neigh and full mask static route for system neighbors is moved to nbrmgr since orchagent should not own kernel objects to avoid complication in warmboot reconciliation The orchagent programs only SAI object for the system neighbors and signals the completion of SAI programming by setting mac address in SYSTE_NEIGH table of STATE_DB. Nbrmgr subscribes to these state entries to program kernel entries corresponding to the system neighs * [swss]VS tests for VOQ system ports The following tests are added as part of virtual chassis tests - VOQ switch objects verification - System port object syncing with chassis app db by checking presence of system port RIFs in chassis app db in supervisor card - Creation of system port RIF record creation in ASIC_DB Following changes are done for the above tests: - SYSTEM_PORT table is added in default_config.json of line cards. These are loaded as part of config_db loading - Core and Port index mapping file is added in line card directories. - use dvs_database.py lib to connect to redis databases instead of using raw connection via swsscommon DBConnectors. - call dvs apis to get access to databases for the local redis instances. - Default handling for speed of the system port is removed in addSystemPorts() - Comment added to clarify types of rif-s synced to chassis app db - vs tests: connection to chassis app dp uses chassis app db id defined in dvs lib instead of using the id from swsscommon Using the chassis app db id from swsscommon instead of chassis app db id from dvd lib . This is to avoid python lgtm alert and test failure. Co-authored-by: vedganes <[email protected]> Co-authored-by: vganesan-nokia <[email protected]>

This PR includes implementation of System ports initialization & orchestration, Interface & Neighbor Synchronization using GLOBAL_APP_DB, Inband Interface Configuration. HLD: sonic-net/SONiC#639 Dependency PR(sonic-swss-common): sonic-net/sonic-swss-common#380 Co-authored-by: vedganes <[email protected]>

…ynchronization... (sonic-net#1431) This PR includes implementation of System ports initialization & orchestration, Interface & Neighbor Synchronization using GLOBAL_APP_DB, Inband Interface Configuration. HLD: sonic-net/SONiC#639 Dependency PR's (sonic-swss-common): sonic-net/sonic-swss-common#380 (sonic-sairedis): sonic-net/sonic-sairedis#657 * [voq] Redis instance for global app db * [voq] System port init and orchestration * [voq] Router interface for system ports, global app db sync * [voq] Inband interface config * [voq] System port neighbors Kernel programming of static neigh and full mask static route for system neighbors is moved to nbrmgr since orchagent should not own kernel objects to avoid complication in warmboot reconciliation The orchagent programs only SAI object for the system neighbors and signals the completion of SAI programming by setting mac address in SYSTE_NEIGH table of STATE_DB. Nbrmgr subscribes to these state entries to program kernel entries corresponding to the system neighs * [swss]VS tests for VOQ system ports The following tests are added as part of virtual chassis tests - VOQ switch objects verification - System port object syncing with chassis app db by checking presence of system port RIFs in chassis app db in supervisor card - Creation of system port RIF record creation in ASIC_DB Following changes are done for the above tests: - SYSTEM_PORT table is added in default_config.json of line cards. These are loaded as part of config_db loading - Core and Port index mapping file is added in line card directories. - use dvs_database.py lib to connect to redis databases instead of using raw connection via swsscommon DBConnectors. - call dvs apis to get access to databases for the local redis instances. - Default handling for speed of the system port is removed in addSystemPorts() - Comment added to clarify types of rif-s synced to chassis app db - vs tests: connection to chassis app dp uses chassis app db id defined in dvs lib instead of using the id from swsscommon Using the chassis app db id from swsscommon instead of chassis app db id from dvd lib . This is to avoid python lgtm alert and test failure. Co-authored-by: vedganes <[email protected]> Co-authored-by: vganesan-nokia <[email protected]>

Distributed VOQ HLD

b562401

kartik-arista mentioned this pull request Jul 17, 2020

IP forwarding in a Distributed VOQ Architecture HLD #650

Closed

skeesara-nokia and others added 19 commits July 22, 2020 09:18

Updated figure for reference architecture

30768a0

Update the HLD with review comments.

b6e8843

Fix the 3.4 callflow diagram with correct link.

3e7ac86

Updated text around switch_id assignment

8aac26e

fixed range for switch_id

facb7ee

Update the HLD with LAG callflow.

f4816f4

Update the HLD based on 7/22/2020 review comments.

0f047d0

Update the voq reference architecture picture.

b3c1d75

Updated text in ection 2.6.2

aff099f

Updated text reflects discussion on the 07/23/2020 call on Option-2. Additional discussion still pending

Minor edit - table of contents

35c1285

Minor edits to section 2.6.2

a68360d

Update VOQ_DB connection paramaters in database_config.json.j2

2c04e13

Fix run_server based on linecard to no.

a9b221e

Update HLD with review comments, and option 2b from Kartik/Eswaran pr…

c6e8af7

…oposal.

Update the picture location and LCPU to linecard CPU.

25e7e21

Update INBAND configuration

238b63b

cpu-to-cpu flows using recycle port

fc42661

recycle port cpu-to-network flows

dec4b1a

Updated figures to add recycling port

42fa7d6

kartik-arista reviewed Aug 6, 2020

View reviewed changes

doc/voq/voq_hld.md Outdated Show resolved Hide resolved

kartik-arista reviewed Aug 6, 2020

View reviewed changes

doc/voq/voq_hld.md Outdated Show resolved Hide resolved

kartik-arista reviewed Aug 6, 2020

View reviewed changes

doc/voq/voq_hld.md Outdated Show resolved Hide resolved

kartik-arista reviewed Aug 6, 2020

View reviewed changes

doc/voq/voq_hld.md Outdated Show resolved Hide resolved

kartik-arista reviewed Aug 6, 2020

View reviewed changes

doc/voq/voq_hld.md Show resolved Hide resolved

kartik-arista reviewed Aug 6, 2020

View reviewed changes

doc/voq/voq_hld.md Outdated Show resolved Hide resolved

Update voq_hld.md

94a6d16

Updated for changes in database object names and example configurations

rck-innovium reviewed Aug 18, 2020

View reviewed changes

doc/voq/voq_hld.md Outdated Show resolved Hide resolved

Updated revision history

3ecfb13

Staphylo mentioned this pull request Sep 10, 2020

Change the loopback interface netmask from /8 to /16 sonic-net/sonic-buildimage#5353

Merged

3 tasks

skeesara-nokia and others added 6 commits September 17, 2020 09:30

Update voq_hld.md

f6feb37

Update voq_hld.md

5c7e3ae

Add files via upload

af3b9a9

Name changes from global_app_db to chassis_app_db

Add files via upload

6a19978

global_app_db name change to chassis_app_db

Add files via upload

8b18881

global_app_db name change to chassis_app_db

Add files via upload

621b92f

global_app_db name change to chassis_app_db

rlhui reviewed Sep 27, 2020

View reviewed changes

doc/voq/voq_hld.md Outdated Show resolved Hide resolved

skeesara-nokia added 4 commits September 30, 2020 09:45

Update voq_hld.md

1ceffb0

Reversed the order of section 2.2 and 2.3 compared to the previous version.

Fixed table of contents

4e67d3a

Added notes on testing based on review comments

ffa35de

Update voq_hld.md

1aab4c1

rlhui approved these changes Oct 19, 2020

View reviewed changes

rlhui merged commit 44919ea into sonic-net:master Oct 19, 2020

anshuv-mfst added the chassis label Jan 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Distributed VOQ HLD#639

Distributed VOQ HLD#639
rlhui merged 71 commits intosonic-net:masterfrom
skeesara-nokia:voq

skeesara-nokia commented Jun 30, 2020

Uh oh!

ghost commented Jun 30, 2020 •

edited by ghost

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rlhui commented Sep 27, 2020

Uh oh!

eswaranb commented Sep 28, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Conversation

skeesara-nokia commented Jun 30, 2020

Uh oh!

ghost commented Jun 30, 2020 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rlhui commented Sep 27, 2020

Uh oh!

eswaranb commented Sep 28, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

ghost commented Jun 30, 2020 •

edited by ghost

Loading