Skip to content

[dash]: Refactor DASH orch by protobuf format#2722

Merged
lguohan merged 21 commits intosonic-net:dashfrom
Pterosaur:pb_orch
Jul 26, 2023
Merged

[dash]: Refactor DASH orch by protobuf format#2722
lguohan merged 21 commits intosonic-net:dashfrom
Pterosaur:pb_orch

Conversation

@Pterosaur
Copy link
Contributor

@Pterosaur Pterosaur commented Apr 4, 2023 •
edited
Loading

What I did
Refactor orchagent input from text schema to protobuf message. The protobuf message definition is at: https://github.com/sonic-net/sonic-dash-api/tree/master

Why I did it
The original text schema format will take

  1. too much memory in Redis
  2. the performance is not efficient due to large of format conversions.
  3. Weak coding restrictions aren't easy to develop and maintain

So, the binary formation, protobuf, will be needed for better utilization.

How I verified it
Check Azp and test dash vstest in locally.

 sudo pytest --dvsname=vs test_dash_vnet.py
============================================================= test session starts ==============================================================
platform linux -- Python 3.8.10, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/zegan/workspace/sonic-swss/tests
plugins: flaky-3.7.0
collected 8 items

test_dash_vnet.py ........                                                                                                               [100%]

=============================================================== warnings summary ===============================================================
sudo pytest --dvsname=vs test_dash_acl.py
============================================================= test session starts ==============================================================
platform linux -- Python 3.8.10, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/zegan/workspace/sonic-swss/tests
plugins: flaky-3.7.0
collected 6 items

test_dash_acl.py ......                                                                                                                  [100%]

=============================================================== warnings summary ===============================================================

Details if related

@Pterosaur
Dash ACL to PB
52c0f6d 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Add pbworker
b92ef1d 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Refactor
4605433 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
dashorch
4458fbc 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
dashrouteorch
587cd28 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Dash vnet orch
b43c9b9 
Signed-off-by: Ze Gan <ganze718@gmail.com>
Pterosaur and others added 8 commits June 18, 2023 10:48
@Pterosaur
Merge branch 'dash' into pb_orch
dd70c31
@Pterosaur
Adapter new dash branch
0ceeade 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Adapter latest dash-api
2645272 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Fix test_dash_vnet.py
5c4e6b3 
Signed-off-by: zegan <zegan@microsoft.com>
@Pterosaur
Refactor dash_api
4a825f3 
Signed-off-by: zegan <zegan@microsoft.com>
@Pterosaur
Fix task name
f238ea4 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Fix
99ff8d6 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Dash acl test
83eea2e 
Signed-off-by: zegan <zegan@microsoft.com>
@prsunny prsunny requested a review from prabhataravind July 6, 2023 00:00
Pterosaur added 2 commits July 6, 2023 22:19
@Pterosaur
Merge branch 'dash' into pb_orch
d2a4c1b
@Pterosaur
Add dash-api in readme
53712d0 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur Pterosaur marked this pull request as ready for review July 6, 2023 15:29
@Pterosaur Pterosaur requested a review from prsunny as a code owner July 6, 2023 15:29
@Pterosaur
Add common libs dependencies
9e87992 
Signed-off-by: Ze Gan <ganze718@gmail.com>
prabhataravind
prabhataravind reviewed Jul 24, 2023
View reviewed changes
prabhataravind
prabhataravind reviewed Jul 24, 2023
View reviewed changes
orchagent/dash/dashorch.h Outdated
dash::eni::Eni metadata;
};

struct QosEntry
Copy link
Contributor

@prabhataravind prabhataravind Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can possibly remove this too right?

Copy link
Contributor Author

@Pterosaur Pterosaur Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it should be removed because it wasn't used anywhere. It's my careless mistake

prabhataravind
prabhataravind reviewed Jul 24, 2023
View reviewed changes
prabhataravind
prabhataravind reviewed Jul 24, 2023
View reviewed changes
Copy link
Contributor

@prabhataravind prabhataravind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in dash* orchs in general look good to me. Could you please validate your changes with the configurations in these issues? If you could send the sairedis records with these configurations applied, that would be great.

https://github.com/nvidia-sonic/sonic-buildimage/issues/183
https://github.com/nvidia-sonic/sonic-buildimage/issues/178

Pterosaur added 4 commits July 24, 2023 14:56
@Pterosaur
Remove useless QosEntry
32e2894 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Remove useleess code
9570d57 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Add dash dependencies
32b0a87 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Add test for multiple vnet map
d35cb86 
Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur
Copy link
Contributor Author

Pterosaur commented Jul 24, 2023

The changes in dash* orchs in general look good to me. Could you please validate your changes with the configurations in these issues? If you could send the sairedis records with these configurations applied, that would be great.

nvidia-sonic/sonic-buildimage#183 nvidia-sonic/sonic-buildimage#178

Check them and add testcases for them.

@Pterosaur Pterosaur requested a review from prabhataravind July 24, 2023 12:24
prabhataravind
prabhataravind approved these changes Jul 24, 2023
View reviewed changes
Copy link
Contributor

@prabhataravind prabhataravind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@lguohan lguohan merged commit daa1fc7 into sonic-net:dash Jul 26, 2023
theasianpianist pushed a commit to theasianpianist/sonic-swss that referenced this pull request Jul 27, 2023
@Pterosaur @theasianpianist
[dash]: Refactor DASH orch by protobuf format (sonic-net#2722)
6d47230 
What I did
Refactor orchagent input from text schema to protobuf message. The protobuf message definition is at: https://github.com/sonic-net/sonic-dash-api/tree/master

Why I did it
The original text schema format will take

too much memory in Redis
the performance is not efficient due to large of format conversions.
Weak coding restrictions aren't easy to develop and maintain
So, the binary formation, protobuf, will be needed for better utilization.

How I verified it
Check Azp and test dash vstest in locally.

Signed-off-by: Ze Gan <ganze718@gmail.com>
theasianpianist pushed a commit to theasianpianist/sonic-swss that referenced this pull request Jul 27, 2023
@Pterosaur @theasianpianist
[dash]: Refactor DASH orch by protobuf format (sonic-net#2722)
c87c86e 
What I did
Refactor orchagent input from text schema to protobuf message. The protobuf message definition is at: https://github.com/sonic-net/sonic-dash-api/tree/master

Why I did it
The original text schema format will take

too much memory in Redis
the performance is not efficient due to large of format conversions.
Weak coding restrictions aren't easy to develop and maintain
So, the binary formation, protobuf, will be needed for better utilization.

How I verified it
Check Azp and test dash vstest in locally.

Signed-off-by: Ze Gan <ganze718@gmail.com>
This was referenced Aug 19, 2023
Merged
Closed
Pterosaur added a commit to Pterosaur/sonic-sairedis that referenced this pull request Aug 20, 2023
@Pterosaur
[azp] Attempt to fix swss missing libs
288f44c 
Swss containter needs dash_api to properly build, based on sonic-net/sonic-swss#2722, attempting to add missing libs

Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur Pterosaur mentioned this pull request Aug 20, 2023
Merged
Pterosaur added a commit to Pterosaur/sonic-sairedis that referenced this pull request Aug 20, 2023
@Pterosaur
[azp] Attempt to fix swss missing libs
6ebcd4f 
Swss containter needs dash_api to properly build, based on sonic-net/sonic-swss#2722, attempting to add missing libs

Signed-off-by: Ze Gan <ganze718@gmail.com>
kcudnik pushed a commit to sonic-net/sonic-sairedis that referenced this pull request Aug 23, 2023
@Pterosaur
[azp] Attempt to fix swss missing libs (#1277)
44cd8c4 
Swss containter needs dash_api to properly build, based on sonic-net/sonic-swss#2722, attempting to add missing libs

Signed-off-by: Ze Gan <ganze718@gmail.com>
Pterosaur added a commit to Pterosaur/sonic-swss-common that referenced this pull request Aug 23, 2023
@Pterosaur
[azp] Fix swss missing libs
3529626 
Swss containter needs dash_api to properly build, based on sonic-net/sonic-swss#2722, attempting to add missing libs

Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur Pterosaur mentioned this pull request Aug 23, 2023
Merged
Pterosaur added a commit to sonic-net/sonic-swss-common that referenced this pull request Aug 23, 2023
@Pterosaur
[azp] Fix swss missing libs (#815)
f8a23c3 
Swss containter needs dash_api to properly build, based on sonic-net/sonic-swss#2722, attempting to add missing libs

Signed-off-by: Ze Gan <ganze718@gmail.com>
SviatoslavBoichuk pushed a commit to SviatoslavBoichuk/sonic-swss-common that referenced this pull request Sep 7, 2023
@Pterosaur
[azp] Fix swss missing libs (sonic-net#815)
046e0f7 
Swss containter needs dash_api to properly build, based on sonic-net/sonic-swss#2722, attempting to add missing libs

Signed-off-by: Ze Gan <ganze718@gmail.com>
@zjswhhh zjswhhh mentioned this pull request Mar 27, 2025
Merged
Janetxxx pushed a commit to Janetxxx/sonic-swss that referenced this pull request Nov 10, 2025
@Pterosaur @theasianpianist
[dash]: Refactor DASH orch by protobuf format (sonic-net#2722)
2b32059 
What I did
Refactor orchagent input from text schema to protobuf message. The protobuf message definition is at: https://github.com/sonic-net/sonic-dash-api/tree/master

Why I did it
The original text schema format will take

too much memory in Redis
the performance is not efficient due to large of format conversions.
Weak coding restrictions aren't easy to develop and maintain
So, the binary formation, protobuf, will be needed for better utilization.

How I verified it
Check Azp and test dash vstest in locally.

Signed-off-by: Ze Gan <ganze718@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@prabhataravind prabhataravind prabhataravind approved these changes

@prsunny prsunny Awaiting requested review from prsunny prsunny is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Pterosaur @prabhataravind @lguohan