[HLD] bmp for monitoring SONiC BGP info by FengPan-Frank · Pull Request #1621 · sonic-net/SONiC

FengPan-Frank · 2024-02-22T14:41:51Z

[HLD] bmp for monitoring SONiC BGP info

Description	Repo	PR link
Yang model definition	sonic-buildimage	sonic-net/sonic-buildimage#18530
Add BMP_STATE_DB dedicated instance	sonic-buildimage	sonic-net/sonic-buildimage#19016
Add BMP_STATE_DB support in swss-common	sonic-swss-common	sonic-net/sonic-swss-common#879
bmpcfgd service implementation	sonic-buildimage	sonic-net/sonic-buildimage#18940
config cli implementation	sonic-utility	sonic-net/sonic-utilities#3286
show cli implementation	sonic-utility	sonic-net/sonic-utilities#3289
openbmpd service implementation	sonic-bmp	sonic-net/sonic-bmp#7
Add sonic-bmp submodule	sonic-buildimage	sonic-net/sonic-buildimage#20780
Build bmp container into sonic-buildimage	sonic-buildimage	sonic-net/sonic-buildimage#18946
Add bmp dynamic feature switch in frr side config.	sonic-buildimage	sonic-net/sonic-buildimage#20895

qiluo-msft · 2024-02-29T01:43:52Z

doc/bmp/bmp.md

Current [BGP] container has been forked from [FRR] -> Current BGP container uses a implementation (sonic-frr) forked from FRR #Closed

zhangyanzhao · 2024-03-12T15:05:18Z

community review recording https://zoom.us/rec/share/t7kdlkY0uSCaf9fVifFaeS2rY8RSlWrQlIcO7CaZsaE0HU1bnp8-orK5IN81SXup.QdSvaujeNaOagbml

venkatmahalingam · 2024-03-12T15:42:08Z

doc/bmp/bmp.md

https://github.com/sonic-net/SONiC/pull/1621/files#diff-0649e404e5d4496d9d6af8f6b9411cb483e3a6d376a927d8660baf476465d7b3R138
"SONiC to stream events occurring in SONiC switch (e.g BGP flap, process not running, ...) in real time via gNMI subscription model.
The events are defined using versioned YANG schema."

We have the above YANG model for streaming the BGP states today and looks like BGP_NEIGHBOR_TABLE supports extensive state information, hope we'll migrate to use one streaming method i.e BMP or events infra.

praveenraja1 · 2024-03-12T15:52:48Z

doc/bmp/bmp.md

Do we new docker for running the "openBMPd" daemon?

I understand that the plan is to have,

config "feature" command to enable/disable this.

Limit CPU resource usage of the new docker .

Even if we leverage the existing frr container to run openBMPd, there are alternate ways to achieve the above, for instance #1 we could have a "config bmp <enable|disable>" which could internally start/stop the openBMPd process. And for #2 we could use command such as "nice/cpulimit" to set process usage limits.

From what i have seen adding a new container (base docker, with no overhead app) increases RAM usage by 20 to 30 MB, so just want to ensure if we really need a new container here.

Thanks, it's indeed feasible as either way, and some tradeoff just exists as consideration.
The advantage of using dedicated container is that we can avoid packaging bmp feature onto some slim images if DUT resource is small and expensive relatively.

rathnasabapathyv · 2024-03-12T18:08:09Z

doc/bmp/bmp.md

Since FRR does support bmp enable/disable as a cmd-line arg (/usr/lib/frr/bgpd -A 127.0.0.1 -M bmp), to support per table level enable/disable as shown below, it's better to discuss this with FRR-community and introduce table level enable/disable support via FRR CLI knob, so that zebra/bgpd doesn't have to spend any cycle in sending change notifications for those tables to BMP ; Otherwise zebra/bgpd will always send notifications to BMP for the disabled tables and bmp ignores it later based on this config-db.

Please discuss with FRR community on this, as discussed in the HLD review community meeting (March-12-2024)

Sure, currently FRR just post data to BMP following standard BGP protocol, thus neighbor table will be sent in BGP_MSG_OPEN message (#https://datatracker.ietf.org/doc/html/rfc4271#page-13) and all other tables including rib/ls_state... will be sent in BGP_MSG_UPDATE. A better option is to update in FRR side and controlled by the config/cli together, so that no extra data is transmitted between FRR and BMP. Will follow up with FRR community.

Thanks. Let's discuss with FRR community and enhance FRR capability to achieve this, as part of bringing in this feature to community-sonic.

venkatmahalingam · 2024-03-12T18:36:17Z

doc/bmp/bmp.md

VTYSH show commands provide extensive dumps today for the debuggability, trying to understand the new information that we support part of openBMP.

On-change subscription for BGP data is clearly an use-case, I would like to see more filtering options (e.g RIB-in or our per neighbor) in the HLD so that we can limit the number entries going into the DB.

Are you asking smaller granularity control like per neighbor? instead of per table?.

I'm asking per neighbor level filtering at least and if possible, per table level filtering is good as well.

Sure, we will support table level filtering first, later could append neighbor level filtering if specific requirement asks.

venkatmahalingam · 2024-03-12T18:45:16Z

doc/bmp/bmp.md

One option is to connect to the local collector and other option is to connect to the remote collector, if we have an use-case not to store into the redis DB but to only stream into the remote collector, are we planning to give such an option?

Correct, that will require dedicated BMP service being deployed and maintained, for now we're not planning of this.

venkatmahalingam · 2024-03-12T18:50:04Z

doc/bmp/bmp.md

Please consider pushing this state information into STATE_DB or new DB instance if that makes our life easy.

venkatmahalingam · 2024-03-12T18:53:16Z

doc/bmp/bmp.md

There was an exercise done to optimize the DB memory usage for storing the entries in DASH WG. Since we may store lot of RIB information into the DB, may be it's ideal to use protobuf format rather than text format for DB entries (refer the below link for more information).
https://github.com/sonic-net/DASH/blob/main/documentation/general/data/AppDBMemoryEstimation.xlsx

Yes, will make some comparison for this, using pb should be feasible just the data will not be readable if we use redis-cli directly, seems now most of sonic database are text? I'm not sure @qiluo-msft may you comment?

yes, most database contents are in text format, we may need to check the possibility of memory optimization as we may push lot of RIB information into the DB.

…nto fenpan_bmp

Why I did it Add yang model support for feature sonic-net/SONiC#1621 Microsoft ADO (number only):27332639

forrestchu · 2024-06-28T02:06:01Z

doc/bmp/images/architecture_diagram.png

I have some concerns about this architecture design:
1）Why not BMP send route to external OpenBMP servers, which may keep more detailed BGP history based on Kafka.
2) Can BMP module in FRR write route to Redis directly? It is more straightforward.

1> That's will require us to maintain dedicated OpenBMP service, which is not in current scope, maybe later we can do that, if so it will be configurable simply from bmp perspective as well.
2> I think it will introduce unnecessary complexities into FRR, if like that FRR can persist Rib info directly instead of BMP, right? We choose BMP path now because BMP was natively supported in FRR in ASYNC mode, and not much extra cost added into FRR.

OK, Got the concept behind your design. Thanks~

#### Why I did it sonic-net/SONiC#1621, need dedicated daemon to monitor config_db and manage openbmpd state. #### How I did it Use swss-common lib to monitor config_db change.

…nto fenpan_bmp

FengPan-Frank requested review from qiluo-msft and zhangyanzhao February 22, 2024 14:42

FengPan-Frank force-pushed the fenpan_bmp branch 3 times, most recently from 6427fc8 to 23d402e Compare February 27, 2024 12:34

FengPan-Frank requested a review from lguohan February 27, 2024 12:34

FengPan-Frank force-pushed the fenpan_bmp branch 5 times, most recently from 7edbcda to b81dfcf Compare February 29, 2024 01:26

qiluo-msft reviewed Feb 29, 2024

View reviewed changes

FengPan-Frank force-pushed the fenpan_bmp branch 2 times, most recently from 1673a55 to d075afa Compare February 29, 2024 12:33

FengPan-Frank force-pushed the fenpan_bmp branch 5 times, most recently from 6f2ae6c to 68234b1 Compare March 12, 2024 14:41

venkatmahalingam reviewed Mar 12, 2024

View reviewed changes

praveenraja1 reviewed Mar 12, 2024

View reviewed changes

rathnasabapathyv reviewed Mar 12, 2024

View reviewed changes

venkatmahalingam reviewed Mar 12, 2024

View reviewed changes

FengPan-Frank force-pushed the fenpan_bmp branch 2 times, most recently from edbe16c to 16f287f Compare March 14, 2024 14:57

[HLD] bmp for monitoring SONiC BGP info

ac631e3

FengPan-Frank force-pushed the fenpan_bmp branch from 16f287f to ac631e3 Compare March 14, 2024 15:02

This was referenced Mar 27, 2024

Add yang models for bmp service sonic-net/sonic-buildimage#18482

Closed

Add support for redis population. sonic-net/sonic-bmp#4

Closed

Add yang models for bmp service. sonic-net/sonic-buildimage#18530

Merged

FengPan-Frank added 3 commits April 4, 2024 09:30

[HLD] bmp for monitoring SONiC BGP info

99fcc30

Merge branch 'sonic-net:master' into fenpan_bmp

d28e315

Merge branch 'fenpan_bmp' of https://github.com/FengPan-Frank/SONiC i…

50cb559

…nto fenpan_bmp

lguohan pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Apr 23, 2024

[yang]: Add yang models for bmp service. (#18530)

b37c26f

Why I did it Add yang model support for feature sonic-net/SONiC#1621 Microsoft ADO (number only):27332639

FengPan-Frank mentioned this pull request May 11, 2024

Add bmpcfgd for monitoring config_db state change. sonic-net/sonic-buildimage#18940

Merged

11 tasks

forrestchu reviewed Jun 28, 2024

View reviewed changes

qiluo-msft assigned FengPan-Frank Jul 9, 2024

FengPan-Frank added 2 commits December 4, 2024 03:20

Merge branch 'fenpan_bmp' of https://github.com/FengPan-Frank/SONiC i…

4b33274

…nto fenpan_bmp

Merge branch 'fenpan_bmp' of https://github.com/FengPan-Frank/SONiC i…

bb04424

…nto fenpan_bmp

qiluo-msft approved these changes Dec 7, 2024

View reviewed changes

qiluo-msft merged commit 2ca5103 into sonic-net:master Dec 7, 2024

vivekverma-arista mentioned this pull request Feb 26, 2025

Add BMP to init config in default disabled state sonic-net/sonic-buildimage#21863

Closed

12 tasks

Conversation

FengPan-Frank commented Feb 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qiluo-msft Feb 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhangyanzhao commented Mar 12, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

venkatmahalingam Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FengPan-Frank Mar 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

FengPan-Frank commented Feb 22, 2024 •

edited

Loading

qiluo-msft Feb 29, 2024 •

edited

Loading

venkatmahalingam Mar 12, 2024 •

edited

Loading

FengPan-Frank Mar 13, 2024 •

edited

Loading