Skip to content

Add custom serdes attributes for saiport.h#2156

Merged
tjchadaga merged 1 commit intoopencomputeproject:masterfrom
longhuan-cisco:serdes_attrs_ext
Apr 23, 2025
Merged

Add custom serdes attributes for saiport.h#2156
tjchadaga merged 1 commit intoopencomputeproject:masterfrom
longhuan-cisco:serdes_attrs_ext

Conversation

@longhuan-cisco
Copy link
Copy Markdown
Contributor

@longhuan-cisco longhuan-cisco commented Mar 18, 2025

Problem description:

Today, saiport.h and SONiC only supports 10+ NPU SI serdes attributes, but there are below situations for certain vendors/platforms:

  1. The total number of attributes can be way more than 10+ for certain platforms/vendors (e.g. 40+ attrs), and the number can keep growing even more for future platforms and new ASIC board versions for existing platforms.
  2. Attributes can be vendor/platform specific. Different platforms may have totally different sets of attributes (or only very few attributes are common). And SONiC’s orchagent (which is platform agnostic and independently built) should not include a platform-specific extension header file containing those attributes enums.
  3. Attributes can be proprietary, which means original names for those attributes cannot be exposed to public, they cannot be upstreamed as it is, unless as dummy/alias attributes.

Proposal:
(credit to @kcudnik for his idea #2156 (comment))

Add one single attribute, whose value is a string that includes a JSON object with key-value pairs. Each pair's key represents the serdes attribute name, and each pair's value is a list of values for every serdes lane. This allows vendor-specific serdes attributes to be forwarded in a JSON string without the sender needing to know the details. The sender simply passes along the data, vendor-defined rules (e.g. for SONiC, it's media_settings.json on platform side) determine which key-value pairs to include, and the vendor SDK interprets the JSON accordingly.

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Mar 18, 2025

custom attributes are per vendor, and are not meant to be public, please remove those, and each vendor should implement its own custom attributes, please reffer to this PR: #2122 and take a look at its documentation

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

cc @prgeor

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

custom attributes are per vendor, and are not meant to be public, please remove those, and each vendor should implement its own custom attributes, please reffer to this PR: #2122 and take a look at its documentation

Thanks for the pointer.
PR #2122 seems to introduce a way to support custom attributes by allowing vendors to put vendor specific headers (containing custom attributes) under SAI/custom folder.

But the situation here is: The SAI requestor side, such as SONiC’s orchagent (which is public, platform agnostic and independently built as separate docker), might not be able to include vendor specific headers.
We would need a way to support custom attributes without putting vendor specific headers at SAI/custom folder of the requestor side (e.g. SONiC orchagent). Thus raised the proposal of adding public dummy/placeholder serdes attributes in saiport.h to allow vendors to use them for whatever attributes if needed.

If today's enum range SAI_PORT_SERDES_ATTR_CUSTOM_RANGE_START is reserved only for the custom attributes supposed to be included in SAI/custom/<headers>. Then, I'm wondering if we could add another enum range (e.g. SAI_PORT_SERDES_ATTR_PUBLIC_CUSTOM_RANGE_START = 0x20000000) dedicated for those dummy/placeholder serdes attributes?

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Mar 19, 2025

your assumption is false, if OA will require custom headers then it will need to include them as metadata generated from headers is per vendor, so if 1st vendor will have custom attribute let say int, and 2nd vendor the same attribute as oid list, you will get crash right away.

if OA want to use custom headers, it need to include them per vendor, and it needs to be compiled per vendor.

Adding unnamed custom attributes make no sense, and it's unsolvable. that's why #2122 is introduced

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

your assumption is false, if OA will require custom headers then it will need to include them as metadata generated from headers is per vendor, so if 1st vendor will have custom attribute let say int, and 2nd vendor the same attribute as oid list, you will get crash right away.

if OA want to use custom headers, it need to include them per vendor, and it needs to be compiled per vendor.

Adding unnamed custom attributes make no sense, and it's unsolvable. that's why #2122 is introduced

In this particular case here (sai_port_serdes_attr_t), the values of SAI_PORT_SERDES_ATTR are always of type sai_s32_list_t where the count is number lanes in a port and the list specifies list of values to be applied to each lane, regardless of vendors. (I updated the diff by adding comments indicating the value type as sai_s32_list_t for all the new attributes, and making the enum range starting from SAI_PORT_SERDES_ATTR_PUBLIC_CUSTOM_RANGE_START = 0x20000000 to avoid conflicts with #2122)

And OA is platform agnostic, it would not be able to include any vendor specific headers.

Any particular reasons to not be able to introduce dummy serdes attributes with the same value type? Is this a deviation from SAI design?

Add @prgeor for comments also

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Mar 20, 2025

Custom range is for each vendor to use, and they are not compatoble, for public interface attributes should be added to main range. Lepsze discuss this on todays sai community meeting

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

Custom range is for each vendor to use, and they are not compatoble,

I understood the SAI_PORT_SERDES_ATTR_CUSTOM_RANGE_START = 0x10000000 is reserved for each vendor to use. That's why I updated my diff to put all the public custom serdes attributers in a separate range starting from SAI_PORT_SERDES_ATTR_PUBLIC_CUSTOM_RANGE_START = 0x20000000.

And, unlike most other types of attributes, a port serdes attribute is always inherently meant to be a list of integers (each integer represents a value for each serdes lane of the specific port), regardless of whether it's common or vendor-specific. That's why I predefined the type of all public custom serdes attributers as sai_s32_list_t, where I didn't see compatibility problem.

On the other hand, in today's SONiC, the serdes attributes are applied to SAI/SDK by OA on a per-platform basis: each platform/vendor defines its own attribute abbreviation->attribute value in /usr/share/sonic/device/<platform>/media_settings.json, and only the ones defined in this file will get applied.
With these proposed public custom serdes attributers, platform/vendor would be able to easily support large number of newly added proprietary vendor-specific serdes attributes by just updating media_settings.json (plus corresponding SDK support), without having to expose the original real names of the attributes to public, without having to change SONiC and SAI.

for public interface attributes should be added to main range. Lepsze discuss this on todays sai community meeting

(Not sure if I'm understanding correctly) Or were you suggesting to put all the public custom serdes attributers at main range right after SAI_PORT_SERDES_ATTR_RX_PRECODING?

Copy link
Copy Markdown
Collaborator

@kcudnik kcudnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets have a live discussion since i dont see point of introducing arbitrary incompatible attributes for all vendors

@tjchadaga
Copy link
Copy Markdown
Collaborator

@kcudnik, @longhuan-cisco - do you want to discuss this in this week's SAI community meeting?

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

sure, let's discuss, could you please send me ([email protected]) the link and include @prgeor also.

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Mar 27, 2025

my proposal:

    /**
     * @brief custom vendor intersface
     *
     * vendor have 3 attributes: custom_1, custom_2, custom_3
     *
     * we carete json/ini/or vendor format:
     *
     * std::string s1 =
     * {
     * "custom_1": "value1"
     * }
     *
     *
     *
     * sai_attribute_t a;
     * a.id = SAI_PORT_SERDES_ATTR_CUSTOM_INTERFACE;
     * a.value.s32list.count = s1.lenght();
     * a.value.s32list.data = s1.data()
     * portserdesapi->set_port_serteds_attribute(&a); // this will set custom_1
     *
     *
     *
     * std::string s2 =
     * {
     * "custom_1": "value1" // string, int, bool
     * "custom_2": "value2"
     * }
     *
     * foo=bar // old ini file format
     *
     *
     * a.value.s32list.count = s2.lenght();
     * a.value.s32list.data = s2.data()
     * portserdesapi->set_port_serteds_attribute(&a); // this will set custom_1 and custom_@
     *
     * @type sai_s32_list_t
     * @flags CREATE_AND_SET
     * @default internal
     */
    SAI_PORT_SERDES_ATTR_CUSTOM_INTERFACE, // etc

@tjchadaga tjchadaga added the reviewed PR is discussed in SAI Meeting label Mar 27, 2025
@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

Thanks @kcudnik for joining today's discussion and the proposal.
I updated the diff as what you proposed, and modified the PR description to reflect the latest.

BTW. Since it's your idea, just feel free to close mine, in case if you feel more comfortable to raise your own PR,

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Mar 29, 2025

Thanks @kcudnik for joining today's discussion and the proposal. I updated the diff as what you proposed, and modified the PR description to reflect the latest.

BTW. Since it's your idea, just feel free to close mine, in case if you feel more comfortable to raise your own PR,

im ok for you to drive this change

@lguohan
Copy link
Copy Markdown
Collaborator

lguohan commented Apr 8, 2025

@prgeor , can you take a look?

@tjchadaga tjchadaga requested review from prgeor and removed request for harshitgulati18 April 8, 2025 18:31
Copy link
Copy Markdown
Collaborator

@prgeor prgeor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we hold the PR merge. @longhuan-cisco will sync with you offline.

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

Hi @prgeor,
Just a gentle follow-up — we went over it last week; just checking in on the merge when you get a chance.

@tjchadaga
Copy link
Copy Markdown
Collaborator

/azp run

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

Committed change to:

  1. Fix Doxygen complaint by adjusting the indentations and encapsulating JSON example with verbatim notation.
  2. Fix the JSON example to exactly follow the sai_json_t grammar defined in SAI_Proposal_Generic_Extensions.md, by having each attribute as separate object under the array of the mandatory top-level key attributes
    "attributes": [
    {
    "virtual_ip": {
    "sai_metadata": {
    "sai_attr_value_type": "SAI_ATTR_VALUE_TYPE_IP_ADDRESS"
    },
    "value": "10.1.1.1"
    }
    },
    {
    "nexthop_list": {
    "sai_metadata": {
    "sai_attr_value_type": "SAI_ATTR_VALUE_TYPE_OBJECT_LIST"
    },
    "value": [ "0x1500000000001fa", "0x1500000000001fb" ]
    }
    }
    ]

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

I saw CodeQL run got canceled due to below two errors, which doesn't seem to be related to my change

1. This is a scheduled Ubuntu 20.04 retirement. Ubuntu 20.04 LTS runner will be removed on 2025-04-15. For more details, 
2. GitHub Actions has encountered an internal error when running your job.

Wondering if there's a way to re-run it?

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Apr 16, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines
Copy link
Copy Markdown

Commenter does not have sufficient privileges for PR 2156 in repo opencomputeproject/SAI

@prgeor
Copy link
Copy Markdown
Collaborator

prgeor commented Apr 16, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@tjchadaga
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

Build complained about ERROR: FATAL: sai_attribute_value_t members were removed on commit a2eb597, NOT ALLOWED! when running ./checkstructs.sh:

2025-04-16T22:21:16.3547431Z ./checkstructs.sh
2025-04-16T22:21:16.3615789Z ancestry graph
2025-04-16T22:21:16.3642179Z * ae57c65 Merge 1dd1356b832da0ec560f034479d87bfdaf29199b into a56c38ed33aab0f81ec1c1e9f716ff722dc94f96
2025-04-16T22:21:16.3642786Z * a56c38e Add a RIF attribute to specify if the corresponding My MAC entry should not be created.. (#2021)
2025-04-16T22:21:16.3649214Z git rev list from 5f75d99 to HEAD
2025-04-16T22:21:16.4561010Z git checkout work tree commits: 5f75d99 91ef792 97c8ed2 ad20062 ab47430 2602b2a d24cbb2 775254c 0aba236 61a1a29 b4358c7 1443ba8 b2be076 dc64a51 5ff3424 6168071 7157445 2a4b1ff 15cf7a2 ee6f49b 1784b9f 59e4d57 7df1c94 750cafa a3d6844 93d80b9 f9b1ddd 626b749 b8fbbcc e128c5b 1f2bca1 f4695e9 28751b9 8245ee6 1906d59 f3a8b04 46e8391 a2eb597 3ff76ad 9a343d7 d158311 2d7bb43 290c475 172c6f2 074e2e8 a56c38e 65d304c 406c0c7 1dd1356 ae57c65
2025-04-16T22:21:19.1663835Z loaded history from structs.5f75d99.history
2025-04-16T22:21:19.1664085Z processing commit 5f75d99
2025-04-16T22:21:19.1664227Z processing commit 91ef792
2025-04-16T22:21:19.1664418Z processing commit 97c8ed2
2025-04-16T22:21:19.1664575Z processing commit ad20062
2025-04-16T22:21:19.1664767Z updating sai_port_oper_status_notification_t since member count changed from 3 to 2
2025-04-16T22:21:19.1664961Z processing commit ab47430
2025-04-16T22:21:19.1665114Z processing commit 2602b2a
2025-04-16T22:21:19.1665426Z updating sai_tam_api_t since member count changed from 48 to 50
2025-04-16T22:21:19.1665597Z processing commit d24cbb2
2025-04-16T22:21:19.1665745Z processing commit 775254c
2025-04-16T22:21:19.1665904Z processing commit 0aba236
2025-04-16T22:21:19.1666046Z processing commit 61a1a29
2025-04-16T22:21:19.1666173Z processing commit b4358c7
2025-04-16T22:21:19.1666319Z processing commit 1443ba8
2025-04-16T22:21:19.1666557Z processing commit b2be076
2025-04-16T22:21:19.1666703Z processing commit dc64a51
2025-04-16T22:21:19.1666844Z processing commit 5ff3424
2025-04-16T22:21:19.1666989Z processing commit 6168071
2025-04-16T22:21:19.1667482Z processing commit 7157445
2025-04-16T22:21:19.1667634Z processing commit 2a4b1ff
2025-04-16T22:21:19.1667779Z processing commit 15cf7a2
2025-04-16T22:21:19.1667925Z processing commit ee6f49b
2025-04-16T22:21:19.1668069Z processing commit 1784b9f
2025-04-16T22:21:19.1668236Z updating sai_port_oper_status_notification_t since member count changed from 2 to 3
2025-04-16T22:21:19.1668414Z processing commit 59e4d57
2025-04-16T22:21:19.1668560Z processing commit 7df1c94
2025-04-16T22:21:19.1668701Z processing commit 750cafa
2025-04-16T22:21:19.1668850Z processing commit a3d6844
2025-04-16T22:21:19.1668977Z processing commit 93d80b9
2025-04-16T22:21:19.1669118Z processing commit f9b1ddd
2025-04-16T22:21:19.1669264Z processing commit 626b749
2025-04-16T22:21:19.1669571Z processing commit b8fbbcc
2025-04-16T22:21:19.1669709Z processing commit e128c5b
2025-04-16T22:21:19.1669844Z processing commit 1f2bca1
2025-04-16T22:21:19.1669964Z processing commit f4695e9
2025-04-16T22:21:19.1670102Z processing commit 28751b9
2025-04-16T22:21:19.1670242Z processing commit 8245ee6
2025-04-16T22:21:19.1670405Z updating sai_attribute_value_t since member count changed from 58 to 59
2025-04-16T22:21:19.1670564Z processing commit 1906d59
2025-04-16T22:21:19.1670684Z processing commit f3a8b04
2025-04-16T22:21:19.1670821Z processing commit 46e8391
2025-04-16T22:21:19.1670955Z processing commit a2eb597
2025-04-16T22:21:19.1671296Z ERROR: FATAL: sai_attribute_value_t members were removed on commit a2eb597, NOT ALLOWED!
2025-04-16T22:21:19.1678288Z make: *** [Makefile:90: all] Error 1

But none of my commits (including a2eb597) touched sai_attribute_value_t . And I ran ./checkstructs.sh locally, it went through fine, didn't hit the complaint:

 $ ./checkstructs.sh
ancestry graph
* 1dd1356 (HEAD -> serdes_attrs_ext, origin/serdes_attrs_ext) Remove unnecessary commas
* 406c0c7 Fix parse.pl complaints for number of spaces after *
* 65d304c Fix doxygen warnings
* 172c6f2 Change flags back to CREATE_AND_SET for flexibility
* 2d7bb43 Change flags to CREATE_ONLY to align with others
* 9a343d7 Update the type as sai_json_t
* 3ff76ad Use one single JSON-string attr to represent a collection of serdes attrs
* a2eb597 Change RANGE_START to avoid conflict with the range reserved for DASH
* b8fbbcc Add comment for each newly added attr
* 626b749 Add custom attributes 1-100
* 93d80b9 (origin/master, origin/HEAD, master) [TAM] SAI TAM enhancements (#2141)
git rev list from 5f75d99 to HEAD
git checkout work tree commits: 5f75d99 91ef792 97c8ed2 ad20062 ab47430 2602b2a d24cbb2 775254c 0aba236 61a1a29 b4358c7 1443ba8 b2be076 dc64a51 5ff3424 6168071 7157445 2a4b1ff 15cf7a2 ee6f49b 1784b9f 59e4d57 7df1c94 750cafa a3d6844 93d80b9 626b749 b8fbbcc a2eb597 3ff76ad 9a343d7 2d7bb43 172c6f2 65d304c 406c0c7 1dd1356
loaded history from structs.5f75d99.history
processing commit 5f75d99
processing commit 91ef792
processing commit 97c8ed2
processing commit ad20062
updating sai_port_oper_status_notification_t since member count changed from 3 to 2
processing commit ab47430
processing commit 2602b2a
updating sai_tam_api_t since member count changed from 48 to 50
processing commit d24cbb2
processing commit 775254c
processing commit 0aba236
processing commit 61a1a29
processing commit b4358c7
processing commit 1443ba8
processing commit b2be076
processing commit dc64a51
processing commit 5ff3424
processing commit 6168071
processing commit 7157445
processing commit 2a4b1ff
processing commit 15cf7a2
processing commit ee6f49b
processing commit 1784b9f
updating sai_port_oper_status_notification_t since member count changed from 2 to 3
processing commit 59e4d57
processing commit 7df1c94
processing commit 750cafa
processing commit a3d6844
processing commit 93d80b9
processing commit 626b749
processing commit b8fbbcc
processing commit a2eb597
processing commit 3ff76ad
processing commit 9a343d7
processing commit 2d7bb43
processing commit 172c6f2
processing commit 65d304c
processing commit 406c0c7
processing commit 1dd1356

Wondering if I shall do a force-push anyway to squash everything into one commit to give a try.

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Apr 18, 2025

yes, you need to force push squashed

@tjchadaga
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

I did force-push squashed.
It's weird, build still complained about below error, even though my commit simply adds one enum, it doesn't remove anything.

ERROR: FATAL: sai_attribute_value_t members were removed on commit 03a9a17, NOT ALLOWED!
make: *** [Makefile:90: all] Error 1

@kcudnik
Copy link
Copy Markdown
Collaborator

kcudnik commented Apr 22, 2025

you will need to squash your commits and rebase to master and then force pushed, since it seems like you are behind master

@longhuan-cisco
Copy link
Copy Markdown
Contributor Author

you will need to squash your commits and rebase to master and then force pushed, since it seems like you are behind master

Thanks, rebased to latest master

@tjchadaga
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@tjchadaga tjchadaga merged commit a08584b into opencomputeproject:master Apr 23, 2025
2 of 3 checks passed
JaiOCP pushed a commit to JaiOCP/SAI that referenced this pull request May 2, 2025
tjchadaga pushed a commit that referenced this pull request Oct 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewed PR is discussed in SAI Meeting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants