Skip to content

Support read default value from yang model and read profile from PROFILE_DB.#625

Closed
liuh-80 wants to merge 37 commits intosonic-net:masterfrom
liuh-80:dev/liuh/default_value_decorator
Closed

Support read default value from yang model and read profile from PROFILE_DB.#625
liuh-80 wants to merge 37 commits intosonic-net:masterfrom
liuh-80:dev/liuh/default_value_decorator

Conversation

@liuh-80
Copy link
Contributor

@liuh-80 liuh-80 commented May 30, 2022

Why I did it

Add default value and profile support to swss-swss-common.

How I did it

Add DefaultValueProvider and ProfileProvider.

How to verify it

Add new UT.
Pass all existing UT and E2E test.

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111

Description for the changelog

Add default value and profile support to swss-swss-common.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

class YangDefaultValueTableDecorator : public Table {
public:
YangDefaultValueTableDecorator(Table* table);
~YangDefaultValueTableDecorator() override;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be marked as virtual?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed, and the 'override' declarator actually will makesure this is a virtual method.

@liuh-80
Copy link
Contributor Author

liuh-80 commented Jun 7, 2022

My move dump() to TableEntryEnumerable interface, we can revert all Table class change and remove YangDefaultValueTableDecorator

@qiluo-msft
Copy link
Contributor

class TableEntryEnumerable {

You can either decorate TableEntryEnumerable or Table directly. The latter is easier?


Refers to: common/table.h:165 in f452126. [](commit_id = f452126, deletion_comment = False)


namespace swss {

class YangDefaultValuePoppableDecorator : public TableEntryPoppable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TableEntryPoppable

TableEntryPoppable subclasses include ConsumerTable/ConsumerStateTable/SubscribeStateTable. However, we only need decorating SubscribeStateTable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From design point of view, there are several requirements:

  1. SubscribeStateTable
  2. Yang Default + SubscribeStateTable (from bottom layer to up layer)
  3. Table + SubscribeStateTable (for example, image default profile with user override)
  4. Yang Default + Table + SubscribeStateTable

The outside interface is SubscribeStateTable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is: if we only decorate the TableEntryPoppable interface, then we can only decorate 2 method:

/* Pop an action (set or del) on the table */
void pop(KeyOpFieldsValuesTuple &kco, const std::string &prefix = EMPTY_PREFIX) override;

/* Get multiple pop elements */
void pops(std::deque<KeyOpFieldsValuesTuple> &vkco, const std::string &prefix = EMPTY_PREFIX) override;

But if we decorate SubscriberStateTable , then there are too much methods we need to decorate and the are not related with default value, for example these methods:
uint64_t readData() override;
bool hasData() override;
bool hasCachedData() override;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improved by create OverlayConfigTable and OverlaySubscriberStateTable.

@qiluo-msft
Copy link
Contributor

class TableEntryEnumerable {

From design point of view, there are several requirements:

  1. Table
  2. Yang Default + Table (from bottom layer to up layer)
  3. Table + Table (for example, image default profile with user override)
  4. Yang Default + Table + Table

The outside interface is Table


In reply to: 1152829037


Refers to: common/table.h:165 in f452126. [](commit_id = f452126, deletion_comment = False)


const DBConnector* getDbConnector() const;

std::string getNamespace() const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

namespace is not property of consumertablebase, its property of dbconnector, and since we have method getDbConnectort() should you use that and then call getnamespace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I will remove this method and improve code to use getDbConnector.

ret.setdefault(table_name, {})[self.deserialize_key(row)] = entry
return ret

class LayeredConfigDBConnectorDecorator(ConfigDBConnector):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIll improve this decorator with re-write ConfigDBConnector API with reflection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, improve this decorator according to isabel's poc code.

config_db_connector.ori_get_config = config_db_connector.get_config
def _append_static_config(table, key, data):
serialized_key = config_db_connector.serialize_key(key)
ns = config_db_connector.getNamespace()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getNamespace

You can directly use config_db_connector.getDbConnector().newConnector(). No need to create new method getNamespace().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config_db_connector.getDbConnector().newConnector() will create a new connector to config DB, but in our case we need connect to a different database: static_config_db.

I will check if we can save static config to config DB with a different table name, for example static_*, then we can reuse current connector.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, now use getDbConnector() for better performance.

common/schema.h Outdated
#define CHASSIS_APP_DB 12
#define CHASSIS_STATE_DB 13
#define APPL_STATE_DB 14
#define STATIC_CONFIG_DB 15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

STATIC_CONFIG_DB

Will you remove this line? It is not used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

}
}

std::map<std::string, std::string> DefaultValueProvider::GetDefaultValues(const std::string &table, const std::string &row)
Copy link
Contributor

@qiluo-msft qiluo-msft Jul 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

row

std::map<std::string, std::string> DefaultValueProvider::GetDefaultValues(const std::string &table, const std::string &row)


row is not well known terminology. Do you mean key in the context of table? #Closed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

{
case 0:
{
tableInfoPtr = new TableInfoDict(fieldInfoMapping);
Copy link
Contributor

@qiluo-msft qiluo-msft Jul 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new

tableInfoPtr = new TableInfoDict(fieldInfoMapping);


Possible memory leak? Better to use smart pointer. #Closed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

kcudnik
kcudnik previously approved these changes Jul 22, 2022
@liuh-80 liuh-80 force-pushed the dev/liuh/default_value_decorator branch 2 times, most recently from 0db1a97 to 1a00040 Compare July 28, 2022 08:57
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Aug 23, 2022

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: liuh-80 / name: Hua Liu (bb1108b)

@liuh-80 liuh-80 force-pushed the dev/liuh/default_value_decorator branch from 42da895 to bb1108b Compare August 25, 2022 04:21
@liuh-80 liuh-80 changed the title [POC] POC code for show default value with yang model v2 [POC] Support read default value from yang model and read profile from PROFILE_DB. Aug 25, 2022
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Aug 25, 2022

CLA Signed

The committers listed above are authorized under a signed CLA.

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 1, 2022

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 8, 2022

/azp run Azure.sonic-swss-common

@azure-pipelines
Copy link

Azure Pipelines failed to run 1 pipeline(s).

@liuh-80 liuh-80 changed the title [POC] Support read default value from yang model and read profile from PROFILE_DB. Support read default value from yang model and read profile from PROFILE_DB. Sep 8, 2022
@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 8, 2022

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liuh-80 liuh-80 marked this pull request as ready for review September 8, 2022 08:23
@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 8, 2022

Will validate all E2E test with this PR: sonic-net/sonic-buildimage#10575

@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 8, 2022

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 8, 2022

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 9, 2022

Will split this PR to 3-4 small PRs for code review.

@liuh-80
Copy link
Contributor Author

liuh-80 commented Sep 9, 2022

Close this PR, will create splited PRs later.

@liuh-80 liuh-80 closed this Sep 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants