[minigraph] Consume golden_config_db.json while loading minigraph

[wen587]

What I did

This PR is for supporting consume golden_config_db.json while loading minigraph. User can put the golden_config_db.json with minigraph file to override configDB after reload minigraph.
The golden_config_db.json looks just like a config_db.json and it will be placed on /etc/sonic/golden_config_db.json.
Step1: Load minigraph to configDB
Step2: If golden_config_db.json exists and contains some Table configuration, that Table in configDB will be overriden by the config in golden_config_db.json. Other config that not included in golden_config_db.json will keep the same as minigraph.
For example, assume below ACL_TABLE is loaded from minigraph:
config from minigraph:

        "ACL_TABLE": {
            "DATAACL": {
                "policy_desc": "DATAACL",
                "ports": [
                    "Ethernet4"
                ],
                "stage": "ingress",
                "type": "L3"
            },
            "NTP_ACL": {
                "policy_desc": "NTP_ACL",
                "services": [
                    "NTP"
                ],
                "stage": "ingress",
                "type": "CTRLPLANE"
            }
        },

The golden_config_db.json also contains ACL_TABLE as below.
config from golden_config_db.json

"ACL_TABLE": {
            "EVERFLOWV6": {
                "policy_desc": "EVERFLOWV6",
                "ports": [
                    "Ethernet12"
                ],
                "stage": "ingress",
                "type": "MIRRORV6"
            }
        },

During load_minigraph, the final config will be the same with golden_config_db.json because ACL_TABLE will be overriden by golden_config_db.json.
configDB ACL_TABLE final state:

"ACL_TABLE": {
            "EVERFLOWV6": {
                "policy_desc": "EVERFLOWV6",
                "ports": [
                    "Ethernet12"
                ],
                "stage": "ingress",
                "type": "MIRRORV6"
            }
        },

How I did it

Add code config override-config-table command to let it consume golden_config_db.json

How to verify it

Add UT tests and run.

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

admin@vlab-01:~$ sudo config override-config-table -h
Usage: config override-config-table [OPTIONS] INPUT_CONFIG_DB

  Override current configDB with input config.

Options:
  --dry_run TEXT  Dry run, writes config to the given file
  -h, -?, --help  Show this message and exit.

[qiluo-msft]

load_module_from_source

We can move some useful functions to an official python module (ref: src/sonic-config-engine/setup.py) so it will be easy to reuse. This ask is in low priority, you can check this after all others resolved. #Pending

[qiluo-msft]

LGTM. Let's wait a while for more eyes.

[prsunny]

Can you please provide more details in the description?

For e.g, what is overridden from the golden_config? Is any config taken from the minigraph in this case?

[wen587]

Can you please provide more details in the description?

For e.g, what is overridden from the golden_config? Is any config taken from the minigraph in this case?

Updated in description.
In short, configDB will be overriden by all golden_config_db.json 's configurations. The other Table configurations will keep the same as minigraph generated config if not listed in Golden Config,

[judyjoseph]

@wen587, @qiluo-msft, to support multi-asic devices, to start with could we also call this API override_config_by() per namespace as well? This will help with the pilot of this feature MACSEC_PROFILE table which is needed in the redis_db per namespace as well -- macsec/swss dockers run per asic/namespace.

Later when we design the table contents in the golden_config_db.json file, we could name the table based on the asic/namespace name eg: ASIC0_PORT_TABLE, ASIC1_PORT_TABLE etc - so that the respective tables will be pushed to the right redis DB in namespaces.