[cfggen] Extend Template Argument to Support Batch Mode#4941
[cfggen] Extend Template Argument to Support Batch Mode#4941tahmed-dev merged 7 commits intosonic-net:masterfrom
Conversation
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
from
6d760d0 to
ba40fd9
Compare
|
This pull request introduces 1 alert when merging ba40fd93dcd033fd210a638faaa23f97c1fa3d4d into 81a9d5a - view on LGTM.com new alerts:
|
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
from
ba40fd9 to
ba85069
Compare
|
This pull request introduces 1 alert when merging b6d5f14c80a61cbf4a40fb822bfb63595b114174 into 81a9d5a - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 7af2d6ad23781590eb8543248a207f2ece860742 into 81a9d5a - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 18406037008789890eb2a2911c1743cd9944ddb0 into 81a9d5a - view on LGTM.com new alerts:
|
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
from
1840603 to
5051ef0
Compare
|
This pull request introduces 1 alert when merging 5051ef050c248d08e17356afd8817e5476953321 into 81a9d5a - view on LGTM.com new alerts:
|
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
from
5051ef0 to
cf16bc6
Compare
|
This pull request introduces 1 alert when merging cf16bc6d4c5216374d7a2ed4329b1fab868b4dfb into 3154e04 - view on LGTM.com new alerts:
|
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
from
cf16bc6 to
b47aaeb
Compare
|
This pull request introduces 1 alert when merging b47aaeb46b7096fffbb7e54506624e4fecfdc10b into bf45e11 - view on LGTM.com new alerts:
|
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
2 times, most recently
from
9136b91 to
9a20d3b
Compare
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
from
9a20d3b to
a27c408
Compare
pavel-shirshov
left a comment
pavel-shirshov
left a comment
There was a problem hiding this comment.
looks good for me
Calls to cfggen take considerable time. With batch mode, ww will have the ability to limit number of calls from service to one call only. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
as an option following the template name and separate by comma
tahmed-dev
force-pushed
the
taahme/cfggen-add-batch-mode
branch
from
a27c408 to
f668924
Compare
| for template_file, dest_file in args.template: | ||
| template = env.get_template(os.path.basename(template_file)) | ||
| template_data = template.render(sorted_data) | ||
| if dest_file == "config-db": |
There was a problem hiding this comment.
"config-db" [](start = 28, length = 11)
This is magic string for a special mode. What if user want to write to a file called 'config-db' ?
There was a problem hiding this comment.
The user can write and rename the file later. Currently when we save config-db.json, we simply redirect stdout to the file.
There was a problem hiding this comment.
maybe we can use -t template-2.j2,[config-db] to indicate this is config-db, not a config-db file. if user really generate a config-db file, then use -t template-2.j2,\[config-db\]
There was a problem hiding this comment.
the user can use this syntax -t template-2.j2,/config-db or -t template-2.j2,./config-db to write to file named config-db. The check is strictly done against config-db as keyword. The / or ./ would serve as a distinction between the keyword and a filename that happens to be the same as the keyword.
I can changed it to [config-db], however the premise of argument still exist. What if the file name is [config-db]. I tend to think this a user error since there is clear way if config-db is used as a file name.
There was a problem hiding this comment.
Just tried the ./config-db and it does generate the file as intended:
admin@str-s6000-acs-14:~$ sudo sonic-cfggen -d -t /usr/share/sonic/templates/rsyslog.conf.j2,./config-db
admin@str-s6000-acs-14:~$ ls -alrt
total 85
-rw-r--r-- 1 admin admin 807 Apr 18 2019 .profile
-rw-r--r-- 1 admin admin 3526 Apr 18 2019 .bashrc
-rw-r--r-- 1 admin admin 220 Apr 18 2019 .bash_logout
drwxr-xr-x 1 root root 4096 Aug 11 03:41 ..
drwxr-xr-x 1 admin admin 4096 Aug 12 21:16 .
-rw-r--r-- 1 root root 1907 Aug 12 21:16 config-db
qiluo-msft
dismissed
their stale review
Unblock, still the design of command line could improve to prevent keyword
tahmed-dev
changed the title
|
@tahmed-dev Please create PR for 201911. Cherry-pick has conflict. |
|
Calls to cfggen take considerable time. With batch mode, we will have the ability to reduce number of calls from services. Example of the batch mode command: sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db. template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Calls to cfggen take considerable time. With batch mode, we will have the ability to reduce number of calls from services. Example of the batch mode command: sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db. template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Calls to cfggen take considerable time. With batch mode, we will have the ability to reduce number of calls from services. Example of the batch mode command: sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db. template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Calls to cfggen take considerable time. With batch mode, we will have the ability to reduce number of calls from services. Example of the batch mode command: sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db. template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Calls to cfggen take considerable time. With batch mode, we will have the ability to reduce number of calls from services. Example of the batch mode command: sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db. template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
Calls to cfggen take considerable time. With batch mode, we will have the ability to reduce number of calls from services. Example of the batch mode command: sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db. template-1.j2 will be rendered to stdout since it is missing the dest part. stdout is default config-db is a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with --write-to-db switch. In the case the user would like to write to a file named config-db, it could be given as /config-db or ./config-db signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
5 participants
Calls to cfggen take considerable time. With batch mode, we will have the ability
to reduce number of calls from service to one call only.
Example of the batch mode command:
sonic-cfggen -t template-1.j2 -t template-2.j2,config-db -t template-3.j2,config-db -t template-4.j2,file1 -t template-5.j2,file2 --write-to-db.template-1.j2will be rendered to stdout since it is missing the dest part. stdout is defaultconfig-dbis a special keyword that will inject the rendered template into internal data structure. The internal data structure gets written to redis-db with--write-to-dbswitch. In the case the user would like to write to a file named config-db, it could be given as/config-dbor./config-dbsigned-off-by: Tamer Ahmed tamer.ahmed@microsoft.com
- Why I did it
In order to reduce number of calls being made to sonic-cfggen during boot process
- How I did it
Added new mode (batch) to cfggen where multiple templates could be rendered and written to disk at once
- How to verify it
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)