-
Notifications
You must be signed in to change notification settings - Fork 1.8k
syslog changes Multi NPU platforms #4738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
a4a10a7
795b2b0
3616a5f
16c6b3d
1deb8a0
b760de0
222489d
84aa4b5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -27,6 +27,22 @@ link_namespace() { | |||||||
| } | ||||||||
| {%- endif %} | ||||||||
|
|
||||||||
| function updateSyslogConf() | ||||||||
| { | ||||||||
| # On multiNPU platforms, change the syslog target ip to docker0 ip to allow logs from containers | ||||||||
| # running on the namespace to reach the rsyslog service running on the host | ||||||||
| # Also update the container name | ||||||||
| if [[ ($NUM_ASIC -gt 1) ]]; then | ||||||||
| TARGET_IP=$(ip -o -4 addr list docker0 | awk '{print $4}' | cut -d/ -f1) | ||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in the latest commit |
||||||||
| CONTAINER_NAME="{{docker_container_name}}$DEV" | ||||||||
| TMP_FILE="/tmp/rsyslog.$CONTAINER_NAME.conf" | ||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Could you use Dockerfile
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The tmp file is created after rendering this template /usr/share/sonic/templates/rsyslog-container.conf.j2, which has updated syslog.conf file. I don't think the tmp file can be avoided even if we pass the ENV to the container. In Multi-NPU platforms it is possible that multi containers can be started at the same time, to avoid container using wrong syslog.conf files I am using the container name in the tmp filename
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's create two ENV:
Pass the ENV values by command The benefit:
In reply to: 439583041 [](ancestors = 439583041)
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @qiluo-msft created a issue to track 4778 to improvements suggested in this comment. |
||||||||
|
|
||||||||
| sonic-cfggen -t /usr/share/sonic/templates/rsyslog-container.conf.j2 -a "{\"target_ip\": \"$TARGET_IP\", \"container_name\": \"$CONTAINER_NAME\" }" > $TMP_FILE | ||||||||
| docker cp $TMP_FILE {{docker_container_name}}$DEV:/etc/rsyslog.conf | ||||||||
| rm -rf $TMP_FILE | ||||||||
| fi | ||||||||
| } | ||||||||
|
|
||||||||
| function getMountPoint() | ||||||||
| { | ||||||||
| echo $1 | python -c "import sys, json, os; mnts = [x for x in json.load(sys.stdin)[0]['Mounts'] if x['Destination'] == '/usr/share/sonic/hwsku']; print '' if len(mnts) == 0 else os.path.basename(mnts[0]['Source'])" 2>/dev/null | ||||||||
|
|
@@ -68,6 +84,7 @@ function preStartAction() | |||||||
| {%- else %} | ||||||||
| : # nothing | ||||||||
| {%- endif %} | ||||||||
| updateSyslogConf | ||||||||
| } | ||||||||
|
|
||||||||
| function postStartAction() | ||||||||
|
|
||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,23 @@ | ||
| #!/bin/bash | ||
|
|
||
| sonic-cfggen -d -t /usr/share/sonic/templates/rsyslog.conf.j2 >/etc/rsyslog.conf | ||
| PLATFORM=`sonic-cfggen -H -v DEVICE_METADATA.localhost.platform` | ||
|
|
||
| # Parse the device specific asic conf file, if it exists | ||
| ASIC_CONF=/usr/share/sonic/device/$PLATFORM/asic.conf | ||
| if [ -f "$ASIC_CONF" ]; then | ||
| source $ASIC_CONF | ||
| fi | ||
|
|
||
| # On Multi NPU platforms we need to start the rsyslog server on the docker0 ip address | ||
| # for the syslogs from the containers in the namespaces to work. | ||
| # on Single NPU platforms we continue to use loopback adddres | ||
|
|
||
| if [[ ($NUM_ASIC -gt 1) ]]; then | ||
| udp_server_ip=$(ip -o -4 addr list docker0 | awk '{print $4}' | cut -d/ -f1) | ||
| else | ||
| udp_server_ip=$(ip -o -4 addr list lo scope host | awk '{print $4}' | cut -d/ -f1) | ||
| fi | ||
|
|
||
| sonic-cfggen -d -t /usr/share/sonic/templates/rsyslog.conf.j2 -a "{\"udp_server_ip\": \"$udp_server_ip\"}" >/etc/rsyslog.conf | ||
|
|
||
| systemctl restart rsyslog |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # | ||
| # /etc/rsyslog.conf Configuration file for rsyslog. | ||
| # | ||
| # For more information see | ||
| # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html | ||
|
|
||
|
|
||
| ################# | ||
| #### MODULES #### | ||
| ################# | ||
|
|
||
| $ModLoad imuxsock # provides support for local system logging | ||
|
|
||
| # | ||
| # Set a rate limit on messages from the container | ||
| # | ||
| $SystemLogRateLimitInterval 300 | ||
| $SystemLogRateLimitBurst 20000 | ||
|
|
||
| #$ModLoad imklog # provides kernel logging support | ||
| #$ModLoad immark # provides --MARK-- message capability | ||
|
|
||
| # provides UDP syslog reception | ||
| #$ModLoad imudp | ||
| #$UDPServerRun 514 | ||
|
|
||
| # provides TCP syslog reception | ||
| #$ModLoad imtcp | ||
| #$InputTCPServerRun 514 | ||
|
|
||
|
|
||
| ########################### | ||
| #### GLOBAL DIRECTIVES #### | ||
| ########################### | ||
|
|
||
| # Set remote syslog server | ||
| template (name="ForwardFormatInContainer" type="string" string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% {{container_name}}#%syslogtag%%msg:::sp-if-no-1st-sp%%msg%") | ||
| *.* action(type="omfwd" target="{{target_ip}}" port="514" protocol="udp" Template="ForwardFormatInContainer") | ||
|
|
||
| # | ||
| # Use traditional timestamp format. | ||
| # To enable high precision timestamps, comment out the following line. | ||
| # | ||
| #$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat | ||
|
|
||
| # Define a custom template | ||
| $template SONiCFileFormat,"%TIMESTAMP%.%timestamp:::date-subseconds% %HOSTNAME% %syslogseverity-text:::uppercase% {{container_name}}#%syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" | ||
| $ActionFileDefaultTemplate SONiCFileFormat | ||
|
|
||
| # | ||
| # Set the default permissions for all log files. | ||
| # | ||
| $FileOwner root | ||
| $FileGroup adm | ||
| $FileCreateMode 0640 | ||
| $DirCreateMode 0755 | ||
| $Umask 0022 | ||
|
|
||
| # | ||
| # Where to place spool and state files | ||
| # | ||
| $WorkDirectory /var/spool/rsyslog | ||
|
|
||
| # | ||
| # Include all config files in /etc/rsyslog.d/ | ||
| # | ||
| $IncludeConfig /etc/rsyslog.d/*.conf | ||
|
|
||
| # | ||
| # Suppress duplicate messages and report "message repeated n times" | ||
| # | ||
| $RepeatedMsgReduction on | ||
|
|
||
| ############### | ||
| #### RULES #### | ||
| ############### |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This IP address is actually available in build time. Check
files/docker/docker.service.conf#ClosedThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I didn't want to hard code the ip address here. So using this method to get the docker0 ip address
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean
docker0IP address is actually available in build time. Checkfiles/docker/docker.service.conf. You are ok to useipcommand, but build time constant may be easier.In reply to: 439581391 [](ancestors = 439581391)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. Your solution is future proof.
In reply to: 440542831 [](ancestors = 440542831,439581391)