Skip to content

Comments

ko: get ovn leader pod on necessary#6273

Draft
zhangzujian wants to merge 1 commit intokubeovn:masterfrom
zhangzujian:fix-ko
Draft

ko: get ovn leader pod on necessary#6273
zhangzujian wants to merge 1 commit intokubeovn:masterfrom
zhangzujian:fix-ko

Conversation

@zhangzujian
Copy link
Member

Pull Request

What type of this PR

Examples of user facing changes:

  • Features
  • Bug fixes
  • Docs
  • Tests

Which issue(s) this PR fixes

Fixes #(issue-number)

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @zhangzujian, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces significant refactoring to the kubectl-ko helper script, enhancing its robustness and precision when interacting with Kube-OVN components. By transitioning from generic pod targeting to explicit leader pod identification for OVN Northbound, Southbound, Northd, and Interconnect databases, the script ensures that administrative and diagnostic commands are consistently directed to the authoritative instances. This change streamlines operations and mitigates potential inconsistencies in clustered OVN environments.

Highlights

  • Explicit Leader Pod Targeting: The script now explicitly identifies and targets leader pods for OVN Northbound, Southbound, Northd, and Interconnect components, ensuring commands are executed against the correct instances.
  • Modular Pod Retrieval Functions: Monolithic pod retrieval logic has been broken down into dedicated functions (e.g., getNBLeaderPod, getSBLeaderPod), improving code organization and maintainability.
  • Consistent Variable Naming: OVN pod variables have been renamed to clearly reflect their role as leader pods (e.g., OVN_NB_POD is now OVN_NB_LEADER_POD).
Changelog
  • dist/images/kubectl-ko
    • Renamed environment variables like OVN_NB_POD to OVN_NB_LEADER_POD to reflect their new leader-specific purpose.
    • Replaced the single getOvnCentralPod function with multiple specialized functions (getNBLeaderPod, getSBLeaderPod, getNorthdLeaderPod, getICNBLeaderPod, getICSBLeaderPod) for retrieving specific OVN leader pods.
    • Updated all command executions (kubectl exec) throughout the script to utilize the newly defined leader pod variables and functions.
    • The getOvnCentralDbStatus function was renamed to getOvnDbStatus and its pod selection logic was updated to use the app=ovn-central label.
Activity
  • The pull request was created by zhangzujian.
  • The pull request description indicates it introduces "Features".
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the shell script kubectl-ko to improve OVN leader pod retrieval by breaking down the getOvnCentralPod function into smaller, more specific functions, enhancing clarity and efficiency. However, a critical security vulnerability has been identified: the script uses unquoted variables, such as $KUBE_OVN_NS and new leader pod variables, in sensitive kubectl commands (exec, cp, delete), which exposes it to argument injection and unexpected shell behavior. Furthermore, a critical runtime error exists due to a remaining call to the deleted getOvnCentralPod function, and there are high-severity issues with inconsistent pod status checks that could result in selecting non-ready pods. Please ensure all variable expansions are quoted for security, and address the identified functional issues to improve the script's reliability and robustness.


getOvnCentralPod(){
getNBLeaderPod(){
NB_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-nb-leader=true | grep ovn-central | awk '{if($2=="1/1" && $3=="Running") print $1}' | head -n 1 )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This refactoring of getOvnCentralPod into smaller functions is a good improvement for modularity. However, it introduces a critical issue: the perf subcommand (at line 1700) still calls the now-deleted getOvnCentralPod function. This will cause the script to fail. Please ensure all call sites are updated.

Additionally, this command can be made more robust and efficient by using a single awk command with a more specific label selector and --no-headers.

Suggested change
NB_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-nb-leader=true | grep ovn-central | awk '{if($2=="1/1" && $3=="Running") print $1}' | head -n 1 )
NB_POD=$(kubectl get pod -n "$KUBE_OVN_NS" -l ovn-nb-leader=true,app=ovn-central --no-headers | awk '$2 == "1/1" && $3 == "Running" {print $1; exit}')

}

getNorthdLeaderPod(){
NORTHD_POD=$(kubectl get pod -n kube-system -l ovn-northd-leader=true | grep ovn-central | head -n 1 | awk '{print $1}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This function is missing a check to ensure the pod is ready and running (1/1 and Running status), which is present in getNBLeaderPod. This inconsistency can lead to selecting a pod that is not yet ready to serve traffic. Please add the status check for consistency and robustness.

The command can also be improved to be more efficient.

Suggested change
NORTHD_POD=$(kubectl get pod -n kube-system -l ovn-northd-leader=true | grep ovn-central | head -n 1 | awk '{print $1}')
NORTHD_POD=$(kubectl get pod -n kube-system -l ovn-northd-leader=true,app=ovn-central --no-headers | awk '$2 == "1/1" && $3 == "Running" {print $1; exit}')

OVN_IC_NB_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-ic-nb-leader=true | grep ovn-ic-server | head -n 1 | awk '{print $1}')
if [ -z "$OVN_IC_NB_POD" ]; then
getICNBLeaderPod(){
OVN_IC_NB_LEADER_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-ic-nb-leader=true | grep ovn-ic-server | head -n 1 | awk '{print $1}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This function is missing a check to ensure the pod is ready and running (1/1 and Running status), which is present in getNBLeaderPod. This inconsistency can lead to selecting a pod that is not yet ready to serve traffic. Please add the status check for consistency and robustness.

The command can also be improved to be more efficient by combining grep, head and awk.

Suggested change
OVN_IC_NB_LEADER_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-ic-nb-leader=true | grep ovn-ic-server | head -n 1 | awk '{print $1}')
OVN_IC_NB_LEADER_POD=$(kubectl get pod -n "$KUBE_OVN_NS" -l ovn-ic-nb-leader=true --no-headers | grep ovn-ic-server | awk '$2 == "1/1" && $3 == "Running" {print $1; exit}')

OVN_IC_SB_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-ic-sb-leader=true | grep ovn-ic-server | head -n 1 | awk '{print $1}')
if [ -z "$OVN_IC_SB_POD" ]; then
getICSBLeaderPod(){
OVN_IC_SB_LEADER_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-ic-sb-leader=true | grep ovn-ic-server | head -n 1 | awk '{print $1}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This function is missing a check to ensure the pod is ready and running (1/1 and Running status), which is present in getNBLeaderPod. This inconsistency can lead to selecting a pod that is not yet ready to serve traffic. Please add the status check for consistency and robustness.

The command can also be improved to be more efficient by combining grep, head and awk.

Suggested change
OVN_IC_SB_LEADER_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-ic-sb-leader=true | grep ovn-ic-server | head -n 1 | awk '{print $1}')
OVN_IC_SB_LEADER_POD=$(kubectl get pod -n "$KUBE_OVN_NS" -l ovn-ic-sb-leader=true --no-headers | grep ovn-ic-server | awk '$2 == "1/1" && $3 == "Running" {print $1; exit}')

set +o pipefail
if [ $af -eq 4 ]; then
dstMac=$(kubectl exec $OVN_NB_POD -n $KUBE_OVN_NS -c ovn-central -- ovn-nbctl --data=bare --no-heading --columns=addresses list logical_switch_port | grep -w "$(echo $dst | tr . '\.')" | awk '{print $1}')
dstMac=$(kubectl exec $OVN_NB_LEADER_POD -n $KUBE_OVN_NS -c ovn-central -- ovn-nbctl --data=bare --no-heading --columns=addresses list logical_switch_port | grep -w "$(echo $dst | tr . '\.')" | awk '{print $1}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The variable $KUBE_OVN_NS is used unquoted in a kubectl exec command. Unquoted variables in bash are subject to word splitting and glob expansion, which can lead to argument injection if the variable contains spaces or other shell-special characters. Although Kubernetes namespaces have restricted characters, it is a security best practice to always quote variables used in command arguments to prevent unexpected behavior and potential injection vulnerabilities.

Suggested change
dstMac=$(kubectl exec $OVN_NB_LEADER_POD -n $KUBE_OVN_NS -c ovn-central -- ovn-nbctl --data=bare --no-heading --columns=addresses list logical_switch_port | grep -w "$(echo $dst | tr . '\.')" | awk '{print $1}')
dstMac=$(kubectl exec "$OVN_NB_LEADER_POD" -n "$KUBE_OVN_NS" -c ovn-central -- ovn-nbctl --data=bare --no-heading --columns=addresses list logical_switch_port | grep -w "$(echo $dst | tr . '\.')" | awk '{print $1}')

kubectl cp $KUBE_OVN_NS/$OVN_NB_POD:/etc/ovn/ovnnb_db.$suffix.backup $(pwd)/ovnnb_db.$suffix.backup
kubectl exec "$OVN_NB_POD" -n $KUBE_OVN_NS -c ovn-central -- rm -f /etc/ovn/ovnnb_db.$suffix.backup
kubectl exec "$OVN_NB_LEADER_POD" -n $KUBE_OVN_NS -c ovn-central -- ovsdb-tool cluster-to-standalone /etc/ovn/ovnnb_db.$suffix.backup /etc/ovn/ovnnb_db.db
kubectl cp $KUBE_OVN_NS/$OVN_NB_LEADER_POD:/etc/ovn/ovnnb_db.$suffix.backup $(pwd)/ovnnb_db.$suffix.backup
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The variable $OVN_NB_LEADER_POD is used unquoted in a kubectl cp command. This can lead to argument injection if the variable content is manipulated. Always quote variables in bash scripts to ensure they are treated as a single argument and to prevent shell expansion issues.

Suggested change
kubectl cp $KUBE_OVN_NS/$OVN_NB_LEADER_POD:/etc/ovn/ovnnb_db.$suffix.backup $(pwd)/ovnnb_db.$suffix.backup
kubectl cp "$KUBE_OVN_NS/$OVN_NB_LEADER_POD":/etc/ovn/ovnnb_db.$suffix.backup $(pwd)/ovnnb_db.$suffix.backup

echo "Delete ovn central $component_name pod"
if [[ $component_name == "nb" ]]; then
kubectl delete pod $OVN_NB_POD -n kube-system
kubectl delete pod $OVN_NB_LEADER_POD -n kube-system
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The variable $OVN_NB_LEADER_POD is used unquoted in a kubectl delete command. In bash, unquoted variables can be exploited for argument injection. Even if the source of the variable (e.g., kubectl get pods output) is currently considered safe, quoting is a critical defense-in-depth measure in shell scripting.

Suggested change
kubectl delete pod $OVN_NB_LEADER_POD -n kube-system
kubectl delete pod "$OVN_NB_LEADER_POD" -n kube-system

getOvnCentralPod
kubectl exec "$OVN_NB_POD" -n $KUBE_OVN_NS -c ovn-central -- ovn-nbctl "$@"
getNBLeaderPod
kubectl exec "$OVN_NB_LEADER_POD" -n $KUBE_OVN_NS -c ovn-central -- ovn-nbctl "$@"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The variable $KUBE_OVN_NS is used unquoted in a kubectl exec command within the nbctl subcommand handler. This pattern is repeated throughout the script and should be corrected by quoting all variable expansions to prevent argument injection.

Suggested change
kubectl exec "$OVN_NB_LEADER_POD" -n $KUBE_OVN_NS -c ovn-central -- ovn-nbctl "$@"
kubectl exec "$OVN_NB_LEADER_POD" -n "$KUBE_OVN_NS" -c ovn-central -- ovn-nbctl "$@"

}

getSBLeaderPod(){
SB_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-sb-leader=true | grep ovn-central | awk '{if($2=="1/1" && $3=="Running") print $1}' | head -n 1 )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This command can be made more robust and efficient by using a single awk command with a more specific label selector and --no-headers to avoid parsing kubectl's default output and remove the dependency on grep and head.

Suggested change
SB_POD=$(kubectl get pod -n $KUBE_OVN_NS -l ovn-sb-leader=true | grep ovn-central | awk '{if($2=="1/1" && $3=="Running") print $1}' | head -n 1 )
SB_POD=$(kubectl get pod -n "$KUBE_OVN_NS" -l ovn-sb-leader=true,app=ovn-central --no-headers | awk '$2 == "1/1" && $3 == "Running" {print $1; exit}')

@coveralls
Copy link

Pull Request Test Coverage Report for Build 21743145260

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.004%) to 22.896%

Totals Coverage Status
Change from base Build 21743134951: 0.004%
Covered Lines: 12436
Relevant Lines: 54316

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants