Add Klish AAA CLI validation tests for VS KVM testing#11
Open
devin-ai-integration[bot] wants to merge 2 commits intomasterfrom
Open
Add Klish AAA CLI validation tests for VS KVM testing#11devin-ai-integration[bot] wants to merge 2 commits intomasterfrom
devin-ai-integration[bot] wants to merge 2 commits intomasterfrom
Conversation
Add integration tests to validate Klish AAA CLI commands implemented in sonic-mgmt-framework (branch devin/1769734716-aaa-klish-cli). Tests cover: - AAA authentication failthrough/fallback/debug/trace enable/disable/reset - AAA authentication login methods (local, tacacs+, radius, multi-method) - AAA authorization and accounting configuration - REST API endpoint validation (when mgmt-framework is available) - Klish CLI command validation via sonic-cli - Full workflow: configure all AAA settings and verify, then negate and verify - show aaa output parsing and validation Tests are added to kvmtest.sh t0 part-1 test suite. Co-Authored-By: Arthur Poon <[email protected]>
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…ed constants Co-Authored-By: Arthur Poon <[email protected]>
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Adds integration tests to validate the Klish AAA CLI commands implemented in
sonic-mgmt-framework(branchdevin/1769734716-aaa-klish-cli). Tests target a VS (Virtual Switch) DUT in the kvmtest pipeline.The Klish AAA CLI implementation adds commands for configuring AAA authentication (failthrough, fallback, debug, trace, login methods), authorization, and accounting via the management framework's Klish shell and REST API layer.
Reviewer should start with
tests/klish_aaa/test_klish_aaa.py— the test is organized into classes by feature area.Link to Devin run: https://cisco-demo.devinenterprise.com/sessions/8afc9b7efc1144cd8f447ee401302b6d
Requested by: @arthurkkp-cog
Type of change
Back port request
Approach
What is the motivation for this PR?
Validate the Klish AAA CLI commands implemented in sonic-mgmt-framework to ensure CONFIG_DB writes,
show aaaoutput, REST API endpoints, and Klish CLI invocation all work correctly on a VS DUT.How did you do it?
New test module
tests/klish_aaa/test_klish_aaa.pywith ~35 test cases organized into 11 classes:TestAAAShowDefaultshow aaaoutput after clearing configTestAAAAuthenticationFailthroughTestAAAAuthenticationFallbackTestAAAAuthenticationDebugTestAAAAuthenticationTraceTestAAAAuthenticationLoginTestAAAAuthorizationTestAAAAccountingTestAAARestAPITestAAAKlishCLIsonic-cliTestAAAFullWorkflowTests use:
sonic-db-cli CONFIG_DBto directly read/write AAA configshow aaacommand to verify outputcurlto call REST API endpoints (when mgmt-framework is available)docker exec mgmt-framework ... clishto invoke Klish CLI (when mgmt-framework is available)Tests are added to
kvmtest.sht0 part-1 suite alongside existing TACACS tests.How did you verify/test it?
Syntax and lint checks passed:
python3 -m py_compile✓flake8 --max-line-length=120✓Any platform specific information?
VS only (
pytest.mark.device_type('vs')). Tests skip if mgmt-framework container is not running.Supported testbed topology if it's a new test case?
pytest.mark.topology('any')— should work on any topology with a VS DUT.Documentation
N/A — test-only change.
Updates since last revision
Fixed issues from initial review:
save_and_restore_aaafixture now saves and restores all AAA table keys (authentication,authorization,accounting) instead of justauthentication. This prevents state leakage between tests.AAA_TABLEandAAA_KEYconstants, replaced withAAA_KEYSlist.Human Review Checklist
Critical items to verify:
Tests were NOT executed on a real VS DUT — This is the primary risk. All test logic is based on code review and assumptions about the runtime environment.
REST API authentication:
rest_api_callusescurl -skwithout credentials. Verify whether the REST server on VS requires authentication. If yes, allTestAAARestAPItests will fail with 401/403.Klish CLI invocation mechanism:
run_klish_commandusesdocker exec mgmt-framework bash -c "source /usr/sbin/cli/klish/clish_start && echo '...' | /usr/sbin/cli/clish -o". Verify:clishworks (may need interactive TTY)TestAAAKlishCLIclass may skip if this doesn't workshow aaaoutput format:parse_show_aaaassumes specific line prefixes like"AAA authentication login". Verify actual output format matches expectations.Tests can run in kvmtest pipeline: Verify the VS image used in kvmtest has mgmt-framework container and the Klish AAA implementation.
Minor items:
duthostfixture shadows global fixture pattern — consider usingrand_selected_dutinstead