feat(sdk): Add get_logs() method to download component logs #12396
Conversation
Implements kubeflow#12388 by adding SDK support for downloading logs from pipeline run components. Changes: - Added get_logs() public method to Client class - Added helper methods _get_workflow_name_from_run() and _extract_component_name_from_pod() - Added comprehensive unit tests with mocking - Leverages existing kubernetes dependency (v30.1.0) Features: - Fetch logs from specific component by name - Fetch logs from all components in a run - Support for multi-user namespace configurations - Graceful fallback for different container configurations Tests: - 11 new test cases covering success and error scenarios - All tests use mocking (no cluster required) - Follows existing test patterns from client_test.py Fixes kubeflow#12388 Signed-off-by: SoumyaRaikwar <[email protected]>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Hi @SoumyaRaikwar. Thanks for your PR. I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
🚫 This command cannot be processed. Only organization members or owners can use the commands. |
|
@VaniHaripriya , @mprahl and @DharmitD could you please review my pr . |
|
@mprahl could you please review my pr |
|
|
||
| namespace = namespace or self.get_user_namespace() | ||
|
|
||
| run = self.get_run(run_id=run_id) |
There was a problem hiding this comment.
I'd rather this leverage the log artifacts automatically uploaded by the KFP launcher. This has the benefit of still working even if the pod gets deleted and it doesn't require the user to have access to the Kubernetes cluster itself.
Changed implementation from Kubernetes pod querying to using KFP's ReadArtifact API. This approach: - Works even after pod deletion - Doesn't require Kubernetes cluster access - Leverages KFP's built-in artifact storage Signed-off-by: SoumyaRaikwar <[email protected]>
|
@mprahl Updated to use artifact-based approach as suggested. Implementation now uses ReadArtifact API instead of direct Kubernetes pod querying. Works even after pod deletion and doesn't require cluster access. Ready for review. |
2 participants
What this PR does
Implements SDK method to download component logs from pipeline runs, addressing the feature request in #12388.
Changes
get_logs(): Public method to retrieve logs by run ID and optional component name_get_workflow_name_from_run(): Helper to extract Argo workflow names_extract_component_name_from_pod(): Helper to parse component names from pod metadataFeatures
Fetch single component logs (returns string)
Fetch all component logs (returns dict)
Multi-user namespace support
Graceful container fallback handling
Testing
Fixes #12388