Audit and standardize exception handling in instructor library#1897
Merged
Audit and standardize exception handling in instructor library#1897
Conversation
This commit significantly enhances the exception handling system across the instructor library with improved documentation, context, and consistency. ## Exception Class Enhancements - Added comprehensive docstrings to all exception classes with: - Detailed descriptions of when exceptions are raised - Common causes and solutions - Practical usage examples - Attribute documentation - Enhanced InstructorError base class with detailed usage patterns - Documented FailedAttempt, IncompleteOutputException, InstructorRetryException - Added usage guidance for ValidationError, ProviderError, ConfigurationError, ModeError, ClientError, and AsyncValidationError ## New Exception Types - **ResponseParsingError**: For LLM response parsing failures - Includes mode and raw_response attributes for better debugging - Used when response format is unexpected or malformed - **MultimodalError**: For multimodal content processing errors - Includes content_type (image/audio/pdf) and file_path attributes - Provides clear context for file and format issues ## Replaced Generic Exceptions ### processing/function_calls.py - Replaced ValueError with ResponseParsingError for: - Missing completion choices - Provider errors - Cohere response parsing failures - Bedrock response parsing failures - Gemini JSON extraction failures - Response tool call failures - Replaced ValueError with ConfigurationError for: - Invalid patch modes - Multiple tool call restrictions - Invalid response_model types ### processing/multimodal.py - Replaced ValueError with MultimodalError for: - File not found errors (with file_path context) - Unsupported image/audio/PDF formats (with content_type) - File reading failures ### processing/response.py - Replaced ValueError with ConfigurationError for: - Invalid/unsupported modes (with list of available modes) ## Improved Error Context All new exceptions include contextual information: - mode: The mode being used when the error occurred - raw_response: The raw LLM response for debugging - content_type: The type of multimodal content (image/audio/pdf) - file_path: The file path when file operations fail This addresses the need for better monitoring and diagnostics by providing access to raw responses and additional context in validation/parsing errors. ## Backwards Compatibility - All existing exception types maintained - Deprecated import path (instructor.exceptions) still works - All exception hierarchy preserved - New exceptions exported in both core and main modules ## Testing - All existing tests pass (43/43 tests passed) - Exception hierarchy tests verify proper inheritance - FailedAttempt tracking tests verify context preservation BREAKING CHANGES: None - all changes are backwards compatible
This commit ensures full backwards compatibility for the new exception types and updates documentation to reflect the enhanced exception system. ## Backwards Compatibility Enhancements ### ResponseParsingError - Now inherits from both ValueError and InstructorError - Existing code catching ValueError will still work - Provides additional context (mode, raw_response) when needed ### MultimodalError - Now inherits from both ValueError and InstructorError - Existing code catching ValueError will still work - Provides additional context (content_type, file_path) when needed This follows the same pattern as AsyncValidationError which already inherits from both ValueError and InstructorError. ## Testing Added comprehensive backwards compatibility tests: - test_response_parsing_error_is_value_error - test_multimodal_error_is_value_error - test_async_validation_error_is_value_error - test_exception_inheritance_chain - test_mixed_exception_catching - test_exception_attributes_preserved All 49 tests pass (26 exception tests + 6 compat tests + 17 other tests) ## Documentation Updates Updated /docs/concepts/error_handling.md with: 1. New exception type documentation: - ResponseParsingError with mode and raw_response attributes - MultimodalError with content_type and file_path attributes - AsyncValidationError documentation 2. Backwards Compatibility section: - Shows how old code (catching ValueError) still works - Shows how new code can access additional context - Clear migration path for users 3. Diagnostic Context section: - Examples of logging ResponseParsingError with full context - Examples of logging MultimodalError with file information - Examples of logging InstructorRetryException with attempt history - Demonstrates monitoring and debugging capabilities ## Breaking Changes NONE - All changes are fully backwards compatible: - Old code catching ValueError continues to work - New exceptions provide additional context when needed - Exception hierarchy preserved - Existing exception types unchanged This ensures a smooth upgrade path for users moving to version 1.14.
This summary document provides a complete overview of all exception handling improvements made for the 1.14 release, including: - All changes made to exception classes - Backwards compatibility strategy and verification - Test results (49/49 tests passing) - Benefits for users (monitoring, debugging, context) - Migration guide (no changes required) - File-by-file change details This serves as both documentation and a reference for the changes.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
instructor | 35ae7e8 | Nov 06 2025, 05:42 AM |
Contributor
There was a problem hiding this comment.
Important
Looks good to me! 👍
Reviewed everything up to 35ae7e8 in 1 minute and 2 seconds. Click for details.
- Reviewed
1181lines of code in8files - Skipped
0files when reviewing. - Skipped posting
7draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. docs/concepts/error_handling.md:119
- Draft comment:
The documentation for new exceptions (ResponseParsingError, MultimodalError, AsyncValidationError) is thorough and clear. Consider explicitly noting in each section that these exceptions remain backwards compatible by inheriting from ValueError and InstructorError. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
2. instructor/processing/function_calls.py:290
- Draft comment:
Replace assert statements (e.g. checking function_call.name) with explicit exception handling (such as raising a ResponseParsingError) for production reliability, since assert may be disabled in optimized runtimes. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
3. instructor/core/exceptions.py:520
- Draft comment:
The new exception classes (ResponseParsingError and MultimodalError) are well documented with clear attributes and example usage. Ensure that the error messages remain consistent across the library. - Reason this comment was not posted:
Comment did not seem useful. Confidence is useful =30%<= threshold85%This comment is asking the author to ensure consistency in error messages across the library. While it is important to maintain consistency, the comment is not specific about what needs to be changed or improved. It doesn't provide a specific suggestion or point out a specific issue. Therefore, it seems to be more of a general reminder rather than a specific actionable comment.
4. instructor/exceptions.py:11
- Draft comment:
The deprecation warning in this backward compatibility module is clear. Confirm that the chosen stacklevel (2) correctly points users to the caller for proper guidance. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
5. tests/test_exception_backwards_compat.py:12
- Draft comment:
The tests verifying exception inheritance and catching are comprehensive and ensure backward compatibility. They effectively confirm that the new exceptions can be caught as ValueError, InstructorError, and their specific types. - Reason this comment was not posted:
Comment did not seem useful. Confidence is useful =0%<= threshold85%This comment is purely informative and does not provide any actionable feedback or suggestions for improvement. It simply praises the existing tests without offering any specific guidance or identifying any issues.
6. instructor/processing/multimodal.py:115
- Draft comment:
Using @lru_cache on methods like from_url and from_path can improve performance, but be cautious of potential staleness if the underlying file or URL content changes. Consider whether cache invalidation is needed in some scenarios. - Reason this comment was not posted:
Confidence changes required:80%<= threshold85%None
7. instructor/core/__init__.py:41
- Draft comment:
Typo found: 'apatch' in line 41 looks like it might be an accidental entry or typo. Please verify if this should be corrected. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
Workflow ID: wflow_yr4SNtodm3hiIVYk
You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Enhances exception handling in the instructor library by adding new exception types and ensuring backward compatibility, with tests to verify these changes.
ResponseParsingError,MultimodalError, andAsyncValidationErrorinexceptions.pyfor specific error scenarios.ValueErrorandInstructorError.function_calls.pyandmultimodal.pyto raise new exceptions for specific error cases.test_exception_backwards_compat.pyto verify backward compatibility and correct inheritance of new exceptions.ValueErrorandInstructorError.This description was created by
for 35ae7e8. You can customize this summary. It will automatically update as commits are pushed.