feat: Improve error handling with comprehensive exception hierarchy

[jxnl]

Summary

• Added a comprehensive exception hierarchy to improve error handling and debugging
• Replaced assertions with proper exceptions for better production reliability
• Enhanced error messages with contextual information

Changes

New Exception Hierarchy

• InstructorError - Base exception for all Instructor-specific errors
• IncompleteOutputException - When LLM output is incomplete due to token limits
• InstructorRetryException - When all retry attempts are exhausted
• ValidationError - When response validation fails
• ProviderError - Provider-specific errors with provider context
• ConfigurationError - Configuration-related errors
• ModeError - Invalid mode for a provider with valid modes listed
• ClientError - Client initialization/usage errors

Code Improvements

• Replaced assert statements with proper exceptions in:
  • client_anthropic.py
  • client_vertexai.py
  • process_response.py
• Updated error handling in:
  • auto_client.py - Better import and configuration errors
  • function_calls.py - More specific validation errors
  • patch.py - Configuration validation
  • retry.py - Parameter validation

Documentation

• Added comprehensive error handling guide at docs/concepts/error_handling.md
• Updated retrying documentation with exception hierarchy
• Enhanced hooks documentation with error handling examples
• Added error handling to navigation

Breaking Changes

Some ValueError and TypeError exceptions are now replaced with more specific exception types from instructor.exceptions. Users catching these generic exceptions should update their code to catch the specific types or use InstructorError as a catch-all.

Test Plan

• All existing tests pass
• Pre-commit hooks pass (linting and formatting)
• Manual testing of error scenarios
• Review by maintainers

🤖 Generated with Claude Code

---

Important

Introduces a comprehensive exception hierarchy for improved error handling, replacing assertions with specific exceptions across the codebase.

• Behavior:
  • Introduces a new exception hierarchy in exceptions.py for improved error handling.
  • Replaces assert statements with specific exceptions in client_anthropic.py, client_vertexai.py, and process_response.py.
  • Updates error handling logic in auto_client.py, function_calls.py, patch.py, and retry.py.
• Documentation:
  • Adds error_handling.md to document the new exception hierarchy and best practices.
  • Updates hooks.md and retrying.md to include examples of the new exceptions.
• Tests:
  • Updates tests in test_auto_client.py to use new exception types like ConfigurationError.
• Breaking Changes:
  • Replaces some ValueError and TypeError with specific exceptions from instructor.exceptions.

^{This description was created by for 84ab2be. You can customize this summary. It will automatically update as commits are pushed.}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	instructor	84ab2be	Visit preview	May 22 2025, 08:23 PM

[ellipsis-dev]

Important

Looks good to me! 👍

Reviewed everything up to f12a519 in 2 minutes and 57 seconds. Click for details.

• Reviewed 859 lines of code in 12 files
• Skipped 0 files when reviewing.
• Skipped posting 11 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. instructor/auto_client.py:90

• Draft comment:
  Good improvement: replacing a bare ValueError with a ConfigurationError to enforce the model string format. This enhances error specificity and production reliability.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 85% This comment is purely informative, as it praises a change without providing any actionable feedback or suggestions. It doesn't align with the rules, which prohibit purely informative comments.

2. instructor/auto_client.py:110

• Draft comment:
  Consistently raising ConfigurationError for missing provider packages (e.g. OpenAI) improves developer guidance. Ensure similar clarity across all provider branches.
• Reason this comment was not posted:
  Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 85% The comment is praising a good practice but also suggesting to ensure consistency across all providers. Looking at the full file, this is actually actionable since many provider blocks still use ImportError instead of ConfigurationError. However, the comment is more congratulatory than actionable, and doesn't clearly state what needs to be changed. The comment might be too vague - it doesn't explicitly point out which provider blocks still need to be updated to use ConfigurationError. Also, it's partially just praise rather than a clear request for changes. While the underlying suggestion is valid, the comment should be more specific about what needs to be changed rather than just praising the current changes. The comment should be deleted because it's more congratulatory than actionable, and doesn't clearly specify what code changes are needed.

3. instructor/client_anthropic.py:68

• Draft comment:
  Replacing assertions with an explicit check and raising ModeError clarifies allowed modes and prevents silent failures in production.
• Reason this comment was not posted:
  Comment looked like it was already resolved.

4. instructor/client_anthropic.py:86

• Draft comment:
  Checking the client type against valid_client_types and raising ClientError is a solid change. Consider verifying that the error message clearly lists expected client class names.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 50% <= threshold 85% The comment suggests verifying the clarity of an error message, which is not allowed as it asks the author to verify something. However, it indirectly suggests improving the error message, which could be useful. The comment could be rephrased to directly suggest improving the error message clarity without asking for verification.

5. instructor/client_vertexai.py:150

• Draft comment:
  Using a similar pattern to raise ModeError if the mode isn’t in the allowed set makes the code consistent and the error messages more descriptive.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 85% This comment is purely informative, praising the use of a consistent pattern for raising ModeError. It doesn't provide any actionable feedback or suggestions for improvement.

6. instructor/exceptions.py:6

• Draft comment:
  Introducing a base InstructorError and having all custom exceptions derive from it is an excellent design choice that simplifies error handling.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 85% This comment is purely informative and does not provide any actionable feedback or suggestions for improvement. It simply praises the introduction of a base InstructorError class, which is not aligned with the rules provided.

7. instructor/function_calls.py:96

• Draft comment:
  Replacing bare re-raises with raising InstructorValidationError (with more context) significantly improves debugging when model validation fails.
• Reason this comment was not posted:
  Comment looked like it was already resolved.

8. instructor/patch.py:72

• Draft comment:
  Good change: using ConfigurationError instead of ValueError when both 'context' and 'validation_context' are provided. This standardizes error types within the library.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 85% This comment is purely informative and does not provide any actionable feedback or suggestions for improvement. It simply acknowledges a change that was made, which violates the rule against making purely informative comments.

9. instructor/process_response.py:207

• Draft comment:
  Replacing an assert with an explicit check that raises ConfigurationError (when stream=True in PARALLEL_TOOLS mode) is much clearer and production‐ready.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 85% This comment is purely informative and does not provide any actionable feedback or suggestions for improvement. It simply praises a change that was made, which violates the rule against making purely informative comments.

10. instructor/retry.py:64

• Draft comment:
  The initialize_retrying function now properly converts an integer into a Tenacity retrying object. This robust handling of the max_retries parameter is a nice improvement.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 85% This comment is purely informative and does not provide any actionable feedback or suggestions for improvement. It simply praises the change without offering any constructive criticism or guidance.

11. mkdocs.yml:235

• Draft comment:
  New navigation entry for 'Error Handling' documentation has been added. This ensures users can easily find info on the new exception hierarchy and error handling best practices.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 85% This comment is purely informative and does not provide any actionable feedback or suggestions. It simply acknowledges a change that was made, which violates the rule against making purely informative comments.

Workflow ID: wflow_NTHib0ON4ak5aveZ

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}

[ellipsis-dev]

Important

Looks good to me! 👍

Reviewed 488455c in 1 minute and 40 seconds. Click for details.

• Reviewed 369 lines of code in 10 files
• Skipped 0 files when reviewing.
• Skipped posting 6 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. instructor/client_bedrock.py:45

• Draft comment:
  Dynamic exception imports are repeated; if possible, centralize these imports (if no circular dependency issues) to reduce redundancy.
• Reason this comment was not posted:
  Confidence changes required: 33% <= threshold 85% None

2. instructor/client_cohere.py:59

• Draft comment:
  Verify that using 'client.chat' as the 'create' function aligns with Cohere’s API, as other clients use a more specific completions method.
• Reason this comment was not posted:
  Confidence changes required: 66% <= threshold 85% None

3. instructor/client_genai.py:57

• Draft comment:
  Using kwargs.pop('stream', False) removes the key from kwargs, which might have unintended side effects. Consider using a copy of kwargs if preserving original parameters is important.
• Reason this comment was not posted:
  Confidence changes required: 66% <= threshold 85% None

4. instructor/client_groq.py:59

• Draft comment:
  Ensure that for Groq’s asynchronous client, using the synchronous 'client.chat.completions.create' is appropriate; if not, consider adding an async wrapper for correct async behavior.
• Reason this comment was not posted:
  Confidence changes required: 66% <= threshold 85% None

5. instructor/client_mistral.py:61

• Draft comment:
  Popping 'stream' from kwargs modifies the original arguments. Consider copying kwargs to avoid unintended side effects for callers.
• Reason this comment was not posted:
  Confidence changes required: 66% <= threshold 85% None

6. instructor/client_writer.py:53

• Draft comment:
  Confirm that 'client.chat.chat' is the correct method for the Writer client, since this naming deviates from patterns used in other clients (e.g. using completions endpoints).
• Reason this comment was not posted:
  Confidence changes required: 66% <= threshold 85% None

Workflow ID: wflow_uaaFedTjNBNOYkGP

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}

[ellipsis-dev]

Important

Looks good to me! 👍

Reviewed 84ab2be in 1 minute and 0 seconds. Click for details.

• Reviewed 25 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 2 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. tests/test_auto_client.py:89

• Draft comment:
  The test now imports and expects ConfigurationError (instead of ValueError), which aligns with the new exception hierarchy. Consider moving the import to the module-level to follow DRY principles.
• Reason this comment was not posted:
  Confidence changes required: 80% <= threshold 85% None

2. tests/test_auto_client.py:98

• Draft comment:
  The test now expects ConfigurationError for an unsupported provider. As above, a module-level import of ConfigurationError would reduce repetition.
• Reason this comment was not posted:
  Confidence changes required: 80% <= threshold 85% None

Workflow ID: wflow_PG2ewheV9GoLUlTR

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}