Skip to content

fix(openai): reask functionality broken in JSON mode since v1.9.0#1793

Merged
jxnl merged 3 commits into567-labs:mainfrom
pnkvalavala:fix/openai-json-mode-retry-on-parse-error
Sep 5, 2025
Merged

fix(openai): reask functionality broken in JSON mode since v1.9.0#1793
jxnl merged 3 commits into567-labs:mainfrom
pnkvalavala:fix/openai-json-mode-retry-on-parse-error

Conversation

@pnkvalavala
Copy link
Copy Markdown
Contributor

@pnkvalavala pnkvalavala commented Aug 29, 2025

Since v1.9.0, from this PR: #1549, the reask functionality with JSON mode is broken.

Irrespective of whether the underlying error is a JSONDecodeError or ValidationError, both are being wrapped under instructor.core.exceptions.ValidationError

Hence in retry_sync/retry_async of instructor/core/retry.py, these wrapped errors are always caught by the generic Exception block instead of the specific (ValidationError, JSONDecodeError) block, even when those specific exception types should trigger retries with error messages included


Important

Fixes reask functionality in JSON mode by removing exception wrapping in _validate_model_from_json() to allow specific exception handling.

  • Behavior:
    • Fixes reask functionality in JSON mode by removing exception wrapping in _validate_model_from_json() in function_calls.py.
    • Allows JSONDecodeError and ValidationError to be caught specifically in retry_sync/retry_async in retry.py.
  • Exceptions:
    • Removes wrapping of exceptions in instructor.core.exceptions.ValidationError in _validate_model_from_json().
  • Logging:
    • Retains debug logging for JSON decode and model validation errors in _validate_model_from_json().

This description was created by Ellipsis for b6a47d9. You can customize this summary. It will automatically update as commits are pushed.

Copy link
Copy Markdown
Contributor

@ellipsis-dev ellipsis-dev Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed everything up to b6a47d9 in 39 seconds. Click for details.
  • Reviewed 18 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. instructor/processing/function_calls.py:97
  • Draft comment:
    By removing the wrapping in a custom InstructorValidationError and simply re-raising the original exception, the original error types (e.g. JSONDecodeError or ValidationError) are correctly propagated. This addresses the reask issue in JSON mode. Ensure tests verify that downstream retry logic now catches these specific exceptions as intended.
  • Reason this comment was not posted:
    Comment did not seem useful. Confidence is useful = 80% <= threshold 85% The comment is informative and suggests ensuring that tests verify the new behavior. It doesn't directly ask the author to confirm their intention or update the PR description. However, it does suggest a specific action related to testing, which is acceptable. The comment is useful as it highlights a change in exception handling and its implications on downstream logic.

Workflow ID: wflow_AP6rXmB7MJQgK42n

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

@jxnl jxnl merged commit 2f9c043 into 567-labs:main Sep 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants