-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Fix for removing usage in every stream chunk response. #1022
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
1. Make Usage field in completions Response to pointer.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1022 +/- ##
=======================================
Coverage 85.83% 85.83%
=======================================
Files 43 43
Lines 2316 2316
=======================================
Hits 1988 1988
Misses 305 305
Partials 23 23 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
1. Make Usage field in completions Response to pointer. 2. Add omitempty to json tag Signed-off-by: Hritik003 <[email protected]>
|
@sashabaranov can you please look into this? |
|
@sashabaranov A reminder to look into this? |
sashabaranov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
…1022) * Fix for sashabaranov#1021: 1. Make Usage field in completions Response to pointer. * Fix for sashabaranov#1021: 1. Make Usage field in completions Response to pointer. 2. Add omitempty to json tag Signed-off-by: Hritik003 <[email protected]> --------- Signed-off-by: Hritik003 <[email protected]>
Describe the change
This PR fixes an issue where the
usagefield is always present in streamed completion responses, even whenstream_options.include_usageis explicitly set tofalse.By changing the
Usagefield inCompletionResponseto a pointer and tagging it withomitempty, we ensure that ifusageis not included in the actual API response, it will be omitted from the parsed Go struct as well.This improves correctness and reduces confusion when users want to opt out of usage metadata in streaming mode.
Provide OpenAI documentation link
API Reference: https://platform.openai.com/docs/api-reference/completions/create
Describe your solution
Modified the
CompletionResponsestruct to update theUsagefield:This allows Go’s
encoding/jsonto omit the field when it's not present in the streamed response. This makes the client-side response cleaner and honors the behavior expected wheninclude_usageis set tofalse.No new libraries or dependencies were added.
Tests
Manually tested by calling
/v1/completionswith bothstream_options.include_usage: trueandfalseVerified that:
include_usage: true, the field is present.include_usage: false, the field is omitted in streamed chunks.Updated integration tests to assert presence/absence of
Usagefield conditionally based on stream option.Additional context
This PR improves alignment with OpenAI’s API behavior and helps consumers avoid parsing unnecessary fields.
Issue: #1021