Replies: 1 comment
-
| ... and the answer should be obvious by now: the parameter needs to be of type 'Json', not 'String'! works great. | 
Beta Was this translation helpful? Give feedback.
                  
                    0 replies
                  
                
            
  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment
  
        
    
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Has anyone been able to enable dynamic (i.e. for each request) setting of the reasoning_effort flag for gpt-oss models when using a remote llama-server provider?
I have added
as a new Assistant parameter. The request does include the parameter:
but seems to be ignored by the provider. ("Thought: Explain simply")
However, when i add the same as a Custom JSON config in llama-server UI client like so:
the request appears to be similar:
and the model here thinks really hard (about 250 tokens versus 3 above).
The main difference seems to be that Jan sends the value as a quoted string: "{"reasoning_effort":"high"}", while the llama-server UI sends it like a JSON object (unquoted): {"reasoning_effort":"high"}.
Beta Was this translation helpful? Give feedback.
All reactions