server : bugfix - stop server from sending empty json during oai chat completions

[m18coppola]

Since #10643, the webui crashes when the model finished generating a response:

Upon investigation, I found that it was because an empty json object is sent from the server before the on_complete json is sent:

$ curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "stream": true,
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'

...

data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"?"}}],"created":1733515737,"id":"chatcmpl-LtMeOt2U3SPq40U172bsTkOzMiL8X4xC","model":"gpt-3.5-turbo-0613","object":"chat.completion.chunk"}

data: {}

data: {"choices":[{"finish_reason":"stop","index":0,"delta":{}}],"created":1733515738,"id":"chatcmpl-LtMeOt2U3SPq40U172bsTkOzMiL8X4xC","model":"gpt-3.5-turbo-0613","object":"chat.completion.chunk","timings":{"prompt_n":1,"prompt_ms":67.807,"prompt_per_token_ms":67.807,"prompt_per_second":14.747739908858966,"predicted_n":25,"predicted_ms":1543.024,"predicted_per_token_ms":61.72096,"predicted_per_second":16.20195149265339},"usage":{"completion_tokens":25,"prompt_tokens":23,"total_tokens":48}}

data: [DONE]

I made a change such that the server skips sending any empty json objects returned from the completion results stream.

[ggerganov]

It might be better to find what is the root cause for having this empty json object and prevent from being created in the first place.

[ggerganov]

I traced it and this empty object is created when an end-of-turn token is generated and the --special CLI arg is not passed to llama-server. In this case, the special token is not rendered and the content is empty.

I think a better solution would be to send a valid object with empty content instead of skipping this message from the stream.