[BUG] Exceptions on tool calls `tojson(ensure_ascii=False)`

[mratsim]

OS

Linux

GPU Library

CUDA 12.x

Python version

3.12

Describe the bug

TabbyAPI throws an exception TypeError: do_tojson() got an unexpected keyword argument 'ensure_ascii' when using GLM-4.6 model with tool calls

Full stacktrace

/app/common/logger.py:68: SyntaxWarning: invalid escape sequence '\<'
  message = message.replace("{", "{{").replace("}", "}}").replace("<", "\<")
2025-12-19 11:37:52.546 INFO:     Using backend exllamav3
2025-12-19 11:37:52.547 INFO:     exllamav3 version: 0.0.18
2025-12-19 11:37:53.262 WARNING:  The provided model does not have vision 
capabilities that are supported by ExllamaV3. Vision input is disabled.
2025-12-19 11:37:53.262 WARNING:  Draft model is disabled because a model name 
wasn't provided. Please check your config.yml!
2025-12-19 11:37:53.267 INFO:     Attempting to load a prompt template if 
present.
2025-12-19 11:37:53.277 INFO:     Using template "chat_template" for chat 
completions.
2025-12-19 11:37:53.278 INFO:     Loading model: 
/app/models/glm-4.6-exl3-3.84bpw-tuned
2025-12-19 11:37:53.279 INFO:     Loading with tensor parallel
2025-12-19 11:38:28.798 INFO:     Model successfully loaded.
Loading model modules ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 95/95 0:00:00
2025-12-19 11:38:28.799 WARNING:  Disabling authentication makes your instance 
vulnerable. Set the `disable_auth` flag to False in config.yml if you want to 
share this instance with others.
2025-12-19 11:38:28.801 INFO:     Generation logging is disabled
2025-12-19 11:38:28.802 INFO:     Applied sampler overrides from file.
2025-12-19 11:38:28.803 INFO:     Developer documentation: 
http://0.0.0.0:5000/redoc
2025-12-19 11:38:28.805 INFO:     Starting OAI API
2025-12-19 11:38:28.806 INFO:     Completions: 
http://0.0.0.0:5000/v1/completions
2025-12-19 11:38:28.806 INFO:     Chat completions: 
http://0.0.0.0:5000/v1/chat/completions
2025-12-19 11:38:28.819 INFO:     Starting KoboldAI API
2025-12-19 11:38:28.819 INFO:     Generation: 
http://0.0.0.0:5000/api/v1/generate
2025-12-19 11:38:28.819 INFO:     Streaming: 
http://0.0.0.0:5000/api/extra/generate/stream
2025-12-19 11:38:28.837 INFO:     Started server process [1]
2025-12-19 11:38:28.837 INFO:     Waiting for application startup.
2025-12-19 11:38:28.837 INFO:     Application startup complete.
2025-12-19 11:38:28.838 INFO:     Uvicorn running on http://0.0.0.0:5000 (Press 
CTRL+C to quit)
2025-12-19 11:39:40.899 INFO:     Received chat completion request 
153952e77e7243da9eb0360be79dfbb7
2025-12-19 11:40:10.697 INFO:     Metrics (ID: 
153952e77e7243da9eb0360be79dfbb7): 707 tokens generated in 29.79 seconds (Queue:
0.0 s, Process: 0 cached tokens and 9 new tokens at 39.13 T/s, Generate: 23.92 
T/s, Context: 9 tokens) 
2025-12-19 11:40:10.698 INFO:     Finished chat completion request 
153952e77e7243da9eb0360be79dfbb7
2025-12-19 11:40:10.698 INFO:     hostname_redacted:39072 - "POST 
/v1/chat/completions HTTP/1.1" 200
2025-12-19 11:40:57.525 INFO:     hostname_redacted:45398 - "POST 
/v1/chat/completions HTTP/1.1" 500
2025-12-19 11:40:57.529 ERROR:    Exception in ASGI application
2025-12-19 11:40:57.529 ERROR:    Traceback (most recent call last):
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py
", line 409, in run_asgi
2025-12-19 11:40:57.529 ERROR:        result = await app(  # type: 
ignore[func-returns-value]
2025-12-19 11:40:57.529 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", 
line 60, in __call__
2025-12-19 11:40:57.529 ERROR:        return await self.app(scope, receive, 
send)
2025-12-19 11:40:57.529 ERROR:               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/applications.py", line 1139, in 
__call__
2025-12-19 11:40:57.529 ERROR:        await super().__call__(scope, receive, 
send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in
__call__
2025-12-19 11:40:57.529 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
186, in __call__
2025-12-19 11:40:57.529 ERROR:        raise exc
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
164, in __call__
2025-12-19 11:40:57.529 ERROR:        await self.app(scope, receive, _send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, 
in __call__
2025-12-19 11:40:57.529 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", 
line 63, in __call__
2025-12-19 11:40:57.529 ERROR:        await 
wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:40:57.529 ERROR:        raise exc
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:40:57.529 ERROR:        await app(scope, receive, sender)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", 
line 18, in __call__
2025-12-19 11:40:57.529 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in 
__call__
2025-12-19 11:40:57.529 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
2025-12-19 11:40:57.529 ERROR:        await route.handle(scope, receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in 
handle
2025-12-19 11:40:57.529 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
2025-12-19 11:40:57.529 ERROR:        await wrap_app_handling_exceptions(app, 
request)(scope, receive, send)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:40:57.529 ERROR:        raise exc
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:40:57.529 ERROR:        await app(scope, receive, sender)
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 106, in app
2025-12-19 11:40:57.529 ERROR:        response = await f(request)
2025-12-19 11:40:57.529 ERROR:                   ^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 430, in app
2025-12-19 11:40:57.529 ERROR:        raw_response = await 
run_endpoint_function(
2025-12-19 11:40:57.529 ERROR:                       
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 316, in 
run_endpoint_function
2025-12-19 11:40:57.529 ERROR:        return await dependant.call(**values)
2025-12-19 11:40:57.529 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File "/app/endpoints/OAI/router.py", line 
126, in chat_completion_request
2025-12-19 11:40:57.529 ERROR:        prompt, embeddings = await 
apply_chat_template(data)
2025-12-19 11:40:57.529 ERROR:                             
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 267, in apply_chat_template
2025-12-19 11:40:57.529 ERROR:        prompt, mm_embeddings, template_vars = 
await format_messages_with_template(
2025-12-19 11:40:57.529 ERROR:                                               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 245, in 
format_messages_with_template
2025-12-19 11:40:57.529 ERROR:        prompt = await 
model.container.prompt_template.render(template_vars)
2025-12-19 11:40:57.529 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File "/app/common/templating.py", line 92, 
in render
2025-12-19 11:40:57.529 ERROR:        rendered_template = await 
self.template.render_async(**template_vars)
2025-12-19 11:40:57.529 ERROR:                            
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 1318, in 
render_async
2025-12-19 11:40:57.529 ERROR:        return self.environment.handle_exception()
2025-12-19 11:40:57.529 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.529 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in 
handle_exception
2025-12-19 11:40:57.529 ERROR:        raise 
rewrite_traceback_stack(source=source)
2025-12-19 11:40:57.529 ERROR:      File "<template>", line 11, in top-level 
template code
2025-12-19 11:40:57.529 ERROR:    TypeError: do_tojson() got an unexpected 
keyword argument 'ensure_ascii'
2025-12-19 11:40:57.949 INFO:     hostname_redacted:45410 - "POST 
/v1/chat/completions HTTP/1.1" 500
2025-12-19 11:40:57.952 ERROR:    Exception in ASGI application
2025-12-19 11:40:57.952 ERROR:    Traceback (most recent call last):
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py
", line 409, in run_asgi
2025-12-19 11:40:57.952 ERROR:        result = await app(  # type: 
ignore[func-returns-value]
2025-12-19 11:40:57.952 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", 
line 60, in __call__
2025-12-19 11:40:57.952 ERROR:        return await self.app(scope, receive, 
send)
2025-12-19 11:40:57.952 ERROR:               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/applications.py", line 1139, in 
__call__
2025-12-19 11:40:57.952 ERROR:        await super().__call__(scope, receive, 
send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in
__call__
2025-12-19 11:40:57.952 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
186, in __call__
2025-12-19 11:40:57.952 ERROR:        raise exc
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
164, in __call__
2025-12-19 11:40:57.952 ERROR:        await self.app(scope, receive, _send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, 
in __call__
2025-12-19 11:40:57.952 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", 
line 63, in __call__
2025-12-19 11:40:57.952 ERROR:        await 
wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:40:57.952 ERROR:        raise exc
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:40:57.952 ERROR:        await app(scope, receive, sender)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", 
line 18, in __call__
2025-12-19 11:40:57.952 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in 
__call__
2025-12-19 11:40:57.952 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
2025-12-19 11:40:57.952 ERROR:        await route.handle(scope, receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in 
handle
2025-12-19 11:40:57.952 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
2025-12-19 11:40:57.952 ERROR:        await wrap_app_handling_exceptions(app, 
request)(scope, receive, send)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:40:57.952 ERROR:        raise exc
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:40:57.952 ERROR:        await app(scope, receive, sender)
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 106, in app
2025-12-19 11:40:57.952 ERROR:        response = await f(request)
2025-12-19 11:40:57.952 ERROR:                   ^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 430, in app
2025-12-19 11:40:57.952 ERROR:        raw_response = await 
run_endpoint_function(
2025-12-19 11:40:57.952 ERROR:                       
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 316, in 
run_endpoint_function
2025-12-19 11:40:57.952 ERROR:        return await dependant.call(**values)
2025-12-19 11:40:57.952 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File "/app/endpoints/OAI/router.py", line 
126, in chat_completion_request
2025-12-19 11:40:57.952 ERROR:        prompt, embeddings = await 
apply_chat_template(data)
2025-12-19 11:40:57.952 ERROR:                             
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 267, in apply_chat_template
2025-12-19 11:40:57.952 ERROR:        prompt, mm_embeddings, template_vars = 
await format_messages_with_template(
2025-12-19 11:40:57.952 ERROR:                                               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 245, in 
format_messages_with_template
2025-12-19 11:40:57.952 ERROR:        prompt = await 
model.container.prompt_template.render(template_vars)
2025-12-19 11:40:57.952 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File "/app/common/templating.py", line 92, 
in render
2025-12-19 11:40:57.952 ERROR:        rendered_template = await 
self.template.render_async(**template_vars)
2025-12-19 11:40:57.952 ERROR:                            
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 1318, in 
render_async
2025-12-19 11:40:57.952 ERROR:        return self.environment.handle_exception()
2025-12-19 11:40:57.952 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:57.952 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in 
handle_exception
2025-12-19 11:40:57.952 ERROR:        raise 
rewrite_traceback_stack(source=source)
2025-12-19 11:40:57.952 ERROR:      File "<template>", line 11, in top-level 
template code
2025-12-19 11:40:57.952 ERROR:    TypeError: do_tojson() got an unexpected 
keyword argument 'ensure_ascii'
2025-12-19 11:40:58.828 INFO:     hostname_redacted:45424 - "POST 
/v1/chat/completions HTTP/1.1" 500
2025-12-19 11:40:58.830 ERROR:    Exception in ASGI application
2025-12-19 11:40:58.830 ERROR:    Traceback (most recent call last):
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py
", line 409, in run_asgi
2025-12-19 11:40:58.830 ERROR:        result = await app(  # type: 
ignore[func-returns-value]
2025-12-19 11:40:58.830 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", 
line 60, in __call__
2025-12-19 11:40:58.830 ERROR:        return await self.app(scope, receive, 
send)
2025-12-19 11:40:58.830 ERROR:               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/applications.py", line 1139, in 
__call__
2025-12-19 11:40:58.830 ERROR:        await super().__call__(scope, receive, 
send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in
__call__
2025-12-19 11:40:58.830 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
186, in __call__
2025-12-19 11:40:58.830 ERROR:        raise exc
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
164, in __call__
2025-12-19 11:40:58.830 ERROR:        await self.app(scope, receive, _send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, 
in __call__
2025-12-19 11:40:58.830 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", 
line 63, in __call__
2025-12-19 11:40:58.830 ERROR:        await 
wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:40:58.830 ERROR:        raise exc
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:40:58.830 ERROR:        await app(scope, receive, sender)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", 
line 18, in __call__
2025-12-19 11:40:58.830 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in 
__call__
2025-12-19 11:40:58.830 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
2025-12-19 11:40:58.830 ERROR:        await route.handle(scope, receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in 
handle
2025-12-19 11:40:58.830 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
2025-12-19 11:40:58.830 ERROR:        await wrap_app_handling_exceptions(app, 
request)(scope, receive, send)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:40:58.830 ERROR:        raise exc
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:40:58.830 ERROR:        await app(scope, receive, sender)
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 106, in app
2025-12-19 11:40:58.830 ERROR:        response = await f(request)
2025-12-19 11:40:58.830 ERROR:                   ^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 430, in app
2025-12-19 11:40:58.830 ERROR:        raw_response = await 
run_endpoint_function(
2025-12-19 11:40:58.830 ERROR:                       
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 316, in 
run_endpoint_function
2025-12-19 11:40:58.830 ERROR:        return await dependant.call(**values)
2025-12-19 11:40:58.830 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File "/app/endpoints/OAI/router.py", line 
126, in chat_completion_request
2025-12-19 11:40:58.830 ERROR:        prompt, embeddings = await 
apply_chat_template(data)
2025-12-19 11:40:58.830 ERROR:                             
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 267, in apply_chat_template
2025-12-19 11:40:58.830 ERROR:        prompt, mm_embeddings, template_vars = 
await format_messages_with_template(
2025-12-19 11:40:58.830 ERROR:                                               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 245, in 
format_messages_with_template
2025-12-19 11:40:58.830 ERROR:        prompt = await 
model.container.prompt_template.render(template_vars)
2025-12-19 11:40:58.830 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File "/app/common/templating.py", line 92, 
in render
2025-12-19 11:40:58.830 ERROR:        rendered_template = await 
self.template.render_async(**template_vars)
2025-12-19 11:40:58.830 ERROR:                            
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 1318, in 
render_async
2025-12-19 11:40:58.830 ERROR:        return self.environment.handle_exception()
2025-12-19 11:40:58.830 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:40:58.830 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in 
handle_exception
2025-12-19 11:40:58.830 ERROR:        raise 
rewrite_traceback_stack(source=source)
2025-12-19 11:40:58.830 ERROR:      File "<template>", line 11, in top-level 
template code
2025-12-19 11:40:58.830 ERROR:    TypeError: do_tojson() got an unexpected 
keyword argument 'ensure_ascii'
2025-12-19 11:41:02.714 INFO:     hostname_redacted:45438 - "POST 
/v1/chat/completions HTTP/1.1" 500
2025-12-19 11:41:02.716 ERROR:    Exception in ASGI application
2025-12-19 11:41:02.716 ERROR:    Traceback (most recent call last):
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py
", line 409, in run_asgi
2025-12-19 11:41:02.716 ERROR:        result = await app(  # type: 
ignore[func-returns-value]
2025-12-19 11:41:02.716 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", 
line 60, in __call__
2025-12-19 11:41:02.716 ERROR:        return await self.app(scope, receive, 
send)
2025-12-19 11:41:02.716 ERROR:               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/applications.py", line 1139, in 
__call__
2025-12-19 11:41:02.716 ERROR:        await super().__call__(scope, receive, 
send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in
__call__
2025-12-19 11:41:02.716 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
186, in __call__
2025-12-19 11:41:02.716 ERROR:        raise exc
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
164, in __call__
2025-12-19 11:41:02.716 ERROR:        await self.app(scope, receive, _send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, 
in __call__
2025-12-19 11:41:02.716 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", 
line 63, in __call__
2025-12-19 11:41:02.716 ERROR:        await 
wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:41:02.716 ERROR:        raise exc
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:41:02.716 ERROR:        await app(scope, receive, sender)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", 
line 18, in __call__
2025-12-19 11:41:02.716 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in 
__call__
2025-12-19 11:41:02.716 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
2025-12-19 11:41:02.716 ERROR:        await route.handle(scope, receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in 
handle
2025-12-19 11:41:02.716 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
2025-12-19 11:41:02.716 ERROR:        await wrap_app_handling_exceptions(app, 
request)(scope, receive, send)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:41:02.716 ERROR:        raise exc
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:41:02.716 ERROR:        await app(scope, receive, sender)
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 106, in app
2025-12-19 11:41:02.716 ERROR:        response = await f(request)
2025-12-19 11:41:02.716 ERROR:                   ^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 430, in app
2025-12-19 11:41:02.716 ERROR:        raw_response = await 
run_endpoint_function(
2025-12-19 11:41:02.716 ERROR:                       
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 316, in 
run_endpoint_function
2025-12-19 11:41:02.716 ERROR:        return await dependant.call(**values)
2025-12-19 11:41:02.716 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File "/app/endpoints/OAI/router.py", line 
126, in chat_completion_request
2025-12-19 11:41:02.716 ERROR:        prompt, embeddings = await 
apply_chat_template(data)
2025-12-19 11:41:02.716 ERROR:                             
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 267, in apply_chat_template
2025-12-19 11:41:02.716 ERROR:        prompt, mm_embeddings, template_vars = 
await format_messages_with_template(
2025-12-19 11:41:02.716 ERROR:                                               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 245, in 
format_messages_with_template
2025-12-19 11:41:02.716 ERROR:        prompt = await 
model.container.prompt_template.render(template_vars)
2025-12-19 11:41:02.716 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File "/app/common/templating.py", line 92, 
in render
2025-12-19 11:41:02.716 ERROR:        rendered_template = await 
self.template.render_async(**template_vars)
2025-12-19 11:41:02.716 ERROR:                            
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 1318, in 
render_async
2025-12-19 11:41:02.716 ERROR:        return self.environment.handle_exception()
2025-12-19 11:41:02.716 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:02.716 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in 
handle_exception
2025-12-19 11:41:02.716 ERROR:        raise 
rewrite_traceback_stack(source=source)
2025-12-19 11:41:02.716 ERROR:      File "<template>", line 11, in top-level 
template code
2025-12-19 11:41:02.716 ERROR:    TypeError: do_tojson() got an unexpected 
keyword argument 'ensure_ascii'
2025-12-19 11:41:03.184 INFO:     hostname_redacted:64306 - "POST 
/v1/chat/completions HTTP/1.1" 500
2025-12-19 11:41:03.185 ERROR:    Exception in ASGI application
2025-12-19 11:41:03.185 ERROR:    Traceback (most recent call last):
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py
", line 409, in run_asgi
2025-12-19 11:41:03.185 ERROR:        result = await app(  # type: 
ignore[func-returns-value]
2025-12-19 11:41:03.185 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", 
line 60, in __call__
2025-12-19 11:41:03.185 ERROR:        return await self.app(scope, receive, 
send)
2025-12-19 11:41:03.185 ERROR:               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/applications.py", line 1139, in 
__call__
2025-12-19 11:41:03.185 ERROR:        await super().__call__(scope, receive, 
send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in
__call__
2025-12-19 11:41:03.185 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
186, in __call__
2025-12-19 11:41:03.185 ERROR:        raise exc
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
164, in __call__
2025-12-19 11:41:03.185 ERROR:        await self.app(scope, receive, _send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, 
in __call__
2025-12-19 11:41:03.185 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", 
line 63, in __call__
2025-12-19 11:41:03.185 ERROR:        await 
wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:41:03.185 ERROR:        raise exc
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:41:03.185 ERROR:        await app(scope, receive, sender)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", 
line 18, in __call__
2025-12-19 11:41:03.185 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in 
__call__
2025-12-19 11:41:03.185 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
2025-12-19 11:41:03.185 ERROR:        await route.handle(scope, receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in 
handle
2025-12-19 11:41:03.185 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
2025-12-19 11:41:03.185 ERROR:        await wrap_app_handling_exceptions(app, 
request)(scope, receive, send)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:41:03.185 ERROR:        raise exc
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:41:03.185 ERROR:        await app(scope, receive, sender)
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 106, in app
2025-12-19 11:41:03.185 ERROR:        response = await f(request)
2025-12-19 11:41:03.185 ERROR:                   ^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 430, in app
2025-12-19 11:41:03.185 ERROR:        raw_response = await 
run_endpoint_function(
2025-12-19 11:41:03.185 ERROR:                       
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 316, in 
run_endpoint_function
2025-12-19 11:41:03.185 ERROR:        return await dependant.call(**values)
2025-12-19 11:41:03.185 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File "/app/endpoints/OAI/router.py", line 
126, in chat_completion_request
2025-12-19 11:41:03.185 ERROR:        prompt, embeddings = await 
apply_chat_template(data)
2025-12-19 11:41:03.185 ERROR:                             
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 267, in apply_chat_template
2025-12-19 11:41:03.185 ERROR:        prompt, mm_embeddings, template_vars = 
await format_messages_with_template(
2025-12-19 11:41:03.185 ERROR:                                               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 245, in 
format_messages_with_template
2025-12-19 11:41:03.185 ERROR:        prompt = await 
model.container.prompt_template.render(template_vars)
2025-12-19 11:41:03.185 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File "/app/common/templating.py", line 92, 
in render
2025-12-19 11:41:03.185 ERROR:        rendered_template = await 
self.template.render_async(**template_vars)
2025-12-19 11:41:03.185 ERROR:                            
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 1318, in 
render_async
2025-12-19 11:41:03.185 ERROR:        return self.environment.handle_exception()
2025-12-19 11:41:03.185 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:03.185 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in 
handle_exception
2025-12-19 11:41:03.185 ERROR:        raise 
rewrite_traceback_stack(source=source)
2025-12-19 11:41:03.185 ERROR:      File "<template>", line 11, in top-level 
template code
2025-12-19 11:41:03.185 ERROR:    TypeError: do_tojson() got an unexpected 
keyword argument 'ensure_ascii'
2025-12-19 11:41:04.162 INFO:     hostname_redacted:64310 - "POST 
/v1/chat/completions HTTP/1.1" 500
2025-12-19 11:41:04.164 ERROR:    Exception in ASGI application
2025-12-19 11:41:04.164 ERROR:    Traceback (most recent call last):
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py
", line 409, in run_asgi
2025-12-19 11:41:04.164 ERROR:        result = await app(  # type: 
ignore[func-returns-value]
2025-12-19 11:41:04.164 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", 
line 60, in __call__
2025-12-19 11:41:04.164 ERROR:        return await self.app(scope, receive, 
send)
2025-12-19 11:41:04.164 ERROR:               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/applications.py", line 1139, in 
__call__
2025-12-19 11:41:04.164 ERROR:        await super().__call__(scope, receive, 
send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in
__call__
2025-12-19 11:41:04.164 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
186, in __call__
2025-12-19 11:41:04.164 ERROR:        raise exc
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 
164, in __call__
2025-12-19 11:41:04.164 ERROR:        await self.app(scope, receive, _send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, 
in __call__
2025-12-19 11:41:04.164 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", 
line 63, in __call__
2025-12-19 11:41:04.164 ERROR:        await 
wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:41:04.164 ERROR:        raise exc
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:41:04.164 ERROR:        await app(scope, receive, sender)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", 
line 18, in __call__
2025-12-19 11:41:04.164 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in 
__call__
2025-12-19 11:41:04.164 ERROR:        await self.middleware_stack(scope, 
receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
2025-12-19 11:41:04.164 ERROR:        await route.handle(scope, receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in 
handle
2025-12-19 11:41:04.164 ERROR:        await self.app(scope, receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
2025-12-19 11:41:04.164 ERROR:        await wrap_app_handling_exceptions(app, 
request)(scope, receive, send)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
53, in wrapped_app
2025-12-19 11:41:04.164 ERROR:        raise exc
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 
42, in wrapped_app
2025-12-19 11:41:04.164 ERROR:        await app(scope, receive, sender)
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 106, in app
2025-12-19 11:41:04.164 ERROR:        response = await f(request)
2025-12-19 11:41:04.164 ERROR:                   ^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 430, in app
2025-12-19 11:41:04.164 ERROR:        raw_response = await 
run_endpoint_function(
2025-12-19 11:41:04.164 ERROR:                       
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/fastapi/routing.py", line 316, in 
run_endpoint_function
2025-12-19 11:41:04.164 ERROR:        return await dependant.call(**values)
2025-12-19 11:41:04.164 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File "/app/endpoints/OAI/router.py", line 
126, in chat_completion_request
2025-12-19 11:41:04.164 ERROR:        prompt, embeddings = await 
apply_chat_template(data)
2025-12-19 11:41:04.164 ERROR:                             
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 267, in apply_chat_template
2025-12-19 11:41:04.164 ERROR:        prompt, mm_embeddings, template_vars = 
await format_messages_with_template(
2025-12-19 11:41:04.164 ERROR:                                               
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/app/endpoints/OAI/utils/chat_completion.py", line 245, in 
format_messages_with_template
2025-12-19 11:41:04.164 ERROR:        prompt = await 
model.container.prompt_template.render(template_vars)
2025-12-19 11:41:04.164 ERROR:                 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File "/app/common/templating.py", line 92, 
in render
2025-12-19 11:41:04.164 ERROR:        rendered_template = await 
self.template.render_async(**template_vars)
2025-12-19 11:41:04.164 ERROR:                            
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 1318, in 
render_async
2025-12-19 11:41:04.164 ERROR:        return self.environment.handle_exception()
2025-12-19 11:41:04.164 ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-12-19 11:41:04.164 ERROR:      File 
"/opt/venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in 
handle_exception
2025-12-19 11:41:04.164 ERROR:        raise 
rewrite_traceback_stack(source=source)
2025-12-19 11:41:04.164 ERROR:      File "<template>", line 11, in top-level 
template code
2025-12-19 11:41:04.164 ERROR:    TypeError: do_tojson() got an unexpected 
keyword argument 'ensure_ascii'

The chat template of GLM-4.6 does include ensure_ascii: https://huggingface.co/zai-org/GLM-4.6/blob/main/chat_template.jinja

[gMASK]<sop>
{%- if tools -%}
<|system|>
# Tools
You may call one or more functions to assist with the user query.
You are provided with function signatures within <tools></tools> XML tags:
<tools>
{% for tool in tools %}
{{ tool | tojson(ensure_ascii=False) }}
{% endfor %}
</tools>
For each function call, output the function name and arguments within the following XML format:
<tool_call>{function-name}
<arg_key>{arg-key-1}</arg_key>
<arg_value>{arg-value-1}</arg_value>
<arg_key>{arg-key-2}</arg_key>
<arg_value>{arg-value-2}</arg_value>
...
</tool_call>{%- endif -%}
{%- macro visible_text(content) -%}
    {%- if content is string -%}
        {{- content }}
    {%- elif content is iterable and content is not mapping -%}
        {%- for item in content -%}
            {%- if item is mapping and item.type == 'text' -%}
                {{- item.text }}
            {%- elif item is string -%}
                {{- item }}
            {%- endif -%}
        {%- endfor -%}
    {%- else -%}
        {{- content }}
    {%- endif -%}
{%- endmacro -%}
{%- set ns = namespace(last_user_index=-1) %}
{%- for m in messages %}
    {%- if m.role == 'user' %}
        {% set ns.last_user_index = loop.index0 -%}
    {%- endif %}
{%- endfor %}
{% for m in messages %}
{%- if m.role == 'user' -%}<|user|>
{{ visible_text(m.content) }}
{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}}
{%- elif m.role == 'assistant' -%}
<|assistant|>
{%- set reasoning_content = '' %}
{%- set content = visible_text(m.content) %}
{%- if m.reasoning_content is string %}
    {%- set reasoning_content = m.reasoning_content %}
{%- else %}
    {%- if '</think>' in content %}
        {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
        {%- set content = content.split('</think>')[-1].lstrip('\n') %}
    {%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_user_index and reasoning_content -%}
{{ '\n<think>' + reasoning_content.strip() +  '</think>'}}
{%- else -%}
{{ '\n<think></think>' }}
{%- endif -%}
{%- if content.strip() -%}
{{ '\n' + content.strip() }}
{%- endif -%}
{% if m.tool_calls %}
{% for tc in m.tool_calls %}
{%- if tc.function %}
    {%- set tc = tc.function %}
{%- endif %}
{{ '\n<tool_call>' + tc.name }}
{% set _args = tc.arguments %}
{% for k, v in _args.items() %}
<arg_key>{{ k }}</arg_key>
<arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>
{% endfor %}
</tool_call>{% endfor %}
{% endif %}
{%- elif m.role == 'tool' -%}
{%- if m.content is string -%}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
    {{- '<|observation|>' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- m.content }}
{{- '\n</tool_response>' }}
{%- else -%}
<|observation|>{% for tr in m.content %}
<tool_response>
{{ tr.output if tr.output is defined else tr }}
</tool_response>{% endfor -%}
{% endif -%}
{%- elif m.role == 'system' -%}
<|system|>
{{ visible_text(m.content) }}
{%- endif -%}
{%- endfor -%}
{%- if add_generation_prompt -%}
    <|assistant|>{{- '\n<think></think>' if (enable_thinking is defined and not enable_thinking) else '' -}}
{%- endif -%}

The docker has the following config

dependencies versions

``` root@9ec625fa246f:/app# pip list Package Version ------------------------- ----------------------- aiofiles 25.1.0 aiohappyeyeballs 2.6.1 aiohttp 3.13.2 aiosignal 1.4.0 annotated-doc 0.0.4 annotated-types 0.7.0 anyio 4.12.0 async-lru 2.0.5 attrs 25.4.0 certifi 2025.11.12 charset-normalizer 3.4.4 click 8.3.1 cramjam 2.11.0 einops 0.8.1 exllamav2 0.3.2+cu128.torch2.8.0 exllamav3 0.0.18+cu128.torch2.8.0 fastapi-slim 0.124.4 fastparquet 2024.11.0 filelock 3.20.1 flash_attn 2.8.3 formatron 0.5.0 frozendict 2.4.7 frozenlist 1.8.0 fsspec 2025.12.0 general_sam 1.0.3 h11 0.16.0 hf_transfer 0.1.9 hf-xet 1.2.0 httptools 0.7.1 huggingface-hub 0.36.0 idna 3.11 infinity_emb 0.0.45 Jinja2 3.1.6 joblib 1.5.3 jsonschema 4.25.1 jsonschema-specifications 2025.9.1 kbnf 0.4.2 loguru 0.7.3 marisa-trie 1.3.1 markdown-it-py 4.0.0 MarkupSafe 3.0.3 mdurl 0.1.2 mpmath 1.3.0 multidict 6.7.0 networkx 3.6.1 ninja 1.13.0 numpy 2.3.5 nvidia-cublas-cu12 12.8.4.1 nvidia-cuda-cupti-cu12 12.8.90 nvidia-cuda-nvrtc-cu12 12.8.93 nvidia-cuda-runtime-cu12 12.8.90 nvidia-cudnn-cu12 9.10.2.21 nvidia-cufft-cu12 11.3.3.83 nvidia-cufile-cu12 1.13.1.3 nvidia-curand-cu12 10.3.9.90 nvidia-cusolver-cu12 11.7.3.90 nvidia-cusparse-cu12 12.5.8.93 nvidia-cusparselt-cu12 0.7.1 nvidia-nccl-cu12 2.27.3 nvidia-nvjitlink-cu12 12.8.93 nvidia-nvtx-cu12 12.8.90 packaging 25.0 pandas 2.3.3 pillow 12.0.0 pip 25.3 propcache 0.4.1 psutil 7.1.3 pydantic 2.11.0 pydantic_core 2.33.0 Pygments 2.19.2 python-dateutil 2.9.0.post0 pytz 2025.2 PyYAML 6.0.3 referencing 0.37.0 regex 2025.11.3 requests 2.32.5 rich 14.2.0 rpds-py 0.30.0 ruamel.yaml 0.18.16 ruamel.yaml.clib 0.2.15 safetensors 0.7.0 scikit-learn 1.8.0 scipy 1.16.3 sentence-transformers 5.2.0 setuptools 80.9.0 six 1.17.0 sse-starlette 3.0.4 starlette 0.50.0 sympy 1.14.0 tabbyAPI 0.0.1 threadpoolctl 3.6.0 tokenizers 0.22.1 torch 2.8.0+cu128 tqdm 4.67.1 transformers 4.57.3 triton 3.4.0 typing_extensions 4.15.0 typing-inspection 0.4.2 tzdata 2025.3 urllib3 2.6.2 uvicorn 0.38.0 uvloop 0.22.1 websockets 15.0.1 yarl 1.22.0 ```

And the jinja2 handling of tojson is there.

https://github.com/pallets/jinja/blob/3.1.6/src/jinja2/filters.py#L1695-L1721

@pass_eval_context
def do_tojson(
    eval_ctx: "EvalContext", value: t.Any, indent: t.Optional[int] = None
) -> Markup:
    """Serialize an object to a string of JSON, and mark it safe to
    render in HTML. This filter is only for use in HTML documents.

    The returned string is safe to render in HTML documents and
    ``<script>`` tags. The exception is in HTML attributes that are
    double quoted; either use single quotes or the ``|forceescape``
    filter.

    :param value: The object to serialize to JSON.
    :param indent: The ``indent`` parameter passed to ``dumps``, for
        pretty-printing the value.

    .. versionadded:: 2.9
    """
    policies = eval_ctx.environment.policies
    dumps = policies["json.dumps_function"]
    kwargs = policies["json.dumps_kwargs"]

    if indent is not None:
        kwargs = kwargs.copy()
        kwargs["indent"] = indent

    return htmlsafe_json_dumps(value, dumps=dumps, **kwargs)

Reproduction steps

Using docker image https://github.com/theroyallab/tabbyAPI/pkgs/container/tabbyapi
SHA256: e04fe69acc6076371ae6bdc69f0158ffdabd1e821f1186c7812512b7c5c07eab
Date: 2025-12-16

Model: https://huggingface.co/mratsim/glm-4.6-exl3/tree/3.84bpw-tuned

Expected behavior

No crash

Logs

No response

Additional context

No response

Acknowledgements

• I have looked for similar issues before submitting this one.
• I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
• I understand that the developers have lives and my issue will be answered when possible.
• I understand the developers of this program are human, and I will ask my questions politely.

[thad0ctor]

This is a pretty easy fix in:

● Update(common/templating.py)                                                 
  ⎿  Added 6 lines                                                             
      105          def raise_exception(message):                               
      106              raise TemplateError(message)                            
      107                                                                      
      108 +        # Custom tojson filter that supports ensure_ascii par       
          +ameter                                                              
      109 +        # Required for compatibility with HuggingFace chat te       
          +mplates                                                             
      110 +        def tojson(value, ensure_ascii=True):                       
      111 +            return json.dumps(value, ensure_ascii=ensure_asci       
          +i)                                                                  
      112 +                                                                    
      113          self.environment.globals["strftime_now"] = strftime_n       
           ow                                                                  
      114          self.environment.globals["raise_exception"] = raise_e       
           xception                                                            
      115 +        self.environment.filters["tojson"] = tojson                 
      116                                                                      
      117          return self.environment.from_string(template_str)           
      118