Skip to content

Conversation

@M0Rf30
Copy link
Collaborator

@M0Rf30 M0Rf30 commented Jan 25, 2025

Description

This PR fixes deepseek-r1 prompt. </think> splitting is missing, I already tried with cutstrings parameter but seems that regex doesn't work.

Notes for Reviewers
This is the entire jinja template

# {% if not add_generation_prompt is defined %}
#   {% set add_generation_prompt = false %}
# {% endif %}
# {% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='', is_first_sp=true) %}
# {%- for message in messages %}
#   {%- if message['role'] == 'system' %}
#     {%- if ns.is_first_sp %}
#       {% set ns.system_prompt = ns.system_prompt + message['content'] %}
#       {% set ns.is_first_sp = false %}
#     {%- else %}
#       {% set ns.system_prompt = ns.system_prompt + '\\n\\n' + message['content'] %}
#     {%- endif %}
#   {%- endif %}
# {%- endfor %}
# {{ bos_token }}{{ ns.system_prompt }}
# {%- for message in messages %}
#   {%- if message['role'] == 'user' %}
#     {%- set ns.is_tool = false -%}
#     {{'<|User|>' + message['content']}}
#   {%- endif %}
#   {%- if message['role'] == 'assistant' and 'tool_calls' in message %}
#     {%- set ns.is_tool = false -%}
#     {%- for tool in message['tool_calls'] %}
#       {%- if not ns.is_first %}
#         {%- if message['content'] is none %}
#           {{'<|Assistant|><|tool▁calls▁begin|><|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}
#         {%- else %}
#           {{'<|Assistant|>' + message['content'] + '<|tool▁calls▁begin|><|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}
#         {%- endif %}
#         {%- set ns.is_first = true -%}
#       {%- else %}
#         {{'\\n' + '<|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}
#       {%- endif %}
#     {%- endfor %}
#   {{'<|tool▁calls▁end|><|end▁of▁sentence|>'}}
#   {%- endif %}
#   {%- if message['role'] == 'assistant' and 'tool_calls' not in message %}
#     {%- if ns.is_tool %}
#       {{'<|tool▁outputs▁end|>' + message['content'] + '<|end▁of▁sentence|>'}}
#       {%- set ns.is_tool = false -%}
#     {%- else %}
#       {% set content = message['content'] %}
#       {% if '</think>' in content %}
#         {% set content = content.split('</think>')[-1] %}
#       {% endif %}
#       {{'<|Assistant|>' + content + '<|end▁of▁sentence|>'}}
#     {%- endif %}
#   {%- endif %}
#   {%- if message['role'] == 'tool' %}
#     {%- set ns.is_tool = true -%}
#     {%- if ns.is_output_first %}
#       {{'<|tool▁outputs▁begin|><|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}
#       {%- set ns.is_output_first = false %}
#     {%- else %}
#       {{'<|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}
#     {%- endif %}
#   {%- endif %}
# {%- endfor -%}
# {% if ns.is_tool %}
#   {{'<|tool▁outputs▁end|>'}}
# {% endif %}
# {% if add_generation_prompt and not ns.is_tool %}
#   {{'<|Assistant|>'}}
# {% endif %}

Signed commits

  • Yes, I signed my commits.

@netlify
Copy link

netlify bot commented Jan 25, 2025

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 217bb2e
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/679448d9b33cb20008a6824a
😎 Deploy Preview https://deploy-preview-4686--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link
Owner

@mudler mudler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good here! 🙏 Thanks!

@mudler mudler merged commit e9cace1 into mudler:master Jan 25, 2025
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants