New user with a few suggestions! #6826
Description
Just trying Jan out Last couple of days after hearing a few good things. So I had settled into LM studio. But I saw someone online mention that you, umm, allowed us to use our own LCPP Runtimes. And that is a brilliant feature. Waiting sometimes for LM studio to update their runtimes for model support or bugs etc is Painful. And bravo on addressing that issue. Now I have one or two other suggestions for you guys. I'm using the Windows desktop version by the way.
- Thinking tag parsing:
-
Some AI models will use thinking tags in the middle or at the end of an output, not just in the beginning. And that seems to break your whole parsing system. LM studio can handle it dynamically with multiple separate thinking tag outputs in a single AI message. I'm sure it's simple for you guys to do the same. But I've noticed if an AI model uses it anywhere outside of just the beginning, it breaks the whole reply. An especially nasty bug in this area is if an AI model ends in a think tag output it will not allow a followup message from the user and basically ends the conversation.
-
Allow the user to set the tags for the thinking parsing. Different AI models will use different XML tags to do their thinking. LM Studio has an option to change the word from the standard to whatever the user wants for that specific model, to have it be parsed correctly and hidden cleanly from the output. Another thing I'm sure is simple to address.
- Clear System Prompt Setting:
- Your system of setting Assistants is unique but can be convoluted. The ability to quickly Set a system prompt for the current session, In a visible sidebar would be helpful, similar to LM Studio. I understand the setting system prompts is hidden within the "Assistant's" system, But sometimes people just want to set a simple system prompt for any AI model they're using.
- Prompt Template Transparency:
- Quick ability to see the current Jinja template for the AI model in use. And maybe switch it to something like Chatml or Alpaca etc quickly would be helpful as well.
Those are a few productive and helpful ideas for you guys, and I hope you adopt them. And thank you for your contribution and your work. Overall, I'm impressed so far. And I'll update this with anything else I think of for you and appreciate it.