Add customizable "Merge Instruction" for Beam Mode to AI Persona

[Slava-256]

What's happening?

Problem:

Standard Merge options in Beam Mode do not take into account AI Persona Chat System Instruction. This may result in a loss of additional steering and context provided by AI Persona when performing merges.

It is understandable that proper incorporation of AI Persona system prompt into merge instruction may be difficult to do in an automated way, as AI Persona instruction may conflict with merge instruction and lead to unpredictable results.

Even if some decent solutions are found to resolve these conflicts without significantly diminishing the quality of the merge or the effectiveness of guidance provided by AI Persona Chat Instruction, there is bound to be some level of compromise in those solutions.

So there will always be a place for manual control over this important piece of functionality in Big-AGI.

Currently there is an option to do "Custom" merge in Beam Mode, but this customization is performed on a per-merge or per chat-turn basis at best. If my earlier request to make edits to Custom merge instructions in Beam Mode persistent (#929) is implemented, that would improve the usability of this feature as a way of experimenting with custom merges, but I think proper integration of AI Persona with merge instruction is both important, as well as very common type of merge customization, that it will benefit from a targeted solution.

Proposed Solution:

I can think of 3 ways to address this:

1. Allow AI Persona modify Fusion merge instruction.

Pros: Simplest to implement. Simplest UI.
Cons: Users will loose access to standard/default Fusion functionality. Reduced clarity about what "Fuse" button means. Confusion as to why Fusion is customized but Guided and Compare modes are not.

2. Allow AI Persona modify Fuse/Guided/Compare merge instructions.

Pros: Consistency due to all Merge modes being customizable.
Cons: Bulky UI in AI Persona editing dialog. Users will loose access to standard/default Fuse/Guided/Compare functionality. Reduced clarity about what Fuse/Guided/Compare buttons mean. More code needed to handle more interface elements and more variables that are customized and need to be piped through.

3. Allow AI Persona so specify AI Persona's merge instruction as a separate merge mode (and a new button) in addition to Fuse/Guided/Compare/Custom modes currently available.

Pros: UI Clarity
Cons: May be somewhat more difficult to implement if you want to dynamically add or remove AI Persona's merge mode button depending on whether it was customized by AI Persona or not.

Overall I think option 3 strikes better balance between UI simplicity, clarity, and implementation difficulty.

I propose the following:

• In AI Personas->Editing dialog, rename "Instruction" tab into "Chat Instruction".

• Add "Merge Instruction" tab that allows specifying "System Instruction:" and "User Instruction:" similar to the window you see when you add Custom merge in Beam Mode. Additional text on the "Merge Instruction" tab can explain the purpose of these instructions.

• In Beam Mode, alongside Fuse/Guided/Compare/Custom buttons, add a button titled "<AI Persona's Name> Merge" and place it between "Compare" and "Custom" buttons.

• If it is not too difficult, make "<AI Persona's Name> Merge" button appear only if Merge Instruction was specified for current AI Persona.

Where does this happen?

Big-AGI Pro (big-agi.com)

Impact on your workflow

Medium - Workaround exists

Environment (if applicable)

No response

Additional context

No response

[enricoros]

Very well put together proposal @Slava-256 with good considerations for the UX.

One other option could be to have a checkbox, pre-checked (true), that says "Use Chat instructions for Beam Merges". However I think I like your ideas more.

Let me know if you have more inputs.

[Slava-256]

I think using Chat Instructions as is for Beam Merges is more of a research topic at the moment. If you try to combine merge instruction and AI Persona Chat Instruction in a single system prompt without any modifications, you may end up with Merge instruction competing and/or contradicting Chat instruction, leading to unpredictable results.

If you try to preface Chat Instruction with some text explaining that the following provides context and is not to be followed, or try to place original Chat Instruction in a separate user message, this may dilute or change the effect of guidance provided in a chat instruction. This may be an interesting area for experimentation, but it is difficult to imagine an automated way of fusing merge and chat instruction without some kind of compromise.

And if compromise is to be made, then I would prefer manual control over it by being able to rephrase my chat instruction so that it does not compete with merge instruction, while still providing the guidance and context that I intended.

[Slava-256]

Also, instead of "<AI Persona's Name> Merge" button appearing or disappearing depending on whether Merge Instruction was specified for current AI Persona, it may be easier to have the button always present but grey it out when Merge Instruction for current AI Persona is not specified.