Using variables for system prompt in prompt.yml?

[mrochon]

Select Topic Area

Question

Body

Is there any way to use a variable, e.g. {{myField}} in the system prompt in the yml file? I would like to test and compare my prompts using the comparer functionality but at run-time my code needs to dynamically replace certain values in the system prompt. I can obviously have my code look and replace a fixed value used in the yaml prompt but that make for somewhat in-elegant code. Here is what I would ideally like to do:

...
messages:
  - role: system
    content: >-
      Create oData filter expressions using {{field_names}}...

  - variables:
    filter_names: Age, Name, ...

[KateCatlin]

Hey there @mrochon! Quick clarifying question: are you asking whether you can include a variable in the system prompt that gets dynamically updated at runtime in your production app, or whether you can dynamically swap out variable values while editing and evaluating the prompt in our UI?

If it's the former, yes, absolutely! You can use variables in the system prompt and feed in real-time data at runtime. Here's a demo video:

field.demo.mp4

And here's a sample .prompt.yml file that demonstrates this, with some placeholder testData you can play around with in the prompt editor or evaluator:

messages:
  - role: system
    content: >-
      You are a helpful assistant that writes oData filter expressions. Use the
      following field names: {{field_names}}
  - role: user
    content: |
      Please create oData filters using those fields for a basic query.
model: openai/gpt-4o
testData:
  - field_names: 'Age, name'
  - field_names: 'Age, name, height'
evaluators: []

In your app, you’d use this prompt and dynamically inject the value for field_names based on your context.

But if you’re asking about pulling in live data (e.g. calling an API) to populate the variable while editing the prompt in our UI, that’s not supported yet! Variables need to be filled in with test data (either uploaded via CSV, saved in your .prompt.yml, or input manually) when testing via the UI.

[rahulps1000]

Great question! Currently, most YAML-based prompt configuration systems—including those used in GitHub Actions and many LLM integration tools—do not natively support variable interpolation within the system prompt using syntax like {{myField}}. That means placing {{field_names}} directly in your YAML will not automatically substitute the variable at runtime unless the tool you are using explicitly supports this feature.

Workarounds and Best Practices

• Manual Replacement in Code: As you mentioned, you can preprocess the YAML or the system prompt in your code before passing it to the model, replacing placeholders like {{field_names}} with the actual values. While this adds some boilerplate, it’s the most common and reliable approach.
• Comparer-Friendly Approach: To keep your comparer functionality clean, you might consider using a unique placeholder (e.g., __FIELD_NAMES__ or [[FIELD_NAMES]]) in your YAML and documenting this convention for your team. Your code can then consistently replace this marker at runtime.
• Templating Engines: Some advanced pipelines allow for templating using Jinja2 or similar engines, which support variable interpolation. If your workflow allows, integrating a templating step can make this process more elegant.
• Feature Requests: If you’re using a specific tool or library that does not support this but could benefit from it, consider submitting a feature request to the maintainers.

Example (Manual Replacement)

messages:
  - role: system
    content: >-
      Create oData filter expressions using [[FIELD_NAMES]]...

# Your code:
system_prompt = yaml_content.replace('[[FIELD_NAMES]]', ', '.join(field_names))

Summary:
At this time, you’ll need to handle variable replacement in your application logic or look into external templating as a pre-processing step. Direct variable interpolation in YAML prompts is not widely supported out-of-the-box.

If you’re using a specific tool or GitHub Action that claims to support this feature, please share its name—there may be tool-specific solutions or workarounds!