Skip to content

Commit 5cede03

Browse files
committed
proxy: implement schemaless metadata in model configs
- add Metadata key to ModelConfig - include metadata in /v1/models under llamaswap_meta key - add recursive macro substitution into Metadata - change macros to be any scalar type
1 parent 9c863d2 commit 5cede03

File tree

9 files changed

+461
-51
lines changed

9 files changed

+461
-51
lines changed

CLAUDE.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,8 @@ llama-swap is a light weight, transparent proxy server that provides automatic m
1111

1212
## Testing
1313

14-
- `make test` - does a quick test run. Generally use this
15-
- `make test-all` - does a more extensive run with timeout testing.
16-
- `make all` - builds llama-swap.go and ui for multiple platforms
14+
- `make test-dev` - Use this when making iterative changes. Runs `go test` and `staticcheck`. Fix any static checking errors.
15+
- `make test-all` - runs at the end before completing work. Includes long running concurrency tests.
1716

1817
## Workflow Tasks
1918

Makefile

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,11 @@ proxy/ui_dist/placeholder.txt:
2323
mkdir -p proxy/ui_dist
2424
touch $@
2525

26+
# use cached test results while developing
27+
test-dev: proxy/ui_dist/placeholder.txt
28+
go test -short ./proxy/...
29+
staticcheck ./proxy/... || true
30+
2631
test: proxy/ui_dist/placeholder.txt
2732
go test -short -count=1 ./proxy/...
2833

@@ -82,4 +87,4 @@ release:
8287
git tag "$$new_tag";
8388

8489
# Phony targets
85-
.PHONY: all clean ui mac linux windows simple-responder test test-all
90+
.PHONY: all clean ui mac linux windows simple-responder test test-all test-dev

ai-plans/issue-264-add-metadata.md

Lines changed: 36 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -211,60 +211,60 @@ The metadata will be schemaless, allowing users to define any key-value pairs th
211211

212212
### Configuration Schema Changes
213213

214-
- [ ] Change `MacroList` type from `map[string]string` to `map[string]any` in [proxy/config/config.go:19](proxy/config/config.go#L19)
215-
- [ ] Add `Metadata map[string]any` field to `ModelConfig` struct in [proxy/config/model_config.go:33](proxy/config/model_config.go#L33)
216-
- [ ] Update `validateMacro()` function signature to accept `any` type for values
217-
- [ ] Add validation logic to ensure macro values are scalar types only
214+
- [x] Change `MacroList` type from `map[string]string` to `map[string]any` in [proxy/config/config.go:19](proxy/config/config.go#L19)
215+
- [x] Add `Metadata map[string]any` field to `ModelConfig` struct in [proxy/config/model_config.go:37](proxy/config/model_config.go#L37)
216+
- [x] Update `validateMacro()` function signature to accept `any` type for values
217+
- [x] Add validation logic to ensure macro values are scalar types only
218218

219219
### Macro Substitution Logic
220220

221-
- [ ] Create generic recursive function `substituteMetadataMacros()` to handle `any` types
222-
- [ ] Implement type-preserving direct substitution logic
223-
- [ ] Implement string interpolation with type conversion
224-
- [ ] Handle maps: recursively process all values
225-
- [ ] Handle slices: recursively process all elements
226-
- [ ] Handle scalar types: perform string-based macro substitution if value is string
227-
- [ ] Integrate macro substitution into `LoadConfigFromReader()` after existing macro expansion
228-
- [ ] Update existing macro substitution calls to use merged macros with correct types
221+
- [x] Create generic recursive function `substituteMetadataMacros()` to handle `any` types
222+
- [x] Implement type-preserving direct substitution logic
223+
- [x] Implement string interpolation with type conversion
224+
- [x] Handle maps: recursively process all values
225+
- [x] Handle slices: recursively process all elements
226+
- [x] Handle scalar types: perform string-based macro substitution if value is string
227+
- [x] Integrate macro substitution into `LoadConfigFromReader()` after existing macro expansion
228+
- [x] Update existing macro substitution calls to use merged macros with correct types
229229

230230
### API Response Changes
231231

232-
- [ ] Modify `listModelsHandler()` in [proxy/proxymanager.go:350](proxy/proxymanager.go#L350)
233-
- [ ] Add `llamaswap_meta` field to model records when metadata exists
234-
- [ ] Ensure empty metadata results in omitted `llamaswap_meta` key
235-
- [ ] Verify JSON marshaling preserves all types correctly
232+
- [x] Modify `listModelsHandler()` in [proxy/proxymanager.go:350](proxy/proxymanager.go#L350)
233+
- [x] Add `llamaswap_meta` field to model records when metadata exists
234+
- [x] Ensure empty metadata results in omitted `llamaswap_meta` key
235+
- [x] Verify JSON marshaling preserves all types correctly
236236

237237
### Testing - Config Package
238238

239-
- [ ] Add test for string macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
240-
- [ ] Add test for int macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
241-
- [ ] Add test for float macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
242-
- [ ] Add test for bool macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
243-
- [ ] Add test for string interpolation in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
244-
- [ ] Add test for model-level macro precedence: [proxy/config/config_test.go](proxy/config/config_test.go)
245-
- [ ] Add test for nested structures in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
246-
- [ ] Add test for unknown macro in metadata (should error): [proxy/config/config_test.go](proxy/config/config_test.go)
247-
- [ ] Add test for invalid macro type validation: [proxy/config/config_test.go](proxy/config/config_test.go)
239+
- [x] Add test for string macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
240+
- [x] Add test for int macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
241+
- [x] Add test for float macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
242+
- [x] Add test for bool macros in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
243+
- [x] Add test for string interpolation in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
244+
- [x] Add test for model-level macro precedence: [proxy/config/config_test.go](proxy/config/config_test.go)
245+
- [x] Add test for nested structures in metadata: [proxy/config/config_test.go](proxy/config/config_test.go)
246+
- [x] Add test for unknown macro in metadata (should error): [proxy/config/config_test.go](proxy/config/config_test.go)
247+
- [x] Add test for invalid macro type validation: [proxy/config/config_test.go](proxy/config/config_test.go)
248248

249249
### Testing - Model Config Package
250250

251-
- [ ] Add test cases to [proxy/config/model_config_test.go](proxy/config/model_config_test.go) for metadata unmarshaling
252-
- [ ] Test metadata with various scalar types
253-
- [ ] Test metadata with nested objects and arrays
251+
- [x] Add test cases to [proxy/config/model_config_test.go](proxy/config/model_config_test.go) for metadata unmarshaling
252+
- [x] Test metadata with various scalar types
253+
- [x] Test metadata with nested objects and arrays
254254

255255
### Testing - Proxy Manager
256256

257-
- [ ] Update `TestProxyManager_ListModelsHandler` in [proxy/proxymanager_test.go](proxy/proxymanager_test.go)
258-
- [ ] Add test case for model with metadata
259-
- [ ] Add test case for model without metadata
260-
- [ ] Verify `llamaswap_meta` key presence/absence
261-
- [ ] Verify type preservation in JSON output
262-
- [ ] Verify macro substitution has occurred
257+
- [x] Update `TestProxyManager_ListModelsHandler` in [proxy/proxymanager_test.go](proxy/proxymanager_test.go)
258+
- [x] Add test case for model with metadata
259+
- [x] Add test case for model without metadata
260+
- [x] Verify `llamaswap_meta` key presence/absence
261+
- [x] Verify type preservation in JSON output
262+
- [x] Verify macro substitution has occurred
263263

264264
### Documentation
265265

266-
- [ ] Verify [config.example.yaml](config.example.yaml) already has complete metadata examples (lines 149-171)
267-
- [ ] No additional documentation needed per project instructions
266+
- [x] Verify [config.example.yaml](config.example.yaml) already has complete metadata examples (lines 149-171)
267+
- [x] No additional documentation needed per project instructions
268268

269269
## Known Issues and Considerations
270270

proxy/config/config.go

Lines changed: 91 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ import (
1616

1717
const DEFAULT_GROUP_ID = "(default)"
1818

19-
type MacroList map[string]string
19+
type MacroList map[string]any
2020

2121
type GroupConfig struct {
2222
Swap bool `yaml:"swap"`
@@ -185,11 +185,13 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
185185
// go through model config fields: cmd, cmdStop, proxy, checkEndPoint and replace macros with macro values
186186
for macroName, macroValue := range mergedMacros {
187187
macroSlug := fmt.Sprintf("${%s}", macroName)
188-
modelConfig.Cmd = strings.ReplaceAll(modelConfig.Cmd, macroSlug, macroValue)
189-
modelConfig.CmdStop = strings.ReplaceAll(modelConfig.CmdStop, macroSlug, macroValue)
190-
modelConfig.Proxy = strings.ReplaceAll(modelConfig.Proxy, macroSlug, macroValue)
191-
modelConfig.CheckEndpoint = strings.ReplaceAll(modelConfig.CheckEndpoint, macroSlug, macroValue)
192-
modelConfig.Filters.StripParams = strings.ReplaceAll(modelConfig.Filters.StripParams, macroSlug, macroValue)
188+
// Convert macro value to string for command/string field substitution
189+
macroStr := fmt.Sprintf("%v", macroValue)
190+
modelConfig.Cmd = strings.ReplaceAll(modelConfig.Cmd, macroSlug, macroStr)
191+
modelConfig.CmdStop = strings.ReplaceAll(modelConfig.CmdStop, macroSlug, macroStr)
192+
modelConfig.Proxy = strings.ReplaceAll(modelConfig.Proxy, macroSlug, macroStr)
193+
modelConfig.CheckEndpoint = strings.ReplaceAll(modelConfig.CheckEndpoint, macroSlug, macroStr)
194+
modelConfig.Filters.StripParams = strings.ReplaceAll(modelConfig.Filters.StripParams, macroSlug, macroStr)
193195
}
194196

195197
// enforce ${PORT} used in both cmd and proxy
@@ -234,6 +236,15 @@ func LoadConfigFromReader(r io.Reader) (Config, error) {
234236
}
235237
}
236238

239+
// Apply macro substitution to metadata
240+
if len(modelConfig.Metadata) > 0 {
241+
substitutedMetadata, err := substituteMetadataMacros(modelConfig.Metadata, mergedMacros)
242+
if err != nil {
243+
return Config{}, fmt.Errorf("model %s metadata: %s", modelId, err.Error())
244+
}
245+
modelConfig.Metadata = substitutedMetadata.(map[string]any)
246+
}
247+
237248
config.Models[modelId] = modelConfig
238249
}
239250

@@ -296,7 +307,7 @@ func AddDefaultGroupToConfig(config Config) Config {
296307
}
297308
} else {
298309
// iterate over existing group members and add non-grouped models into the default group
299-
for modelName, _ := range config.Models {
310+
for modelName := range config.Models {
300311
foundModel := false
301312
found:
302313
// search for the model in existing groups
@@ -374,15 +385,24 @@ var (
374385
)
375386

376387
// validateMacro validates macro name and value constraints
377-
func validateMacro(name, value string) error {
388+
func validateMacro(name string, value any) error {
378389
if len(name) >= 64 {
379390
return fmt.Errorf("macro name '%s' exceeds maximum length of 63 characters", name)
380391
}
381392
if !macroNameRegex.MatchString(name) {
382393
return fmt.Errorf("macro name '%s' contains invalid characters, must match pattern ^[a-zA-Z0-9_-]+$", name)
383394
}
384-
if len(value) >= 1024 {
385-
return fmt.Errorf("macro value for '%s' exceeds maximum length of 1024 characters", name)
395+
396+
// Validate that value is a scalar type
397+
switch v := value.(type) {
398+
case string:
399+
if len(v) >= 1024 {
400+
return fmt.Errorf("macro value for '%s' exceeds maximum length of 1024 characters", name)
401+
}
402+
case int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, float32, float64, bool:
403+
// These types are allowed
404+
default:
405+
return fmt.Errorf("macro '%s' has invalid type %T, must be a scalar type (string, int, float, or bool)", name, value)
386406
}
387407

388408
switch name {
@@ -392,3 +412,64 @@ func validateMacro(name, value string) error {
392412

393413
return nil
394414
}
415+
416+
// substituteMetadataMacros recursively substitutes macros in metadata structures
417+
// Direct substitution (key: ${macro}) preserves the macro's type
418+
// Interpolated substitution (key: "text ${macro}") converts to string
419+
func substituteMetadataMacros(value any, macros MacroList) (any, error) {
420+
switch v := value.(type) {
421+
case string:
422+
// Check if this is a direct macro substitution
423+
if strings.HasPrefix(v, "${") && strings.HasSuffix(v, "}") && strings.Count(v, "${") == 1 {
424+
macroName := v[2 : len(v)-1]
425+
if macroValue, exists := macros[macroName]; exists {
426+
return macroValue, nil
427+
}
428+
return nil, fmt.Errorf("unknown macro '${%s}' in metadata", macroName)
429+
}
430+
431+
// Handle string interpolation
432+
macroPattern := regexp.MustCompile(`\$\{([a-zA-Z0-9_-]+)\}`)
433+
matches := macroPattern.FindAllStringSubmatch(v, -1)
434+
result := v
435+
for _, match := range matches {
436+
macroName := match[1]
437+
macroValue, exists := macros[macroName]
438+
if !exists {
439+
return nil, fmt.Errorf("unknown macro '${%s}' in metadata", macroName)
440+
}
441+
// Convert macro value to string for interpolation
442+
macroStr := fmt.Sprintf("%v", macroValue)
443+
result = strings.ReplaceAll(result, match[0], macroStr)
444+
}
445+
return result, nil
446+
447+
case map[string]any:
448+
// Recursively process map values
449+
newMap := make(map[string]any)
450+
for key, val := range v {
451+
newVal, err := substituteMetadataMacros(val, macros)
452+
if err != nil {
453+
return nil, err
454+
}
455+
newMap[key] = newVal
456+
}
457+
return newMap, nil
458+
459+
case []any:
460+
// Recursively process slice elements
461+
newSlice := make([]any, len(v))
462+
for i, val := range v {
463+
newVal, err := substituteMetadataMacros(val, macros)
464+
if err != nil {
465+
return nil, err
466+
}
467+
newSlice[i] = newVal
468+
}
469+
return newSlice, nil
470+
471+
default:
472+
// Return scalar types as-is
473+
return value, nil
474+
}
475+
}

proxy/config/config_posix_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,7 @@ groups:
163163
expected := Config{
164164
LogLevel: "info",
165165
StartPort: 5800,
166-
Macros: map[string]string{
166+
Macros: MacroList{
167167
"svr-path": "path/to/server",
168168
},
169169
Hooks: HooksConfig{

0 commit comments

Comments
 (0)