Skip to content

fix: extract evolved skill body from optimized predictor#49

Open
steezkelly wants to merge 2 commits intoNousResearch:mainfrom
steezkelly:fix/ghost-improvement-extraction-focused
Open

fix: extract evolved skill body from optimized predictor#49
steezkelly wants to merge 2 commits intoNousResearch:mainfrom
steezkelly:fix/ghost-improvement-extraction-focused

Conversation

@steezkelly
Copy link
Copy Markdown

@steezkelly steezkelly commented May 5, 2026

Summary

  • embeds the skill body in the DSPy predictor instructions instead of passing it as a normal input field
  • extracts the evolved skill body from the optimized predictor's mutated instruction/docstring surface using a sentinel
  • adds regression coverage proving the saved evolved body changes when the optimized predictor text changes

Scope control

This is the focused split from the broad PR #39 for the ghost-improvement bug only.
It intentionally does not include:

  • provider/model routing changes
  • GEPA metric/threaded LM changes
  • generated output/ artifacts
  • Kanban board-state churn

Test plan

  • /home/steve/hermes0/hermes-agent-self-evolution/venv/bin/python -m pytest tests/skills/test_skill_module.py tests/skills/test_evolve_skill_extraction.py -q
  • /home/steve/hermes0/hermes-agent-self-evolution/venv/bin/python -m pytest -q

Result: 144 passed, 11 warnings.

Issue #38 relationship

This PR addresses the ghost-improvement extraction portion of #38: the optimized predictor must expose the actual evolved skill body, rather than reporting improved scores while saving the baseline body.

It intentionally leaves the remaining #38 follow-ups open for focused PRs:

  • provider/model routing and Nous API integration changes
  • GEPA API/threaded LM compatibility changes
  • constraint validator body-substance checks
  • JSON robustness fixes
  • generated output artifact cleanup

@steezkelly
Copy link
Copy Markdown
Author

Local verification refresh from a clean temporary checkout of this PR:

  • gh pr checkout 49 --repo NousResearch/hermes-agent-self-evolution
  • /home/steve/hermes0/hermes-agent-self-evolution/venv/bin/python -m pytest tests/skills/test_skill_module.py tests/skills/test_evolve_skill_extraction.py -q → 12 passed, 11 warnings
  • /home/steve/hermes0/hermes-agent-self-evolution/venv/bin/python -m pytest -q → 144 passed, 11 warnings

GitHub currently reports no visible Actions runs/checks for this branch, so local verification is the available signal for now.

@steezkelly
Copy link
Copy Markdown
Author

Update: added a minimal GitHub Actions pytest workflow to this branch so the repo has an in-repo CI definition going forward.

Local verification after the workflow commit (0fe6673421b201e63b0247ef11658b4c6e933313):

cd /tmp/self-evolution-pr49-ci
/home/steve/hermes0/hermes-agent-self-evolution/venv/bin/python -m py_compile $(find evolution -name '*.py' -print)
/home/steve/hermes0/hermes-agent-self-evolution/venv/bin/python -m pytest tests/skills/test_skill_module.py tests/skills/test_evolve_skill_extraction.py -q
# 12 passed, 11 warnings
/home/steve/hermes0/hermes-agent-self-evolution/venv/bin/python -m pytest -q
# 144 passed, 11 warnings

Note: GitHub currently shows no regular Actions runs for this repo/branch; upstream only has the Copilot code review workflow visible. The new workflow is included in the PR so future pushes/PRs can run python -m pytest -q once Actions picks it up/after merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant