Skip to content

Improve MCP tool usability for LLMs with auto-creation and workflow guidance#10

Merged
MWG-Logan merged 3 commits intodevfrom
copilot/fix-3bbac0c1-3fa6-4e77-bbc2-5b0e37c07a9d
Aug 27, 2025
Merged

Improve MCP tool usability for LLMs with auto-creation and workflow guidance#10
MWG-Logan merged 3 commits intodevfrom
copilot/fix-3bbac0c1-3fa6-4e77-bbc2-5b0e37c07a9d

Conversation

Copy link
Contributor

Copilot AI commented Aug 26, 2025

Addresses the core issue where "Copilot seems to have trouble using the tools quite often" by implementing comprehensive usability improvements that reduce friction and provide clear workflow guidance.

Problem Statement

LLMs frequently struggled with the memory MCP tools due to:

  • Failed operations when trying to add observations to non-existent entities
  • Lack of clear workflow guidance (should view graph first, then create entities, then add observations)
  • Confusing error messages without helpful examples
  • Missing guidance on tool usage sequence

Key Improvements

🔄 Auto-Creation Features

  • add_observation: Now automatically creates missing entities with configurable entityType (defaults to "Unknown")
  • create_relations: Automatically creates missing entities referenced in relationships
  • create_entities: Auto-creates empty observations arrays when missing

Example - this now works even if "John" doesn't exist:

#memory-test_add_observation workspaceId="project" entityName="John" observation="Software developer" entityType="Person"

📋 Enhanced Tool Descriptions & Workflow Guidance

  • read_graph: Marked as "RECOMMENDED FIRST STEP" with clear guidance
  • All tools now include workflow hints about when to use them
  • Parameter descriptions include examples and expected formats
  • Tools guide users through proper sequence: read → search → create → relate → observe

⚡ Improved Error Messages & Validation

  • Specific error messages with corrective examples
  • Better parameter validation with helpful guidance
  • Auto-handling of edge cases to reduce friction

Before:

Error: Entity 'Alice' not found

After:

Error: entityName and observation are required. Please provide both parameters.
Example: {"entityName": "Alice", "observation": "Software engineer", "entityType": "Person"}

📚 Updated Documentation

  • README now shows recommended workflow sequence with step-by-step examples
  • Clear feature summary highlighting auto-creation capabilities
  • Better examples demonstrating proper tool usage order

Impact

These changes significantly improve LLM usability by:

  1. Reducing failed operations - Auto-creation handles missing dependencies
  2. Providing clear guidance - Tools now guide proper workflow sequence
  3. Better error recovery - Helpful messages show how to fix issues
  4. Lowering complexity - Common edge cases handled automatically

The tools should now be much more intuitive and reliable for AI assistants to use correctly.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits August 26, 2025 22:50
Co-authored-by: MWGMorningwood <2997336+MWGMorningwood@users.noreply.github.com>
Co-authored-by: MWGMorningwood <2997336+MWGMorningwood@users.noreply.github.com>
Copilot AI changed the title [WIP] Evaluate the usability of the memory MCP defined here for LLMs and improve input validation and descriptions. Copilot seems to have trouble using the tools quite often. Specifically, it may help to inform the agent to view the knowledge graph, create e... Improve MCP tool usability for LLMs with auto-creation and workflow guidance Aug 26, 2025
Copilot AI requested a review from MWG-Logan August 26, 2025 22:53
@MWG-Logan MWG-Logan marked this pull request as ready for review August 27, 2025 14:52
@MWG-Logan MWG-Logan merged commit 0fdc117 into dev Aug 27, 2025
@MWG-Logan MWG-Logan deleted the copilot/fix-3bbac0c1-3fa6-4e77-bbc2-5b0e37c07a9d branch November 13, 2025 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants