Fix/mcp schema validation #3
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update PromptSchema and ResponseSchema to use union types that explicitly accept both string and object inputs at the MCP protocol level, while preserving existing JSON string preprocessing behavior. - Add union types to schemas: z.union([z.string(), BaseSchema]) - Add root-level descriptions for prompt and response objects - Apply descriptions at tool input schema level following established patterns - Maintain backward compatibility with existing preprocessing logic Fixes: MCP error -32602: Expected object, received string 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add comprehensive tests for PromptSchema and ResponseSchema validation - Test both string and object inputs for schema flexibility - Update Jest config to use SWC for faster compilation (~0.4s vs 3-5s) - Add moduleNameMapper to support .js extension imports in TypeScript 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add @swc/core and @swc/jest for fast TypeScript compilation - Add ts-node as dev dependency for TypeScript tooling - Update package-lock.json with new dependencies 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Fixed MCP protocol validation for evaluation prompt/response parameters and added comprehensive test coverage with optimized Jest configuration. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add focused debug logging for evaluation start/success with key identifiers - Add error logging for missing API key in Mandoline client - Remove verbose debug logs (payload details, client creation steps) - Keep essential logs for troubleshooting and auditing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Remove JSON string input flexibility from PromptSchema and ResponseSchema - Eliminate BasePromptSchema/BaseResponseSchema intermediate constants - Update parameter descriptions to clarify object-only requirement - Fix evaluation payload tests to match new schema behavior - Clean up unused JSON parsing logic and debug logging 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Simplifies logging by using the base logger directly instead of creating request-specific child loggers for minimal usage. Also removes verbose debug logging for evaluation creation. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Simplifies MCP schema validation by removing JSON string parsing and requiring direct object inputs
Note – "prompt" and "response" (both in the evaluation payload) were union types over object and stringified JSON (which was then parsed into an object). We found that this added flexibility (which was intended to help the model) just caused more confusion. So, now we accept only objects.
Changes
better type safety
Test plan