MCP Tool Execution Errors Use Wrong Format

[brandonmikeska]

Code of Conduct

• I agree to follow this project's Code of Conduct

AI Policy

• I agree to follow this project's AI Policy, or I agree that AI was not used while creating this issue.

Versions

Environment

• Elixir: 1.19.0
• Erlang/OTP: 28
• ash_ai: 0.4.0 (latest on hex as of 2025-11-23)
• ash: 3.11.3
• ash_json_api: 1.5.1

Verified in Latest Version

I confirmed the issue exists in ash_ai 0.4.0 (latest). The CHANGELOG shows no fixes for MCP error serialization in 0.3.0 or 0.4.0. The Jason.encode!() call is still present in the current main branch source code.

Operating system

Ubuntu

Current Behavior

MCP tool execution errors are returned as JSON-RPC errors instead of MCP-compliant isError: true responses. This causes clients to receive generic "Tool execution failed" messages instead of actionable error details.

When a tool execution fails (e.g., validation error), the MCP server returns:

{
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": -32000,
    "message": "Tool execution failed",
    "data": {
      "errors": [...]
    }
  }
}

Result for LLM/Client: MCP error -32000: Tool execution failed - no useful error details.

Reproduction

1. Set up an MCP server with any Ash resource
2. Call a tool action with invalid input (e.g., missing required fields)
3. Observe the error response has data.error as a JSON string instead of a structured array

Expected Behavior

According to the MCP specification (2025-06-18):

If the tool execution fails, servers SHOULD set isError to true and include error details in the content. Error handling should be done within tool results rather than at the protocol level.

Tool execution errors should return a successful response with isError: true:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "isError": true,
    "content": [
      {
        "type": "text",
        "text": "/data/relationships/personas: At least one persona is required"
      }
    ]
  }
}

Result for LLM/Client: /data/relationships/personas: At least one persona is required - actionable error message.

Proposed Fix

The issue has two parts:

1. Wrong Error Format in server.ex

In lib/ash_ai/mcp/server.ex, the handle_json_rpc_request function returns tool execution errors as JSON-RPC errors:

# Current (incorrect)
{:error, errors} ->
  response = %{
    "jsonrpc" => "2.0",
    "id" => id,
    "error" => %{
      "code" => -32_000,
      "message" => "Tool execution failed",
      "data" => %{"errors" => errors}
    }
  }

2. Double-Serialization in tools.ex

In lib/ash_ai/tools.ex, the run_mcp_resource_action/3 function JSON-encodes the error before returning:

# Current (incorrect)
{:error, error} ->
  error = Ash.Error.to_error_class(error)

  {:error,
   domain
   |> AshJsonApi.Error.to_json_api_errors(resource, error, action.type)
   |> AshAi.Serializer.serialize_errors()
   |> Jason.encode!()}  # <-- This causes double-serialization

The Fix

1. Change Error Format to MCP-Compliant isError: true

In lib/ash_ai/mcp/server.ex, replace the JSON-RPC error with a successful response containing isError: true:

{:error, errors} ->
  # Per MCP specification (2025-06-18), tool execution errors should be
  # returned as successful responses with isError: true, NOT as JSON-RPC errors.
  # This allows LLMs to see and handle the error appropriately.
  # See: https://modelcontextprotocol.io/specification/2025-06-18/server/tools
  error_text = format_tool_errors(errors)

  result = %{
    "isError" => true,
    "content" => [%{"type" => "text", "text" => error_text}]
  }

  response = %{
    "jsonrpc" => "2.0",
    "id" => id,
    "result" => result
  }

  {:json_response, Jason.encode!(response), session_id}

2. Add Error Formatting Helpers

Add helper functions to format errors into readable text:

@doc """
Format tool execution errors into a human-readable text message.
"""
def format_tool_errors(errors) when is_list(errors) do
  formatted =
    errors
    |> Enum.map(&format_single_error/1)
    |> Enum.reject(&is_nil/1)
    |> Enum.join("\n")

  if formatted == "" do
    "Tool execution failed"
  else
    formatted
  end
end

def format_tool_errors(_), do: "Tool execution failed"

defp format_single_error(%{} = error) do
  field =
    case error do
      %{source: %{pointer: pointer}} when is_binary(pointer) -> pointer
      %{source: %{parameter: param}} when is_binary(param) -> param
      %{"source" => %{"pointer" => pointer}} when is_binary(pointer) -> pointer
      %{"source" => %{"parameter" => param}} when is_binary(param) -> param
      _ -> nil
    end

  detail =
    case error do
      %{detail: detail} when is_binary(detail) -> detail
      %{"detail" => detail} when is_binary(detail) -> detail
      %{title: title} when is_binary(title) -> title
      %{"title" => title} when is_binary(title) -> title
      %{message: message} when is_binary(message) -> message
      %{"message" => message} when is_binary(message) -> message
      _ -> nil
    end

  cond do
    field && detail -> "#{field}: #{detail}"
    detail -> detail
    field -> "Error at #{field}"
    true -> nil
  end
end

defp format_single_error(_), do: nil

3. Remove Double-Serialization in tools.ex

In lib/ash_ai/tools.ex, remove the Jason.encode!() call:

{:error, error} ->
  error = Ash.Error.to_error_class(error)

  # Return structured errors, not JSON-encoded string
  # The MCP server will handle formatting via format_tool_errors/1
  {:error,
   domain
   |> AshJsonApi.Error.to_json_api_errors(resource, error, action.type)
   |> AshAi.Serializer.serialize_errors()}

Why JSON-RPC Errors Are Wrong for Tool Execution

The MCP specification distinguishes between two types of errors:

Error Type	When to Use	Format
Protocol Errors	Invalid requests, unknown methods, transport failures	JSON-RPC error object
Tool Execution Errors	Validation failures, business logic errors, resource not found	isError: true in result

Using JSON-RPC errors for tool execution:

• Hides error details from LLMs (they only see "Tool execution failed")
• Prevents LLMs from self-correcting based on error messages
• Breaks the MCP contract that tools should handle their own errors

Test Results

Before fix:

MCP error -32000: Tool execution failed

After fix:

/data/relationships/personas: At least one persona is required

References

• MCP Specification - Tools
• MCP Error Handling Best Practices

Implementation

A working implementation is available at:

• Repository: https://github.com/catalioapp/ash_ai
• Branch: fix/mcp-error-serialization

Environment

• ash_ai version: 0.3.x (from hex)
• Tested with MCP Inspector and curl

[zachdaniel]

PR welcome to resolve 🙇