Skip to main content

MCP Server

Releval ships with a built-in Model Context Protocol (MCP) server that exposes a curated subset of the API as tools an AI assistant can call. Point Claude Code or another MCP-compatible client at it. You can ask, in natural language, to "list my evaluations", "create a run for evaluation X", or "show me the judgments I've made for this query", without navigating the UI or writing code to call the API directly.

The server is mounted at /mcp on the same host as your Releval REST API and uses MCP's Streamable HTTP transport.

Tools cover the main editorial surfaces — search endpoints, query sets, query templates, evaluations and runs, and judgments. What the assistant is allowed to do is governed by the role of the App Client whose token it presents; the same authorization, validation, and audit-logging rules that gate any other request to Releval gate tool calls too. Browse the live tool list and input schemas from your client of choice (see Inspecting the server below).

Authentication​

The MCP server uses the same OAuth 2.0 client-credentials flow as the rest of the API. You authenticate by:

  1. Creating an App Client for the MCP integration.
  2. Exchanging the client credentials for a JWT bearer token at /oauth2/token.
  3. Sending the token in the Authorization: Bearer … header on MCP requests.

Most MCP clients (including Claude Code and the Inspector) handle the OAuth flow automatically once you tell them the server URL; they discover the authorisation server from the protected-resource metadata Releval exposes.

Inspecting the server​

The MCP Inspector is the official tool for poking at a running MCP server: list its tools, view their schemas, invoke them with hand-crafted arguments, and watch the JSON-RPC traffic. Run it without installing anything:

npx @modelcontextprotocol/inspector

The Inspector opens a browser UI. Pick Streamable HTTP as the transport, point it at https://${RELEVAL_HOST}/mcp, and complete the OAuth flow against your App Client. Once connected, the Tools tab lists every tool the server exposes; selecting one shows its input schema and a form for calling it.

This is the fastest way to:

  • Confirm the server is reachable and authentication is configured correctly.
  • See exactly what tools are available on your version of Releval and how each one is shaped, without reading the source.
  • Reproduce a misbehaving tool call from an assistant by replaying it with the same arguments, so you can tell whether the bug is in the tool or in the prompt.

Connecting an MCP client​

Claude Code​

Add Releval as an MCP server in Claude Code's settings. The exact UI depends on your version, but the configuration looks like:

{
  "mcpServers": {
    "releval": {
      "url": "https://${RELEVAL_HOST}/mcp"
    }
  }
}

Claude Code will prompt you to authenticate the first time it connects and will perform the OAuth exchange against /oauth2/token using the App Client credentials you supply.

Claude Desktop​

Claude Desktop's claude_desktop_config.json accepts the same shape as Claude Code's config, under the mcpServers key. The OAuth flow is identical.

Other MCP-compatible clients​

Any MCP client that supports OAuth-protected resources and the Streamable HTTP transport can connect. The two coordinates the client needs are:

  • Resource URL: https://${RELEVAL_HOST}/mcp
  • Authorisation server: https://${RELEVAL_HOST}/oauth2/token (client_credentials grant)

A spec-compliant client will discover the authorisation server itself from the protected-resource metadata served at the resource URL — supplying it manually is only necessary as a fallback.

Permissions​

MCP calls inherit the role and license limits of the App Client whose token is presented:

  • A client without Admin role cannot perform admin-only operations (e.g. creating shared resources on behalf of others).
  • License limits are enforced inside the tool: an evaluation-creating tool returns a license-limit error when the workspace is at its evaluation cap.
  • Every tool invocation is audit-logged with the calling member.

When to use MCP vs. the REST API​

Use MCP when an AI assistant is the natural driver: exploration ("what evaluations have I run this week?"), conversational construction ("create a run for the latest production endpoint, NDCG@10 and MAP, tag it experiment-rerank-v2"), or pulling Releval data into an LLM workflow.

Use the REST API for everything else: programmatic integration, CI pipelines, scheduled jobs, anything that benefits from explicit, replayable requests.