Skip to main content

v1.0.0

Releval 1.0.0 (12 July 2026) is the first release: a self-hosted search relevance evaluation platform that runs real queries against your search system, collects human and AI judgments, and reports the metrics that show whether a ranking change helped.

Features​

  • Search endpoints for Elasticsearch, OpenSearch, Solr, Vespa, any HTTP search API, or a rendered search results page, with per-endpoint authentication and a test facility for checking a request before it is used in a run.
  • Corpora, query sets and query templates: the documents results are matched against, the queries worth measuring, and Handlebars templates that turn each query into the request an endpoint expects.
  • Evaluation runs that execute a query set against an endpoint, capture every response, and prepare the candidates for judgment, with runs compared side by side on a dashboard.
  • Judgments on binary, graded or detailed scales, entered by hand with a keyboard-driven UI or produced by AI judges backed by OpenAI, Anthropic, Amazon Bedrock, Azure OpenAI, Ollama or any OpenAI-compatible endpoint; a judgment carries forward to later runs of the same query and result.
  • Judgment lists uploaded from existing relevance data and used for metrics alongside human and AI judgments.
  • Metrics at run, query and candidate level: NDCG, MAP, MRR, ERR, precision, recall and F-score at any depth.
  • User Behavior Insights: track the queries your application serves and the events users generate against them into ClickHouse, register Sites for browser-side tracking, and query the data in a SQL workspace.
  • A REST API, a gRPC API and an MCP server, with App Clients for server-to-server access, the Admin, Member and Rater roles, and Individual, Team and Enterprise licence tiers.