Skip to directory

Scorecard MCP

Scorecard

Streamable HTTP (remote); stdio (local package)2 documented tools0

About Scorecard MCP

Connects AI clients to the Scorecard AI evaluation platform to search SDK docs, manage projects/testsets/metrics/runs, and analyze evaluations through an SDK code sandbox.

What you can use it for

  • Search SDK documentation for integration guidance.
  • Create/manage testsets, metrics, evaluation runs, and review results when the user authorizes those changes.
Connection and setup
Hosting
Remote and local
Transport
Streamable HTTP (remote); stdio (local package)
Endpoint
https://mcp.scorecard.io/mcp
Authentication
Remote: interactive OAuth; API key may be sent as Bearer token for clients that cannot complete OAuth. Local: SCORECARD_API_KEY and SCORECARD_ENVIRONMENT environment variables.

Requirements

  • Scorecard account/API access
  • MCP client supporting remote Streamable HTTP/OAuth or local stdio
  • Deno on PATH if using local package; local execute runs code in Deno sandbox

Setup steps

  1. Remote: add https://mcp.scorecard.io/mcp in an MCP client and complete OAuth in the browser.
  2. For clients unable to complete OAuth, configure Authorization: Bearer YOUR_SCORECARD_API_KEY.
  3. Local alternative: use npx -y scorecard-ai-mcp@latest with SCORECARD_API_KEY and SCORECARD_ENVIRONMENT; install Deno.
  4. Use search_docs to identify SDK methods. Treat execute as potentially write-capable; use documented restrictions such as --code-allow-http-gets or allowed/blocked method flags when running locally.
Tools 2

Current Scorecard docs identify two Code Mode tools; legacy endpoint-specific tools are retired and operations now run through execute. The execute surface can invoke write/delete SDK operations, so the two-tool count does not mean read-only.

  • search_docs

    Search Scorecard TypeScript SDK documentation for relevant methods and parameters.

    01
  • execute

    Execute assistant-written TypeScript against a pre-authenticated Scorecard SDK in a sandbox and return result/log output.

    02
Limitations and notes
  • execute calls are independent; variables do not persist. API requests time out after 30 seconds and an execute call after about five minutes.
  • Sandbox is restricted to Scorecard API; no filesystem or other network access.
  • Local package requires Deno and may have a different execution sandbox than hosted service.
  • Older tool names such as list_projects/create_testsets are no longer MCP tools.
Source discrepancies
  • Directory discovery may reflect older per-endpoint tools; current publisher docs say those tools no longer exist and have been replaced by search_docs and execute.
Original sources
  • official current tool reference

    Scorecard documents current two-tool Code Mode, retired legacy tools, OAuth/API-key auth and possible mutating SDK operations.

  • official repository

    Publisher repository identifies the hosted MCP implementation and OAuth metadata support.

Documentation checked 2026-10-28T14:00:00.000Z. Connection has not been runtime tested.

Similar tools