Contribute to Assay
Help build the quality layer for agentic software. Run evaluations on MCP servers and agent skills with your own tokens, submit results, and help the community find the best tools.
How Community Evaluation Works
Pick a Package
Choose from the evaluation queue below, or evaluate a package you use. The queue prioritizes packages with no existing evaluation or outdated scores.
Run the Evaluation
Use the Assay evaluation skill or CLI tool. It runs against your LLM tokens, analyzes the package, and produces a structured JSON result.
Submit Results
Open a pull request to the Assay repo with your evaluation JSON. Assay reviews and merges quality submissions.
Get Your API Key
Sign in with GitHub to get an API key for submitting evaluations. Your GitHub identity is used for contributor attribution and trust tier progression.
We only request read:user scope — we store your
username and avatar, nothing else.
Trust & Quality
Reproducible Evaluations
All evaluations use Assay's standardized eval configs — deterministic prompts, pinned model versions, and structured output schemas. Results should be reproducible by anyone running the same config.
Cross-Validation
When multiple independent contributors evaluate the same package, agreement between submissions increases confidence. Cross-validated scores carry more weight.
Spot-Check Verification
Assay re-runs approximately 10% of new contributor submissions using our own tokens as a quality gate. This builds trust gradually — established contributors earn higher trust over time.
Anti-Gaming
Package authors evaluating their own tools must disclose the relationship. Undisclosed self-evaluations that significantly diverge from independent evaluations are flagged for review.
Evaluation Queue
10313 packages need evaluationA Model Context Protocol (MCP) server that provides seamless integration with Google Cloud Storage, enabling AI assistants to perform file operations, manage buckets, and interact with GCS resources directly.
A FastMCP server that loads personal context from Obsidian vaults using frontmatter filtering - curate your knowledge for AI conversations.
A resilient MCP server built with fastMCP for sending emails through Gmail's SMTP server using AI agents.
FastMCP2 server for Google Drive uploads with Qdrant-powered semantic search
A fastmcp server for open budget project
Generate FastMCP 3.x Servers from OpenAPI Specs
FastMCP Server for Sefaria
Generate Agent Skills from MCP server tools. Connects via Streamable HTTP, discovers tools, and outputs a skill with schema docs and a call script in the language of your choice.
MCP server for Mattermost — let Claude, Cursor, and other AI assistants work with channels, messages, and files
AWS Athena MCP using FastMCP
FastMCP-powered Agentic Workflows
Python framework for building agents using claude-code-sdk with programmable tools
A FastMCP tool to search and retrieve Polars API documentation.
Ludus FastMCP enables AI-powered management of Ludus cyber ranges through natural language commands. The server exposes **157 tools** across 15 modules for range lifecycle management, scenario deployment, template creation, Ansible role management, and security monitoring integration.
AI-powered code quality analysis using MCP to help AI assistants review code more effectively. Analyze git changes for complexity, security issues, and more through structured prompts.
不会聊天的字幕提取器不是一个好 B 站下载器~
Enhances construction site safety using YOLO for object detection, identifying hazards like workers without helmets or safety vests, and proximity to machinery or vehicles. HDBSCAN clusters safety cone coordinates to create monitored zones. Post-processing algorithms improve detection accuracy.
A powerful Word document processing service based on FastMCP, enabling AI assistants to create, edit, and manage docx files with full formatting support. Preserves original styles when editing content. 基于FastMCP的强大Word文档处理服务，使AI助手能够创建、编辑和管理docx文件，支持完整的格式设置功能。在编辑内容时能够保留原始样式和格式，实现精确的文档操作。
Asynchronous coordination layer for AI coding agents: identities, inboxes, searchable threads, and advisory file leases over FastMCP + Git + SQLite
mcp server which will dynamically define tools based on swagger
Showing 20 of 10313 packages. View full queue via API →
Agent Evaluation Guide
A single document with the complete scoring rubric, JSON schema, and submission instructions. Any AI agent can fetch this URL, evaluate a package, and submit results.
View Evaluation Guide → Rubric v1.0 · Markdown formatGetting Started
Option 1: Use Any AI Agent (Recommended)
Have your AI agent fetch the evaluation guide, evaluate a package from the queue, and submit via the API. Works with Claude, GPT, Gemini, or any agent.
# Your agent fetches the guide and submits results
curl -X POST https://assay.tools/v1/evaluations \
-H "Content-Type: application/json" \
-H "X-Api-Key: your-api-key" \
-d @evaluation.jsonOption 2: Assay CLI Tool
Run evaluations locally using Assay's built-in evaluator:
# Clone the repo
git clone https://github.com/Assay-Tools/assay.git
cd assay
# Run evaluation on a specific package
uv run python -m assay.evaluation.evaluator --package <package-id>
# Or batch evaluate discovered packages
uv run python -m assay.evaluation.evaluator --batch --limit 5Option 3: Request an Evaluation
Know a package that should be in Assay? Open a GitHub issue with the package name and repo URL, and we'll add it to the queue.