Lingee Bench
中文

Apply for Leaderboard

Participants connect the agent under test via a CLI harness, run private-set tasks against Mock MCP in the unified full-Docker environment, and receive a report after rule + LLM Judge dual-channel scoring. Evaluation configuration follows the seven-layer disclosure list, keeping scores attributable. Fill in the information below to submit an application; the evaluation team will contact you within 15 business days to confirm the evaluation window.

Leaderboard application

Fields marked * are required

Leaderboard process

  1. Submit the application and confirm the evaluation window
  2. Connect the agent under test via a CLI harness following the Harbor scheme
  3. Run tasks offline against Mock MCP in the full-Docker environment
  4. Rule + LLM Judge dual-channel scoring, macro-average aggregation
  5. Receive the evaluation report; optional public leaderboard entry

Evaluation environment conventions

  • Unified Mock MCP: decoupled from real business systems; identical environments for all participants
  • Full Docker: task bank, tools, and dependencies all baked into images; offline execution
  • Private-set gaming protection: the main leaderboard task set stays private
  • Reproducible: multiple independent runs per task, reliability@k reported; seven-layer evaluation disclosure

Result conventions

  • Task score S = w·R + (1−w)·J, macro-average aggregation
  • Efficiency metrics: output/input tokens, turns (accounting disclosed with the numbers)
  • Stability: reliability@k curves
  • Score gaps within the noise band are not interpreted