Community benchmark: Gemini 4 Argon with agy CLI on Enterprise-Bench (frontier) #132

Goal

Run Gemini 4 Argon with agy CLI on Enterprise-Bench L1-L2 and submit a public, reproducible leaderboard result. Category: Frontier. Open for a community contributor to claim.

Why this one

This pairs Gemini 4 Argon with the agy CLI to see how it does in Google's own agent harness, and compares with the existing Gemini 4 Argon row on the leaderboard (see #123). Keep the tool surface matched where possible.

Run and submission checklist

Start with the repository's runbook. The live Harbor dataset slug is case-sensitive: Enterprise-Bench/l1-l2-bench.

• Comment with your proposed provider, access needs and rough timeline before starting, so we can avoid duplicate runs.
• Pin the unmodified L1-L2 dataset revision or repo commit, harness/Harbor versions, and model configuration. Connect the standard CRM, PM and file-server MCP tools.
• Smoke-test one task and inspect its tool calls before a full run.
• Run all 14 L1-L2 tasks with 10 independent trials per task (140 trials). Disclose every failure, retry or exclusion. Do not discard failed trials.
• Report accuracy, pass@k, cached/uncached/output tokens, runtime, and cost. If cost is an estimate, label it as such.
• Upload a public Harbor job with trajectories that reviewers can open while logged out. Remove credentials, not failure evidence.
• Submit one result PR using leaderboard/row-template.yaml under leaderboard/entries/, and run make validate-leaderboard.

Done means a reviewed, reproducible result submission, not an intent or a placeholder score. This issue requests a run; it does not claim a result or reserve a leaderboard rank.

You can also join the DevRev Community Slack to discuss setup.