Community benchmark: Grok 4.7 on Enterprise-Bench (frontier) #88

Goal

Run Grok 4.7 on Enterprise-Bench L1-L2 and submit a public, reproducible leaderboard result. Category: Frontier / proprietary. Open for a community contributor to claim.

Why this model

The Grok 4.7 release emphasizes longer-running tasks, self-verification, and professional knowledge work, and is available through Grok Build and the API.[5] This makes it a useful update to the existing Grok Build + Grok 4.5 baseline, with a focus on whether retrieval accuracy improves without excessive token usage.

Coverage checked (2026-09-28)

This model is absent from the live Harbor leaderboard and the repository's merged entry files.[8][9] Existing issue and PR titles/bodies were checked before opening this request. Related but different: #46 proposes Grok 4.6. This issue targets 4.7; coordinate reusable harness setup with that contributor.

Suggested starting point

• Target: Grok 4.7, standard variant. Resolve and record the exact API model identifier from the provider before running.
• Suggested harness: Grok Build, matching the existing Grok 4.5 row. Confirm your pinned Harbor adapter accepts the selected model and connects to all benchmark MCP servers.
• Access: Grok API / Grok Build access; the release announcement links the official setup and API documentation.[5] Check account limits and agree a spend cap.
• Record: exact model version, Grok Build and adapter versions, effort, context settings, provider and token accounting. Do not allow an automatic model update or mix the fast variant into the same submission.

The harness is a recommendation, not a restriction. Share a different reproducible pairing here if it is easier to run. This issue does not imply that a new adapter or provider integration has already been tested.

Run and submission checklist

Start with the repository's runbook. The live Harbor dataset slug is case-sensitive: Enterprise-Bench/l1-l2-bench.[9]

• Comment with your proposed harness, provider, access needs and rough timeline before starting, so we can avoid duplicate runs. No benchmark run is needed to express interest.
• Pin the unmodified L1-L2 dataset revision or repo commit, harness/Harbor versions, and model configuration. Connect the standard CRM, PM and file-server MCP tools.
• Smoke-test one task and inspect its tool calls before committing to a full run. Share any adapter or access blocker here.
• Run all 14 L1-L2 tasks with 10 independent trials per task, targeting 140 trials. Keep the judge and tool surface consistent, record concurrency, and disclose every failure, retry or exclusion. Do not discard failed trials.
• Report accuracy, pass@k, cached/uncached/output tokens, runtime, and cost. Record self-hosted infrastructure cost separately where relevant, rather than presenting free API usage as zero total cost.
• Upload a public Harbor job with trajectories that reviewers can open while logged out. Remove credentials, not failure evidence.
• Submit one result PR using leaderboard/row-template.yaml under leaderboard/entries/. Fill every required field with observed results, ensure n_trials equals the unique trial_ids count, and run make validate-leaderboard.[7]

Done means a reviewed, reproducible result submission, not an intent or a placeholder score. This issue requests a run; it does not claim a benchmark result or reserve a leaderboard rank. Maintainers handle publication after review.[7]

Want to pick this up?

Comment here with your harness + provider and any setup or access blocker. If compute/API cost is a blocker, flag it before spending. Leave secrets out of comments and logs. You can also join the DevRev Community Slack to discuss setup.

Sources

[5] https://x.ai/news/grok-4-7
[7] https://github.com/devrev/enterprise-bench/blob/main/docs/submit-results.md
[8] https://github.com/devrev/enterprise-bench/tree/main/leaderboard/entries
[9] https://hub.harborframework.com/datasets/Enterprise-Bench/l1-l2-bench?tab=leaderboard&leaderboard=main