RoleCall

IBM Bob calls every endpoint as every role before you merge, proves the permission holes a PR opened, and fixes them with a regression test.

Broken access control is #1 in the OWASP Top 10:2025. OWASP found some form of it in 100% of the applications it tested. It slips through code review because test suites log in as the admin or the owner and stay green. RoleCall catches these regressions in the pull request, before merge. It does this with evidence (real HTTP requests) rather than an AI opinion.

Built for the IBM Bob 2.0 Hackathon (lablab.ai, Sept 2026) · Team 1680.

---

What it does

1. Map. In the 🛡️ RoleCall custom mode, IBM Bob drafts rolecall.policy.yaml (who should reach what), using parallel subagents, one per route file. A human approves the draft.
2. Probe. RoleCall starts the app twice, from main and from the PR, in throwaway git worktrees. It calls every endpoint as every principal: anonymous, each role, and a second user of the same role for object-level (BOLA/IDOR) checks. It records the real status codes.
3. Diff. It compares the two matrices, and checks them against the approved policy. Findings are ESCALATION, BOLA, LOCKOUT, ERROR, POLICY_VIOLATION and POLICY_LOCKOUT, each backed by the request and response.
4. Explain and fix. Bob's authz-critic skill traces each finding to the exact diff hunk. The authz-actor skill writes the minimal fix plus a regression test. RoleCall then probes again to prove the fix.

The probe decides; Bob explains and fixes. No LLM ever decides whether access is allowed.

Results on the demo repo

The demo target is Acme API, a small Express app with 13 endpoints and 5 principals. It has three seeded pull requests:

PR	What it looked like	Acme's own tests	RoleCall
pr-1-admin-audit-logging	"add audit logging"; the admin guard is moved below the user routes	✅ 7/7 pass	❌ 8 ESCALATION (anonymous DELETE /api/admin/users/:id → 204)
pr-2-invoice-lookup-helper	"extract shared findInvoice helper"; the ownership check is dropped	✅ 7/7 pass	❌ 2 BOLA (another user reads inv-1 → 200)
pr-3-tidy-constants	pure refactor	✅ 7/7 pass	✅ 0 findings (no noise)

• Holes caught: 10/10 by RoleCall; 0/10 by the existing test suite.
• Audit speed: about 0.55 s per full audit (130 real HTTP requests and two app boots, on a MacBook Air).
• Bob's fixes: 3 lines for PR-1 and 1 line for PR-2, each with regression tests. After each fix, the re-audit found 0 findings.

The policy, fixes, regression tests and before/after reports from the recorded Bob run are in docs/demo-run/.

How IBM Bob is used

Bob is used both to build RoleCall and to run it. Every Bob task's consumption summary is in bob_sessions/.

Bob feature	Where
Plan mode	Task 01: Bob turned SPEC.md and the acceptance tests into docs/PLAN.md, covering build order, risks and parallelization
Agent mode + parallel subagents	Task 02: Bob implemented the engine (src/, bin/) with three parallel subagent groups until the acceptance tests passed
Bob-built MCP server	Task 03: mcp/server.js exposes rolecall_audit, rolecall_get_evidence and rolecall_list_endpoints to Bob
Custom mode	bob-package/.bob/custom_modes.yaml: 🛡️ RoleCall, with edits restricted by fileRegex to src/, test/ and the policy file
Skills (actor/critic)	authz-critic (read-only root cause) and authz-actor (minimal fix + regression test), following IBM's secure-code actor/critic pattern
Mode rules	rules-rolecall/01-workflow.md: the six-step audit loop
Human in the loop	Bob asks for approval of the drafted policy before auditing
AGENTS.md	Persistent project context for every Bob task

Division of work: the specification, acceptance tests and Acme demo fixture were prepared up front as the target for Bob. IBM Bob wrote the RoleCall engine, CLI, MCP server, custom mode, skills and rules, and performed every audit and fix in the demo.

Quick start

Requires Node ≥ 20 and git.

npm install
npm test                                   # 16 acceptance tests
scripts/setup-demo.sh                      # builds ../demo-workspace/acme-api with main + 3 PR branches
node bin/rolecall.js audit --repo ../demo-workspace/acme-api --head pr-1-admin-audit-logging
open ../demo-workspace/acme-api/.rolecall/report.html

audit exits 1 when it finds any critical or high finding, so it can gate CI.

Using it with IBM Bob:

1. Open ../demo-workspace/acme-api in the Bob IDE. The setup script installs .bob/ (the mode, skills, rules and the MCP registration).
2. Check Settings → MCP, where rolecall should be connected.
3. Pick the 🛡️ RoleCall mode and ask: "Audit this branch against main for authorization regressions and fix anything exploitable."

Using it on your own app

Add a rolecall.config.json to the target repo. See examples/acme-api/rolecall.config.json and SPEC.md §3. It defines:

• the start command, port and health path;
• an optional reset hook, so every probe sees the same seed data;
• how to mint tokens for each principal (HS256 JWT today);
• the endpoints to probe, with an optional owner for object-level checks.

RoleCall talks to the app only over HTTP, so the app can be written in any stack.

Project layout

src/            engine: config, tokens, launcher, prober, differ, policy, report, audit
bin/            CLI (audit | probe | diff | report)
mcp/            MCP stdio server for IBM Bob
bob-package/    .bob/ custom mode, rules and actor/critic skills installed into target repos
examples/       Acme API demo target + seeded PR overlays
test/           acceptance tests (unit + end-to-end against the seeded PRs)
bob_sessions/   IBM Bob task-session evidence
SPEC.md         the build specification · AGENTS.md: project context for Bob

Roadmap

• OIDC principals (Keycloak/Auth0 password or client-credentials grants) instead of locally minted JWTs
• Endpoint discovery from OpenAPI or framework route tables
• A GitHub Action that posts report.md as a PR comment and blocks the merge on critical findings
• SARIF output for code-scanning dashboards

License

MIT