Skip to content

[Catalog] Audit every published agent — keep / fix / unpublish #64

Closed

Description

@serge-ivo

[Catalog] Audit every published agent — keep / fix / unpublish

Tracking issue for a data-driven catalog cleanup. A live pull (2026-08-01) showed 22 published agents, all first-party (creator: operator), 0 external creators, 0 external users, lifetime AI spend $0.86 / 69 calls over 4 days, with real usage concentrated on one agent (Coder, 47 calls — the operator dogfooding). The catalog also contains smoke-test fixtures and agents whose descriptions promise capabilities they don't have.

Do

Produce a table for every published agent: slug · subscribers · 30d calls · store_type · declared capabilities (surfaces/runtime/workflow) · description-matches-capabilities? · verdict. Verdicts: keep (works + used), fix (works, needs description/cleanup), unpublish (test/dupe/vaporware).

Known findings to action (children)

  • Smoke tests in the live store → unpublish (#c2)
  • Description ≠ capability (Creator OS, QA Automation, job-application-assistant) → fix + add lint (#c3)
  • Duplicate/dogfood agents + instances → collapse (#c4)

Acceptance

  • One audit doc/table covering all published agents with the columns above.
  • Every agent has a keep/fix/unpublish decision.
  • The three child issues are filed and linked.

Evidence: GET /v1/admin/agents + list_agents + usage_summary (2026-08-01).

Children

Activity

  1. serge-ivo commented on Aug 6, 2026

    @serge-ivo
    ContributorAuthor

    Catalog audit — complete. Public catalog 29 → 13.

    Live pull 2026-08-06 (GET /v1/agents, /v1/admin/agents, /v1/admin/instances, GET /v1/usage?days=30). Every agent in the store is first-party; coder is owned by the proagentstore seed account, all 37 others by the operator. Still 0 external creators; the only non-operator instance rows are the operator's own second identities (sivochkin@gmail.com, GitHub serge-ivo).

    The bar applied: would a first-time visitor think this is a real product? An agent stays if it has a real description, a capability that actually exists, and a plausible use case.

    KEEP — 13

    slug subs 30d calls store_type declared capabilities desc ≈ caps? why it stays
    coder 2 739 agent (seeded flagship) yes The flagship. All real usage concentrates here.
    coder-repo 4 291 agent surfaces:[coding] runtime:coding workflow:CODING_SESSION + 6 tools yes Most-subscribed agent; 4 live instances.
    coder-lead 1 65 agent 6 delegation/github tools yes Real cross-repo supervisor, in use.
    facebook-friend-confirmer 1 117 agent runtime:browser workflow:BROWSER_TASK yes Real browser workflow, real usage.
    small-business-website-lead-finder 1 45 agent — (standalone worker repo) yes Real usage; first link of the lead chain.
    lead-outreach-tj6qrr 1 66 (pipeline) agent instance pipelines yes Real pipeline usage; second link of the chain. Slug flagged below.
    tmux-operator 1 20 agent surfaces:[tmux] runtime:coding + 6 terminal_* tools yes Canonical terminal agent — see the dedup note.
    local-repo-chat 1 4 agent runtime:coding + 4 repo_* tools yes Complete, declared, distinct from repo-chat (local checkout vs GitHub).
    repo-chat 1 1 agent surfaces:[repo] + 3 declared tools yes The reference "built the creator way" agent (migration 0050).
    doc-chat 1 1 agent base chat + files + RAG yes Works on platform primitives; one instance holds 4 files / 5 vectors / 25 messages.
    language-buddy 1 0 agent settingsSchema (migration 0041) yes Complete product; the seeded settingsSchema + voice-language reference.
    job-application-assistant 2 0 agent surfaces:[apply] runtime:browser workflow:JOB_APPLY no — see #66 Real pipeline. Description leaks internal vocabulary.
    site-builder 0 0 agent 2 pipelines + settingsSchema (migration 0057) yes Newest first-party agent, fully declarative. 0 subs because it's new, not because it's fake.

    UNPUBLISH → draft — 16 (done, nothing deleted)

    Every one is now visibility:"draft". Drafting only gates POST /v1/instances/:agentId/subscribe; existing instances, chat and runs are untouched (verified — no other route reads agents.visibility). Fully reversible, and the rows survive, which matters because migrations 0033/0059/0072 use SELECT owner_id FROM agents WHERE slug='data-analyst' as their operator-account anchor.

    Self-declared test fixtures (3) — the description literally begins "Test agent for…". Published 2026-08-05, before the #65 guard; the guard's whole-word test marker catches all three, so they cannot come back accidentally.

    • terminal-operator, iterm-operator, kitty-operator

    Vaporware — config is {}, worker_name null, cron_schedule null, 0 instances, 0 usage (11). Each description promises a capability that does not exist anywhere in the platform for that agent:

    • site-monitor — "every hour… webhook alerts": no cron, no webhook
    • competitor-intel — "daily": no cron
    • seo-auditor — "daily crawl": no cron
    • content-pipeline — "daily": no cron
    • support-escalator — "daily summary": no cron
    • lead-qualifier — "receives leads via webhook": no webhook
    • invoice-parser — store_type:tool, "POST invoice text": no endpoint, no worker
    • data-analyst — "generates SQL, runs queries": no such capability
    • email-drafter — no capability at all
    • chatbot — lowercase generic name, scaffold description
    • qa-automation — 0 instances, 0 usage, empty config, and it advertises ProAppStore apps, a sibling store, in the PAGS catalog

    Vaporware with a dogfood instance (2) — drafted, instance left running:

    • creator-os-agent — promises "scheduled news scans… per-platform rate limits with a retry queue… publishing through your own accounts"; config:{}, no runtime, no workflow, no cron, no connectors. 1 instance (23 msgs, 8 KB docs), 0 30d usage. Also has a creator-os twin already sitting in draft.
    • meeting-notes — "Webhook receives transcripts… Daily digest cron"; none of it exists. Its 1 instance belongs to sivochkin@gmail.com — see the question below.

    FIX (not unpublish) — for #66

    • job-application-assistant — "Rented job application assistant… uses the approved local browser runner task job.apply_basic for basic resume-upload forms." That is internal task-type vocabulary on the storefront, and "Rented" reads as a placeholder. The agent works; the copy doesn't.

    Prevention — shipped (cec87d8)

    POST /v1/batch/bulk-visibility bypassed the #65 guard entirely — it was only wired onto PUT /v1/agents/:id, so anything the single-agent route refused could still be published in bulk from the same session. That is precisely how this cleanup would have undone itself. Now guarded per agent and refused as a batch, with the same allowTestAgent override; un-publishing is never blocked. 6 tests in workers/api/src/routes/batch.test.ts.

    No further lint proposed: the #65 guard covers the fixture class, and lintAgentClaims (#66) already covers the description-vs-capability class.

    Questions for the owner — NOT acted on

    1. The four terminal operators. I kept tmux-operator (real product description, seeded by migration 0072, most usage) and drafted the three whose descriptions say "Test agent for the … backend". But the tools are the generic terminal_* set covering tmux/kitty/iTerm2, so the surviving agent is arguably mis-named — "Terminal Operator" with the generic copy is the better storefront entry. Renaming it means diverging from migration 0072's seed. Your call.
    2. lead-outreach-tj6qrr. Genuine agent, scaffolded slug. PUT /v1/agents/:id does not accept slug, so renaming needs a migration — and it breaks the public URL /agents/lead-outreach-tj6qrr/. Left published as-is.
    3. meeting-notes. Drafted as vaporware, but its one instance belongs to sivochkin@gmail.com. If that is not you, someone outside the operator account has a subscription to an agent that does nothing — worth a look either way.
    4. site-builder at 0 subscribers. Kept on merit (newest, fully declarative, migration-seeded). Flagging it because it is the only survivor with no usage at all.

    All three acceptance boxes are met: the table covers all 29 formerly-published agents, every one has a keep/fix/unpublish decision with a reason, and #65/#66/#67 are filed and linked. Leaving this tracking issue open — it's the parent of #66 (still open) and #67 (awaiting your ruling on the instance sets), and of the four questions above.

  2. serge-ivo commented on Aug 6, 2026

    @serge-ivo
    ContributorAuthor

    Re-audit 2026-08-06 (later same day) — the audit above is done; here is what it got wrong

    I did not redo the table above. I verified it against live state and re-ran the guard/lint over the catalog as it stands right now. The cleanup landed and mostly holds — but it introduced a live storefront regression, missed a newly-published agent, and its usage evidence is stale.

    Evidence: GET https://api.proagentstore.online/v1/agents (14 rows), my_agents (38 rows), usage_summary(range:"all"), agent_analytics, vector_stats, and blockPublishReason/lintAgentClaims run over every published row.


    1. The cleanup broke the storefront filters — 5 of 9 buttons now tell visitors the store is empty

    This is the highest-value finding, and it is a direct consequence of drafting 16 agents.

    store/index.html filters on an exact string match and falls through to the empty state:

    if (currentType     !== 'all') filtered = filtered.filter(a => (a.store_type || 'agent') === currentType);
    if (currentCategory !== 'all') filtered = filtered.filter(a => a.category === currentCategory);
    ...
    if (filtered.length === 0) { empty.style.display = ''; return; }

    …and that empty state reads “Be the first to publish — No agents published yet.”

    Live right now, against the 14 published agents:

    filter button published agents matched
    Category → Chat 0
    Category → Data 0
    Category → Creative 0
    Type → Workers 0
    Type → Tools 0

    Every chat/data/creative agent was one of the commodity rows drafted in this cleanup, so those three buttons went dead the moment it landed. Workers/Tools match 0 because all 14 published rows are store_type:'agent'.

    So the first visitor this catalog was cleaned up for can click 5 of the 9 filter buttons and be told the store has nothing in it — and invited to be the first to publish — on a store with 14 agents. That is exactly the "don't embarrass the platform" failure the audit was for.

    2. 9 of the 14 published agents are unreachable by any category filter

    Same exact-match filter, against categories the hardcoded button row doesn't offer:

    category has a button? agents
    code yes coder-repo, coder-lead, coder
    productivity yes doc-chat, job-application-assistant
    developer-tools no repo-chat, local-repo-chat, mcp-client
    Sales no site-builder, small-business-website-lead-finder
    coding no tmux-operator
    social no facebook-friend-confirmer
    education no language-buddy
    general no lead-outreach-tj6qrr

    There are three different category vocabularies in the codebase and no shared constant:

    • the store's filter row — 5 values (store/index.html)
    • the creator-facing picker — 6 values, ["general","chat","code","data","creative","productivity"] (store/console/src/pages/AgentDetail.tsx:11) — note it offers general, which the store cannot filter
    • the catalog as it actually exists — 8 values, 5 of which appear in neither list

    Applied (migration 0084, pushed — the only change I made): the two that were genuinely wrong data rather than missing buttons.

    • coding → code. Same concept, two spellings; code is what the store button, the console picker and the other three coding agents use. tmux-operator was the one coding agent missing from the "Code" filter purely because of the spelling.
    • Sales → sales. The table's only capitalized category.

    I deliberately did not re-label developer-tools / education / social / general. Those are good data; the button row is the bug, and degrading accurate categories to fit a stale hardcoded list would hide it.

    Safety checked before writing it: category feeds only the storefront chip/filter and the fallback capability derivation in lib/agent-capabilities.ts:477, which is short-circuited (line 459) for any agent declaring a capabilities.surfaces array. tmux-operator declares surfaces:["tmux"], so its resolved capabilities are unchanged. The fallback's category rules are code and insurance; neither Sales nor sales matches, so the Sales agents are unaffected.

    Recommended, not applied — store/index.html is outside the files I was scoped to: derive the category buttons from the loaded agents and hide any type/category button matching zero. ~10 lines, and it fixes both tables above permanently and makes every future catalog cleanup self-correcting instead of silently emptying the storefront. Worth doing before any external visitor arrives.

    3. mcp-client is published and is not in the audit table

    Created 2026-08-06 02:50, after the table above was written. Published, 2 subscribers, 16 calls. The KEEP table has 13 rows; the live catalog has 14. Verdict in the table below.

    4. job-application-assistant is published and the #65 guard would block it today

    I ran blockPublishReason over all 14 published agents. Exactly one hit, and it's the flagship apply agent:

    description contains “Runs a safe fill-only test by default” → whole-word test marker

    It survives only because it is already published — the guard fires on nextVisibility === 'published', and nothing has sent that since. Any attempt to re-publish it (after a draft, or via /v1/batch/bulk-visibility) now 400s without allowTestAgent:true.

    This is a guard false positive, and it collapses neatly into the #66 copy fix already wanted for this same agent: rewriting “Runs a safe fill-only test by default and stops before Submit” as e.g. “Does a safe fill-only dry run by default and stops before Submit” removes the internal vocabulary and clears the guard in one edit. I did not touch published copy — owner's call.

    lintAgentClaims over all 14 published: 0 violations.

    5. The usage evidence in the issue body is stale by ~50×

    #64 says “$0.86 / 69 calls over 4 days”, concentrated on one agent. Live usage_summary(range:"all"):

    2,862 calls · ~$42.30 estimated · 20 subscriptions across 14 agents.

    And it is no longer one agent — 11 of 14 have real usage:

    agent calls est. cost
    coder 739 $17.19
    coder-repo 403 $7.74
    coder-lead 174 $3.02
    facebook-friend-confirmer 117 $3.43
    lead-outreach-tj6qrr 111 $0.22
    small-business-website-lead-finder 48 $1.47
    tmux-operator 20 $0.20
    mcp-client 16 $0.20
    local-repo-chat 4 $0.05
    repo-chat 1 $0.02
    doc-chat 1 $0.02
    language-buddy · job-application-assistant · site-builder 0 —

    Still all the owner's own traffic — 0 external users — so the shop-window framing stands. But #67's premise that “the real functioning catalog is ~4–5 agents” no longer holds.


    Verdict on the 14 published agents as of now

    slug subs calls category icon verdict
    coder 2 739 code 💻 keep
    coder-repo 5 403 code ⌨️ keep
    coder-lead 1 174 code 🧭 keep
    tmux-operator 1 20 coding → code — keep (category fixed in 0084; needs an icon)
    local-repo-chat 1 4 developer-tools 📖 keep
    repo-chat 1 1 developer-tools 🔍 keep
    site-builder 0 0 sales 🏗️ keep — newest, fully declarative reference
    doc-chat 1 1 productivity — keep (needs an icon)
    language-buddy 1 0 education — keep (needs an icon)
    small-business-website-lead-finder 1 48 sales — keep (needs an icon)
    mcp-client 2 16 developer-tools — fix — unaudited; storefront copy is internal jargon (“the private instance”, “the MCP key slot”, “consent-gated MCP connector”); no icon
    lead-outreach-tj6qrr 1 111 general — fix — scaffold slug in the public URL, scaffold category, no icon
    job-application-assistant 2 0 productivity — fix — trips the #65 guard (§4) + the #66 copy issue; same one-line edit clears both
    facebook-friend-confirmer 1 117 social — unpublish → draft (recommended) — see below

    The one unpublish I'd add: facebook-friend-confirmer

    The audit above kept it on usage alone (117 calls). Usage is the wrong test here — it's all the owner's own dogfooding, and the question this catalog has to answer is what a first visitor thinks.

    It is a published, public storefront listing for an agent that automates a logged-in Facebook account — confirming friend requests and “optionally adds a few suggested friends” by driving the owner's real browser session. That is against Meta's platform terms, and it is the only social agent in the store. For a marketplace that wants to open to third-party creators, having that as a public catalog entry is a standing reputational and ToS exposure that no amount of dogfood traffic justifies.

    Recommendation: set visibility:"draft". As this issue already established, drafting only gates POST /v1/instances/:agentId/subscribe — the owner's existing instance, its chats and its browser runs keep working untouched. Fully reversible. Not applied: unpublishing a working agent is the owner's call, not mine.

    Cosmetic, but it is the first impression: 8 of 14 cards are the same tile

    store/index.html renders const icon = a.icon || '⚡' on iconBg = a.icon_bg || '#7c3aed'. Eight published agents have icon:"", so a majority of the storefront grid is the identical purple ⚡ tile: mcp-client, tmux-operator, lead-outreach, facebook-friend-confirmer, small-business-website-lead-finder, doc-chat, language-buddy, job-application-assistant.

    Cheap and reversible (PUT /v1/agents/:id {icon}), but it's eight aesthetic choices on the owner's live catalog, so I've listed it rather than picked emoji.


    What I changed vs. recommended

    Changed — 1 commit (fix(catalog): spell two categories the way the storefront filters them, migration 0084_normalize_catalog_categories.sql): two narrow, idempotent UPDATEs. No schema change, no capability change, no visibility change, nothing deleted. pnpm typecheck clean, 3713/3713 tests pass, biome unaffected (SQL is not linted; the 30 pre-existing errors are on main already).

    Recommended, not applied: the storefront filter fix (§1–2), facebook-friend-confirmer → draft, the job-application-assistant copy edit (§4, also closes #66's item), the 8 missing icons, and lead-outreach-tj6qrr's slug/category.

    Was the ticket still accurate? Partly. Its ask was already satisfied earlier today. Its evidence is stale by ~50× on spend and wrong on the “one agent has all the usage” premise. And its acceptance criteria are met only if you don't count mcp-client, which was published after the table was written — which is itself the argument for the derived-filter fix: a hand-maintained snapshot of a live catalog is stale the same day.

  3. serge-ivo commented on Aug 7, 2026

    @serge-ivo
    ContributorAuthor

    Third audit pass — 2026-08-08. Catalog is 15 published. Read-only; I changed nothing.

    Two audits already ran on 2026-08-06. This pass re-verifies both against live state, and it finds one class of defect neither of them looked for: an agent that works for its owner and is dead on arrival for anyone who subscribes to it. Two published agents are in that state.

    All access through the configured MCP server per AGENTS.md — list_agents, my_agents, my_instances, agent_analytics, usage_summary(range:"all"), list_pipeline_runs, list_connections, list_errors. No REST, no DB, no writes. Repo cross-reference against main @ db8a2af.

    What moved since 2026-08-06

    • portal-watch is new and published (seeded migration 0087, created 2026-08-06 19:41). Neither prior table has it. 15 published, not 14.
    • The storefront filter regression is fixed — b0d5486 replaced the hardcoded button rows with renderFilters() counting store_type/category off the live catalog. That was §1 of the second audit and it is closed. (I could not confirm the deployed HTML: https://proagentstore.online/ returns 403 to my fetch. The fix is in main's history and host deploys on push, so I am reporting it as fixed in source, not verified in production.)
    • Nothing else recommended on 2026-08-06 was actioned. job-application-assistant still trips the guard, facebook-friend-confirmer is still published, the same 8 cards still have no icon, lead-outreach-tj6qrr still has its scaffold slug.
    • Instances 25 → 27 active; subscriptions 21 across 15 published agents (+6 instances on drafted agents).

    The three findings that matter

    1. Two published agents hand a new subscriber an empty chat box

    POST /:agentId/subscribe copies agents.config.pipelines into the new instance — defaultPipelinesFor() in workers/api/src/routes/instances.ts:71, and the comment there states the reason exactly:

    For an agent whose whole behaviour IS its pipelines, those have to arrive with the subscription — loadPipeline reads the INSTANCE config, so without this a fresh subscriber gets an agent that can do nothing until they attach each pipeline by hand.

    That is precisely the state of two published agents:

    agent agents.config where the working pipeline actually lives
    small-business-website-lead-finder {} instance 933ebec5…, pipeline lead_finder — 45 leads from Launceston on 2026-08-04, verified via list_pipeline_runs
    lead-outreach-tj6qrr {} instance 3c09069a…, pipeline draft_outreach — 5 completed runs fed by the lead.created connection

    Both work. Both work only for the owner's own instance, because the behaviour was attached to the instance by hand and never promoted to the agent row. Anyone who subscribes from the storefront today gets a plain chat agent with base tools and no pipeline, no leads collection, and no way to do the thing the card promises. site-builder is the control case and does it right — its config carries pipelines, and migration 0057 embeds both definitions.

    It is worse than an oversight for the lead finder specifically: workers/api/src/lib/pipelines/lead-finder.json exists in the repo, is the epic #94 capstone, and is proven end-to-end in lead-finder.test.ts against the real runner. Its own README says so:

    None of it touched the running lead-finder agent; the lead-finder class is now buildable with zero bespoke code.

    So the reference definition is written, tested and version-controlled, and the published catalog row it describes carries nothing. draft_outreach is worse still — there is no reference JSON for it anywhere in the repo, so the second link of the lead chain exists only as live instance data: untested, unversioned, and lost if that instance is ever cancelled.

    Fix: seed both into agents.config.pipelines by migration, the way 0057 does for site-builder, and extend seed-drift.test.ts to cover them (today it pins site-builder only). Export draft_outreach from instance 3c09069a… to workers/api/src/lib/pipelines/lead-outreach.json first — that is the only copy.

    2. lead_finder crashes on any city big enough to matter, and the failure is invisible from the storefront

    list_errors (source pipeline-run), all on instance 933ebec5…:

    2026-08-03 08:08  pipeline "lead_finder" crashed: WorkflowInternalError: Step s3-flatten-1 output is too large. Maximum allowed size is 1MiB.
    2026-08-03 06:49  pipeline "lead_finder" crashed: WorkflowInternalError: Step s4-map-1   output is too large. Maximum allowed size is 1MiB.
    2026-08-03 06:32  pipeline "lead-finder" crashed: WorkflowInternalError: Step s3-map-1   output is too large. Maximum allowed size is 1MiB.
    2026-08-03 06:25  step 0 (geocode) failed: No geocode result for "Sydney, NSW".
    

    Every subsequent successful run is a small one — Hobart and Launceston at radius: 900, 83–116 places seen. The crashing step names (s3-flatten-1, s4-map-1) match the reference lead-finder.json shape exactly — geocode → fan_out → http_request → flatten → map → enrich → map → filter → dedupe_upsert, index 3 being flatten and index 4 map. There is no slice anywhere in that chain, so the payload carried between steps grows with the city and hits Cloudflare's 1MiB Workflow step limit. The slice primitive that caps exactly this ("the missing top N: cap a list before a per-item connector call so a long source isn't unbounded spend") already exists and is not used. (I read the reference JSON, not the live instance's copy — no read-only MCP tool returns an instance's pipeline definition. The step names make them the same shape; I did not diff them byte for byte.)

    This is the single agent in the catalog whose card names a general capability ("Search by city and business type") that demonstrably fails on the first city a visitor would try.

    Fix: add slice after flatten in lead-finder.json, bounded by a subscriber setting, and seed it per finding 1.

    3. The catalog's own answer to "is anyone looking at this?" is structurally unavailable

    agent_analytics returns funnel: {views, trials, subscribes}. Across every agent I checked — coder (739 calls, 2 subs), coder-repo (6 subs), portal-watch, site-builder, meeting-notes — views and trials are 0 without exception, while subscribes is correct.

    • trials is dead by construction: nothing in workers/api/src ever writes event = 'trial_start'. The public trial route /v1/public/agents/:id/try writes no usage row at all.
    • views is written by GET /v1/public/agents/:id (routes/public.ts:33) — but it binds user_id = '' into a column declared user_id TEXT NOT NULL REFERENCES users(id) (0001_init.sql:58), inside a waitUntil(...).catch(() => {}). If D1 enforces the foreign key, every view insert fails and is swallowed.

    store/agents/detail.html:465 tries the public route first for every visitor including the owner, so a genuine zero would mean nobody — not even the owner, mid-audit — has opened an agent page. I lean strongly toward the FK, but I could not prove it: confirming requires a direct D1 query, which AGENTS.md forbids and which no read-only MCP tool exposes.

    Either way the operational conclusion is the same and it is the important one: this audit cannot tell you whether anyone browses the store, and neither can you. Every "is anyone using it" judgement below rests on subscriptions and token usage, both of which are the owner's own traffic.


    Verdicts — 15 published agents

    Guard + lint were run over all 15 by replicating blockPublishReason (lib/test-agent-guard.ts) and lintAgentClaims (lib/agent-claims-lint.ts) verbatim against each agent's live description and declared capabilities. Result: 1 guard hit, 0 lint violations.

    slug subs calls (all-time) declared capabilities guard lint verdict
    coder-repo 6 552 coding / CODING_SESSION + 7 tools + surfaceOptions + settingsSchema ok ok KEEP
    coder-lead 1 255 no surfaces, 6 delegation/github tools ok ok KEEP
    tmux-operator 2 75 surfaces:[tmux], runtime:coding, 6 terminal_* tools ok ok KEEP
    local-repo-chat 1 4 runtime:coding + 4 repo_* tools ok ok KEEP
    repo-chat 1 1 surfaces:[repo] + 3 declared tools ok ok KEEP
    doc-chat 1 1 none — platform primitives ok ok KEEP
    language-buddy 1 0 settingsSchema (migration 0041) ok ok KEEP
    portal-watch 0 0 runtime:browser / BROWSER_TASK + browserTask.readOnly + settingsSchema ok ok KEEP
    site-builder 0 0 2 seeded pipelines + settingsSchema (migration 0057) ok ok KEEP
    mcp-client 2 16 mcp_list_tools + mcp_call_tool + settingsSchema ok ok FIX
    small-business-website-lead-finder 1 48 {} ok ok FIX
    lead-outreach-tj6qrr 1 111 {} ok ok FIX
    job-application-assistant 2 0 surfaces:[apply] / browser / JOB_APPLY blocked ok FIX
    coder 2 739 coding / CODING_SESSION, no tool allowlist ok ok UNPUBLISH → draft
    facebook-friend-confirmer 1 117 runtime:browser / BROWSER_TASK, tools:[] ok ok UNPUBLISH → draft

    KEEP 9 · FIX 4 · UNPUBLISH 2.

    The nine KEEPs, briefly

    coder-repo is the catalog's best entry and the busiest: 6 instances, one per repo, exactly the shape its card describes, with the fullest declaration in the store (tools + surfaceOptions + a 4-field settingsSchema including the merge-authority control from 0091). coder-lead sits over it and is genuinely used. tmux-operator picked up a second subscriber since the last audit ("Heartfull (tmux)") and its category spelling was fixed in 0084; its config still carries nothing but capabilities — no identity, no welcome message — which is why it reads thinner on the card than it behaves.

    repo-chat and local-repo-chat are the read-only pair and are correctly distinct (GitHub API vs. your own checkout over the runner); both declare their tools properly and repo-chat remains the "built the creator way" reference. doc-chat and language-buddy are low-traffic but honest: they promise only what platform primitives already deliver, and lintAgentClaims agrees.

    portal-watch is the strongest new entry and deserves saying so. It is the only agent in the catalog whose card states its own limits ("It can only ever look: the runtime blocks every Pay, Submit, Send, Delete and Confirm before the click reaches the page"), and the claim is backed in config — browserTask: {readOnly: true} — not just in prose. It is the model for how the rest of these should read.

    site-builder keeps its place on merit at 0 subscribers: fully declarative, seeded, drift-tested. But note the consequence — with no instance, the third link of the lead chain has never run. list_connections on the lead finder shows exactly one edge, lead.created → draft_outreach. The site.live wiring documented as the chain's payoff does not exist in live data because there is nothing to wire it to.

    The four FIXes

    job-application-assistant — this is the one the guard stops. Its description contains "Runs a safe fill-only test by default", which testAgentMarker matches as a whole word, so blockPublishReason returns a 400 for any attempt to re-publish it; it survives only because it is already published and nothing has sent visibility:"published" since. This is the same one-line edit #66 already wants — "Does a safe fill-only dry run by default and stops before Submit" removes the internal vocabulary and clears the guard. Two further things, both new here: it has 0 calls all-time across 2 subscribers, so the flagship apply pipeline has not been exercised once since usage tracking began on 2026-07-14; and its model column is @cf/meta/llama-3.3-70b-instruct-fp8-fast while every other published agent is claude-sonnet-4-6. The model is harmless in practice — it is in TOOL_CAPABLE_MODELS, and runAgentAi hardcodes claude-sonnet-4-6 whenever a BYOK Anthropic key exists — but it is visible on the public card and it makes the flagship look like the one agent running a small open model.

    small-business-website-lead-finder and lead-outreach-tj6qrr — findings 1 and 2 above. Additionally lead-outreach-tj6qrr still carries its scaffold slug in the public URL and category: "general", the only agent in the catalog with either.

    mcp-client — works (16 calls, mcp_list_tools/mcp_call_tool succeeding on instance dfc1cf92…), but the storefront copy is written for someone who already knows the platform: "Configure its Streamable HTTP endpoint on the private instance, store its bearer token under the MCP key slot … through ProAgentStore's consent-gated MCP connector." Three internal terms in one sentence. Nothing to fix in the capability; rewrite the card.

    The two UNPUBLISHes

    coder → draft. This is the six-overlapping-coding-agents problem, and the repo has already made the call in code. workers/api/src/lib/first-party-agents.ts names it LEGACY_CODER and comments: "The legacy hardcoded Coder: declares no tool allowlist, so it keeps the drive tools." coder-repo supersedes it on every axis — 6 subscribers to its 2, a declared tool allowlist, surfaceOptions, a real settingsSchema — and coder-lead supplies the multi-repo case coder was covering by accident. It is also the only agent in the store attributed to a different creator (proagentstore, with no avatar), so the flagship card is the one that looks least like it belongs to anyone.

    Drafting gates POST /:agentId/subscribe and nothing else, so both existing instances ("Coder", "Coder Home", 739 calls between them) keep working untouched, and it is one field to reverse. What this buys is the thing the catalog is actually for: a visitor sees five coding agents instead of six, and the first one is the one you should start with.

    facebook-friend-confirmer → draft. The second audit recommended this and it was not actioned, so I am re-entering it with the evidence rather than dropping it. It is a public catalog listing for an agent that drives a logged-in Facebook session to confirm friend requests and "optionally adds a few suggested friends" — against Meta's platform terms, and the only social entry in the store. Its 117 calls are entirely the owner's own dogfooding, which is not a usage argument for a shop window. Drafting costs the owner nothing: the instance, its chats and its browser runs are untouched.

    Cosmetic, and it is still the first thing a visitor sees

    store/index.html renders const icon = a.icon || '⚡' on iconBg = a.icon_bg || '#7c3aed'. 8 of 15 published agents have icon: "" — mcp-client, tmux-operator, lead-outreach-tj6qrr, facebook-friend-confirmer, small-business-website-lead-finder, doc-chat, language-buddy, job-application-assistant — so a clear majority of the grid is the identical purple ⚡ tile. Unchanged since 2026-08-06. Eight PUT /v1/agents/:id {icon} calls; I have not picked the emoji because it is the owner's catalog.

    One dead field, worth knowing about

    agents.status is returned in the public GET /v1/agents payload for every agent. It is written once — 'inactive' at INSERT, in both create paths (routes/agents.ts:331, :668) — and nothing ever updates it. The only rows reading active are the ones a seed migration wrote that way, which is why the split falls exactly along seeded-vs-API-created and not along anything meaningful: 8 published agents currently advertise status: "inactive" in the store's own API response, including job-application-assistant and tmux-operator. Nothing in store/ renders it today, so this is latent rather than broken — but it is a public field that means nothing, and the next thing to render it will render a lie.


    Overlaps with #67 — actionable from this same evidence

    27 active instances. Nothing recommended on 2026-08-06 was actioned, and one new near-duplicate has appeared.

    instance id evidence recommendation
    ProAgentStore MCP client a9ee6079… 1 call all-time (usage_summary). Sibling MCP client (dfc1cf92…) has 4 and is the live one. Its failed mcp_list_tools and empty content were established on 2026-08-06 above; I re-verified only that it still exists with 1 call. cancel
    Codex MCP Browser Test 7867ed7e… Still active. Absent from usage_summary entirely — 0 calls all-time. Its self-describing marker content is quoted in the 2026-08-06 comment; I did not re-read it. cancel
    Google Drive Doc Chat Test 3d51aceb… Same — still active, 0 calls all-time, contents established above. cancel
    Terminal Operator · iTerm2 Operator · kitty Operator da39100b… · 8d21ea1a… · 59cc5144… 3 calls each, all-time, on agents bdae7a27… / d40dbdf6… / 01905f59…. Created, smoke-tested once, never used since. All three agents already drafted. cancel all three (owner's call — loses 3 exchanges each)
    Creator OS b619b3da… 8 hand-authored knowledge docs (established 2026-08-06). Do not cancel — re-flagging because the agent's draft status makes it look disposable and it is not. keep + export the docs
    Coder + Coder Home cd489be0… · 12ebf1f0… Deliberate per-machine instances, ~50 msgs each. Unaffected by drafting the agent. keep
    Heartfull + Heartfull (tmux) f8ddc272… (coder-repo) · cda75e28… (tmux-operator) New since the last audit — the same repo now driven by two different agents. Not accidental duplication, but it is the accretion pattern this issue tracks. owner's call
    coder-repo × 6 FAS · FWS · FGS · Heartfull · PAS Coder · Chess coder One per repo is the agent's intended shape. "Chess coder" is new. keep all

    Still unresolved from 2026-08-06: meeting-notes has subscribers: 1 and that instance is not among the owner's 27 — someone outside the operator account holds a subscription to a drafted config:{} agent, with 0 chats and 0 executions. I could not identify them: that needs /v1/admin/*, which is behind the Cloudflare Access edge and which AGENTS.md puts out of bounds. Not touched.

    Also worth a separate issue: pump_test, a leftover test pipeline, is attached to the live lead-finder instance and appears in its run history (3 runs). Harmless, but it will be copied into every subscriber's instance the moment finding 1 is fixed by promoting that instance's pipelines wholesale. Promote lead_finder only.


    Correction to the usage figures on this issue

    usage_summary(range:"all") today: 3,347 calls, costMicros 889,238,209 — but only chargedCostMicros 3,771,185 (~$3.77) is money anyone was actually charged. The ~$889 headline is notional list-price arithmetic, and $842 of it is a single Claude Code engine (claude-opus-5[1m], 113.8M cache-read tokens, payer unknown) running on a subscription. The second audit's "$42.30 estimated" was the same notional figure before that engine's rows landed; quoting either as spend overstates it by two orders of magnitude. 11 of 15 published agents have real usage; all of it is still the owner's own.

  4. serge-ivo commented on Aug 8, 2026

    @serge-ivo
    ContributorAuthor

    Fourth pass — 2026-08-08, evening. Read-only, nothing changed. Two of the third pass's three findings have moved, and one of them is now wrong as written.

    The third pass (2026-08-07 21:06Z) is a good audit and I did not redo its table. I re-verified it
    against live state and against main @ 64598dd, ~21 hours later. In that window six commits
    landed that touch this issue directly, and one of them inverted the meaning of config: {} —
    which is the finding this pass exists for.

    Evidence: MCP list_agents (15 rows), my_agents (39), my_instances (28), usage_summary(range:"all"),
    agent_analytics × 15, list_pipeline_runs, list_errors, coding_repos_list; plus
    curl https://proagentstore.online/ and git show on the six commits. No writes, no REST mutations.


    1. config: {} stopped meaning "unconstrained" today — and it took the lead finder down with it

    This is new since the third pass and it is the most consequential thing on the board.

    #381 shipped this morning (b11a740, 2026-08-08 07:41 +1000): a pipeline step is now bounded by
    what its agent declares. The gate is declaredToolsFor (lib/pipeline-tool-policy.ts:113) and it is
    explicit about the empty case:

    "An agent that resolved fine but declares no tools yields an EMPTY array, which is a real
    assertion: it declares no connector tools."

    #396 (1905259, 10:33 +1000) then moved the refusal to attach time. Both are correct. The
    consequence is that config: {} is no longer a neutral state — it is now an assertion that the
    agent may run zero connector tools
    , and small-business-website-lead-finder's whole behaviour is
    one http_request step.

    6b37070's own commit message says so, unprompted:

    "Separately and reported in #394, not changed: since #381 the live pipeline is refused at kick
    regardless of city, because lead-finder.json names the connector-provided http_request and the
    agent declares no capabilities.tools."

    Correction to a fact in circulation: the refusal dates from today, not from 2026-08-06. I
    checked the commit dates (b11a740 c:2026-08-08 07:41:50 +1000; 1905259 c:10:33:44). The last
    lead_finder run is 2026-08-06 04:38 UTC and it completed (83 seen / 45 added), so no run was
    actually lost — the exposure window is hours, not days. That does not soften the finding, because the
    same shape is still live on three other published agents (§3).

    The agent row now carries a 16-tool capabilities.tools (verified live: http_request plus the KB /
    files / collections set). That was the right call and it is not a narrowing in practice —
    toolNamesFor (agent-do-tools.ts:189-193) seeds the set from BASE before adding the declared
    names, so the always-on base tools survive. search_knowledge / list_knowledge / read_knowledge
    are in the 16, so the Drive/WorkDrive connector gate (64598dd, #352) still passes for it.

    What is still broken and was not fixed by that declaration: the agent row has capabilities and
    nothing else. No pipelines. Third pass finding 1 stands unchanged.

    2. The Sydney crash is fixed — in a file that nothing in production reads

    #394 closed today (6b37070). It is a better diagnosis than the audit's: the audit prescribed
    slice after flatten, and the commit explains why that could not have worked (flatten is the step
    that fails; no later step shrinks an earlier step's journal entry). The real fix projects the four geo
    strings inside the connector's own responseMap, so Google's addressComponents arrays never cross a
    step boundary, plus a slice and a max_places cap with a visible "N were left unexamined" line.

    Measured: the fixed definition lives at workers/api/src/lib/pipelines/lead-finder.json. Its only
    importer in the whole tree is a test:

    $ grep -rn "pipelines/.*\.json" workers/api/src --include='*.ts'
    workers/api/src/lib/step-dispatch.test.ts:19:import leadFinder from "./pipelines/lead-finder.json" with { type: "json" };
    

    The reference now has 10 steps (geocode · fan_out · http_request · flatten · slice · map · enrich · map · filter · dedupe_upsert). Every completed production run reports 8 steps. The commit says
    the same thing in its own words — "It does not touch the live instance's own copy of the pipeline …
    which demonstrably differs from the reference."

    So: the storefront card still promises "Search by city and business type", and Sydney still fails in
    production.
    The fix exists, is tested, and is unreachable from the live agent. This is the sharpest
    possible argument for finding 1 — a pipeline that lives only in one instance's config is a pipeline
    that cannot be fixed.

    3. Three more published agents are one connector step from the same refusal

    Resolved capabilities read live off /v1/instances/my/instances:

    published agent resolved capabilities.tools what that now means
    lead-outreach-tj6qrr null any registry tool → refused at attach
    doc-chat null same
    language-buddy null same

    None of the three uses a connector tool today, so none is broken. Declaring tools on them now would be
    pure narrowing with no benefit, and I am not recommending it. What I am recommending is that the
    platform stop letting a published agent carry config: {} — see §7.

    4. The funnel is no longer structurally zero. Third pass finding 3 is delivered.

    #383 closed today (0a6abf7, migration 0095 agent_funnel_daily). routes/analytics.ts:63-69
    now reads views/trials from a table whose write can succeed, and names the exact defect the third pass
    hypothesised — the usage.user_id foreign-key violation inside a swallowed catch. The hypothesis was
    right; the fix landed before the comment did.

    Measured now, all 15 published agents:

    agent views trials subscribes
    tmux-operator 3 0 2
    coder-repo 2 0 8
    portal-watch 0 1 0
    language-buddy 0 1 1
    the other 11 0 0 —

    So the third pass's conclusion — "this audit cannot tell you whether anyone browses the store, and
    neither can you"
    — no longer holds. The counters work. They are near-zero because the store is
    near-unvisited, which is a different and more useful fact.

    What IS still dead, and nobody has filed it: totalExecutions and recentExecutions. Both read
    agent_executions, and the only INSERT into that table in the entire API is routes/run.ts:98 —
    POST /v1/agents/:id/run, the legacy direct-inference route. So the creator's analytics reports
    totalExecutions: 0 for small-business-website-lead-finder (13 pipeline runs on record) and for
    facebook-friend-confirmer (123 metered calls). Every one of the 15 reads 0. Worth its own ticket:
    either feed it from pipeline_runs + runtime tasks, or stop rendering a counter that measures a route
    nobody calls.

    5. The storefront filter fix is verified in production, not just in source

    The third pass could only report it fixed in main because https://proagentstore.online/ returned
    403 to its fetch. It serves 200 to a browser User-Agent, and the deployed HTML carries the fix:

    $ curl -sS -o store.html -w "%{http_code}\n" https://proagentstore.online/ -H 'User-Agent: Mozilla/5.0 … Chrome/130.0 …'
    200
    $ grep -n "renderFilters" store.html
    175:    <!-- Agent-type tabs. Rendered from the live catalog by renderFilters() — a hardcoded
    265:        renderFilters();
    302:    function renderFilters() {
    

    The second pass's §1 storefront regression is closed, in production. That was the highest-value
    finding on this issue and it is done.

    6. tmux-operator is published, and #402's target ceiling cannot reach it

    Two individually-correct decisions composing into a gap, which is the shape this board keeps
    producing.

    So targets: "single" + a subscriber-bound target — the part of #402 written specifically for "a
    named operator ('Heartfull tmux') should be targets: "single". One pane, one job. A mis-selected
    target here is a shell command on a real machine"
    — can be declared on the three drafted operators
    and not on the one published one.
    Verified live: iterm-operator and kitty-operator carry
    surfaceOptions.terminal.backends; tmux-operator carries no surfaceOptions at all, and
    tmux_run_command is scope: "write" arbitrary shell behind a single per-connector consent row.

    Fix is small — add a tmux entry to CONNECTOR_CONSTRAINTS with a targets binding on the
    session/target argument — and it is the thing #249 should buy instead of a tmux Coder backend. I
    have written it up there.

    7. Zero agents declare a statsSchema — the Stats surface has never had a catalog entry to render

    Checked all 39 agent rows: statsSchema appears in none of them.

    #423 (fixed today, 7852232) established that the rollup had also been crashing on every cron tick
    for 29.6 hours. Both facts are true and they are independent: even with the rollup healthy,
    agent_stats_daily stays empty because nothing declares a schema to roll up. So #310/#311/#312/
    #313 — a per-agent Stats tab, a card schema, an MCP configurator and a daily rollup — are shipped and
    have never rendered a single card in production.

    That is a catalog problem, not an engineering one, and it has an obvious owner: the same agent that
    already needs work in §1–2. small-business-website-lead-finder should be the reference
    statsSchema
    — "leads found per day", "by suburb", "with/without a website" — the way repo-chat is
    the reference for declared tools and site-builder for declarative pipelines. One agent config, and
    the feature stops being theoretical.


    Verdicts — 15 published agents, 2026-08-08 evening

    Usage is all-time, summed from usage_summary(range:"all") across each agent's agent-scoped row and
    its instances' rows (the ledger keys both ways; reading only one undercounts). All of it is still the
    owner's own traffic — 0 external creators, 0 external users.

    slug subs calls agents.config verdict why
    coder-repo 7 833 caps + identity + settingsSchema KEEP Busiest and best-declared: 9 tools, surfaceOptions.coding {repos:single, drive:false, copilot:false}, a real settingsSchema. 109 chats, 25 today.
    coder-lead 1 375 caps + identity + settingsSchema KEEP 93 chats. The multi-repo case, genuinely used.
    local-repo-chat 1 6 caps + identity + settingsSchema KEEP Correctly distinct from repo-chat (own checkout over the runner vs GitHub API).
    repo-chat 1 4 caps + identity KEEP The declared-tools reference (migration 0050).
    doc-chat 1 4 {} KEEP Honest: promises only platform primitives. Flagged in §3, not broken.
    language-buddy 1 0 settingsSchema KEEP The settingsSchema + voice-language reference. 50 chats, 1 trial.
    portal-watch 0 0 browserTask + caps + identity + settingsSchema KEEP Still the best-written card in the store — states its own limits and backs them in config (browserTask.readOnly). Picked up its first trial.
    site-builder 0 0 caps + identity + pipelines + settingsSchema KEEP The only agent that ships its behaviour on the agent row. It is the control case for §1–2.
    tmux-operator 2 111 caps only FIX §6: targets:"single" cannot be declared on it. Also: config carries nothing but capabilities (no identity, no welcome), no icon, and status: "inactive" on a public payload.
    lead-outreach-tj6qrr 1 111 {} FIX 100 draft_outreach runs, all real, all fed by the pump — and the definition exists only in instance 3c09069a…'s config. No reference JSON anywhere in the repo. Export it before anything else. Plus the scaffold slug in the public URL and category:"general".
    small-business-website-lead-finder 1 84 caps only FIX §1 + §2: tools now declared, pipeline still not; the #394 fix cannot reach the live definition. Also the §7 statsSchema candidate.
    mcp-client 2 21 caps + settingsSchema FIX Works — 12 messages on dfc1cf92… show mcp_list_tools/mcp_call_tool succeeding against FreeAgentStore's MCP endpoint, which is a nice demonstration and worth saying on the card. The card instead says "the private instance", "the MCP key slot", "consent-gated MCP connector". Rewrite.
    job-application-assistant 2 0 caps only FIX Unchanged since the third pass: the #65 guard would 400 any re-publish ("a safe fill-only test"), the model column is @cf/meta/llama-3.3-70b-instruct-fp8-fast while all 14 others read claude-sonnet-4-6, and it has 127 chats but 0 metered calls (all pre-2026-07-14). One copy edit closes the guard and #66 together.
    coder 2 872 (seed account) UNPUBLISH → draft Third pass's call, re-entered with one more fact: instance cd489be0… ("Coder") has had no message since 2026-07-08. first-party-agents.ts already calls it LEGACY_CODER; coder-repo has 7 subs to its 2 and the fuller declaration. It is also the only card attributed to a different creator (proagentstore, no avatar).
    facebook-friend-confirmer 1 123 caps + settingsSchema UNPUBLISH → draft Third time this has been entered. A public listing for an agent that drives a logged-in Facebook session to confirm friend requests and "optionally adds a few suggested friends" — against Meta's platform terms, and the only social card in the store. Its 123 calls are the owner's own. Drafting costs nothing: the instance, its chats and its browser runs are untouched.

    KEEP 8 · FIX 5 · UNPUBLISH 2. No agent changed verdict class since the third pass; two acquired new
    reasons (tmux-operator, small-business-website-lead-finder).

    Cheapest-first, so a partial fix ships

    1. Export draft_outreach from instance 3c09069a… to workers/api/src/lib/pipelines/lead-outreach.json. It is the only copy of a working pipeline and it is one cancelled instance away from gone. Zero risk, do it first.
    2. Seed lead_finder (the fixed 10-step definition) and draft_outreach onto their agent rows by migration, the way 0057 does for site-builder, and extend seed-drift.test.ts to cover them — today it pins site-builder only. Do not promote instance 933ebec5…'s pipelines wholesale: it also carries pump_test, a leftover with 3 runs, which would then be copied into every subscriber's instance.
    3. One copy edit on job-application-assistant — "a safe fill-only dry run" — clears the [Catalog] Unpublish smoke-test agents from the live store #65 guard and [Catalog] Fix description-vs-capability integrity + add a claims lint #66's item in the same line.
    4. Add a tmux entry to CONNECTOR_CONSTRAINTS (§6) so the published operator can be bound to one session.
    5. Declare a statsSchema on the lead finder (§7) so the Stats surface has a first tenant.
    6. 8 icons. Still 8 of 15 rendering the identical purple ⚡ tile (mcp-client, tmux-operator, lead-outreach-tj6qrr, facebook-friend-confirmer, small-business-website-lead-finder, doc-chat, language-buddy, job-application-assistant). Eight PUT /v1/agents/:id {icon} calls; they are the owner's aesthetic choices, so I have not picked them.

    Alternatives considered and rejected

    • Declare capabilities.tools on lead-outreach-tj6qrr, doc-chat and language-buddy now, for consistency. Rejected: none of them reaches a connector tool, so it is narrowing with no benefit and three more rows to keep correct. The right guard is at publish time (below), not per agent.
    • A publish-time lint that refuses config: {}. Tempting after §1, and I am not proposing it: doc-chat and language-buddy are legitimately configuration-free, and lintAgentClaims ([bug] The catalog claims lint runs on agent create but not on update — the path where descriptions actually change #362) already catches the case that actually harms a visitor — a description promising something the capabilities do not back. The narrower rule that is worth writing is "a published agent whose behaviour is a pipeline must carry that pipeline on the agent row", which seed-drift.test.ts can assert directly.
    • Merge the two lead agents into one. Rejected: the split is the architecture — lead.created routed by agent_connections is the pump's reference wiring, and 100 draft_outreach runs prove it works. Collapsing it would delete the platform's only live demonstration of agent-to-agent choreography.

    Corrections to figures on this issue

    usage_summary(range:"all"), measured now: 3,602 calls · costMicros 955,247,356 · chargedCostMicros 7,185,088.
    So the honest spend number is ~$7.19, not ~$955 and not the ~$42 the second pass quoted. By payer:
    byok-api 414 calls / $7.18 (real money), platform 96 calls / $0.005, and unknown 3,092 calls /
    $948 — which is a Claude Code engine on a subscription (claude-opus-5[1m], 115M cache-read tokens,
    67 calls, $849.78 notional) plus claude-opus-5 (12 calls, $54.73). Anyone quoting costMicros as
    spend is out by two orders of magnitude; quote chargedCostMicros.

    11 of 15 published agents have real usage. Zero external users. The shop-window framing still stands.

    Acceptance

    The three original boxes were met on 2026-08-06 and stay met — the table above covers all 15 published
    agents with a verdict and a reason, and #65/#66/#67 are filed and linked. Leave this open: #66 is
    open, #67 has fresh evidence today (companion comment there), and items 1, 2 and 5 above are unfiled
    work that belongs to this parent.

    Verified by reading; not verified: I did not open a coding session, run a pipeline, or trigger a
    browser task — every one of those spends the owner's tokens or writes production data.

  5. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Fifth pass — 2026-08-18. Read-only. Three of the fourth pass's six cheapest-first items have landed; the one it called "zero risk, do it first" has not

    Ten days on. I did not redo the verdict table — I checked what moved, against production and origin/main @ 6ceec4f.

    Done since the fourth pass

    • Item 2 (half). migrations/0111_seed_lead_finder_pipeline.sql seeds the fixed 10-step lead_finder onto the agent row, and its header states the mechanism precisely: "the fixed definition landed in a file production never loads — the only importers of lead-finder.json in the whole tree are two tests, and loadPipeline reads agent_instances.config.pipelines[key]." Confirmed live: small-business-website-lead-finder no longer has config: {}.
    • Item 4. CONNECTOR_CONSTRAINTS now has two keys — terminal (lib/surface-options.ts:214) and tmux (:235) — so §6's finding is closed and the published Operator can be bound to one session.
    • Item 5, sort of. Exactly one agent in the account now declares a statsSchema. It is facebook-friend-confirmer — the agent this issue has recommended drafting three times. So the Stats surface has a first tenant and it is the one entry two audits asked to remove.

    Not done — and item 1 is the one that matters

    Item 1 — export draft_outreach before it can be lost — has not happened. workers/api/src/lib/pipelines/ contains exactly lead-finder.json, site-builder.json, site-deploy.json (+ their tests and the README). There is no lead-outreach.json.

    And the agent row is still empty: lead-outreach-tj6qrr is one of only two published agents whose config is {} (the other is doc-chat, which is legitimately configuration-free). So the second link of the lead chain — 100+ completed draft_outreach runs, the platform's only live demonstration of agent-to-agent choreography — still exists only as instance data on 3c09069a…, untested, unversioned, and one cancelled subscription away from gone.

    Ten days ago this was written up as "Zero risk, do it first." It is the only item on that list that cannot be redone from the repo if it is lost.

    (I could not re-read the instance's pipeline definition to confirm byte-for-byte that it is still there — GET /v1/instances/my/instances does not return instance config, and no read-only MCP tool returns a pipeline definition. What is verified is that the agent row is still {} and that no reference JSON exists in the tree.)

    • Item 3 — job-application-assistant's copy still contains the whole-word test that blockPublishReason refuses, so it remains un-republishable.
    • Item 6 — the icons. Not checked this pass; nothing suggests they moved.

    One new published agent, and it belongs in this inventory

    tmux-coder (agent_tmux_coder, migration 0123, #515) — the catalog is 16, not 15. It was seeded published deliberately, with lintAgentClaims and a 22-test seed guard, so it is a clean addition rather than drift. It has one subscribed instance (AIPA tmux Coder, 25501ef7) whose last activity is 2026-08-12 11:44 — the day it was created. Six days idle. Worth knowing before the next pass reads it as a working agent.

    Recommendation for this tracking issue

    The audit work here has been thorough four times over and the ask is long since satisfied. What keeps it open is a small list of unfiled follow-ups, and a tracking issue is a poor home for them — item 1 has now survived two audits by being on a list rather than on a ticket.

    Suggest: file item 1 as its own one-line issue today, file the job-application-assistant copy edit onto #66's successor, and close this. Its verdict tables are the durable artefact and they stay readable whether it is open or shut.

    Verified: the pipelines directory contents, surface-options.ts:214/235, migration 0111's presence and header, the 16-row published catalog, the two config: {} published agents, the single statsSchema holder, and 25501ef7's last-activity timestamp. Not verified: the icon count and the live contents of instance 3c09069a…'s config. Not touched: nothing was published, drafted, edited or cancelled.

  6. added
    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guards
    pipelinesPipelines, steps, triggers and the event pump — what CALLS a connector
    on Aug 18, 2026
  7. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Closing — the audit ask has been satisfied four times over, and what kept this open was a list of unfiled follow-ups, which a tracking issue is a poor home for. Item 1 had survived two audits by being an item on a list rather than a ticket.

    Item 1 is now #706 — the lead-outreach pipeline existing only as instance data on 3c09069a, with no reference JSON and an empty agent row. Filed as P1 because it carries a loss risk nothing else on this list did: every other pipeline can be rebuilt from the repo, that one cannot.

    Remaining items, recorded here rather than lost:

    • Item 3 — job-application-assistant's copy still contains the whole-word test that blockPublishReason refuses, so it stays un-republishable. A copy edit.
    • Item 6 — the icons; not re-checked, nothing suggests they moved.

    Also worth carrying forward from the last pass: the published catalogue is 16, not 15 — tmux-coder (agent_tmux_coder, migration 0123, #515) was seeded published deliberately, with lintAgentClaims and a 22-test seed guard, so it is a clean addition rather than drift. Its single instance (25501ef7) has been idle since the day it was created.

    The verdict tables in this issue are the durable artefact and stay readable closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guardscatalogAgent catalog integrity/cleanupenhancementNew feature or requestpipelinesPipelines, steps, triggers and the event pump — what CALLS a connector

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions