Design feedback: verifying leased work outside a SQLite transaction without duplicate acceptance

[Andylaw95]

Discussion type

Question

Context

I'm developing a small open-source coordination prototype for voluntary compute sharing, with a longer-term aim of making AI compute more accessible. At present it only runs fixed synthetic image statistics on three trusted CPU processes on one computer; no AI model, remote volunteers, payments, or GPU claims are involved.

I'm looking for design feedback on one concrete problem: moving expensive result verification outside a coordinator lock without allowing expired or replayed work to be accepted or credited twice.

Current implementation and evidence

Jobs split into chunks. Each lease has an attempt ID and a deadline. In Coordinator.accept, the coordinator holds a lock/SQLite transaction while checking the attempt, recomputing the result, marking the chunk accepted, and inserting a unique ledger entry. Expired attempts can be reissued. This gives a simple baseline, but long recomputation can block heartbeats and other requests.

The relevant implementation is in prototype/engine.py. The repository includes a validation report: 15 local tests passed, including process exit/retry, stale attempts, invalid results, and concurrent result replay. Full recomputation took roughly as much compute time as the worker in these small trials. Worker and verifier also share an implementation, so this does not rule out common bugs.

Reproduce with Python 3.12 on macOS or Linux:

git clone http://github.com/Andylaw95/community-ai-compute.git
cd community-ai-compute
python3 -m examples.quickstart

This opts in to a temporary local CPU job; it verifies 96 images and cleans up its workers/database.

Proposed approach and questions

My starting proposal is to capture an immutable candidate result and attempt version, verify in a bounded worker pool outside the transaction, then perform a short conditional transaction that rechecks attempt ownership, lease validity, and whether the chunk was already accepted before inserting the ledger entry.

• Which state/version checks and race-condition tests would you recommend for expiry/reassignment during verification? Would an explicit verifying state and generation counter make this clearer?
• Are there established implementations or patterns worth reusing for this boundary? I would also welcome explanations of where this proposal is flawed.

I am asking for architectural feedback, not a finished implementation. The present scope is trusted local processes; correctness and affordable validation on hostile remote machines remain separate open problems.

Guidelines and assistance

I have read the Programming Help guidelines. This question and prototype were prepared with assistance from OpenAI Codex (GPT-6); the linked report describes the actual local verification. This is a project question, not an official GitHub response.

[github-actions[bot]]

Thank you for your interest in contributing to our community! We currently only accept discussions created through the GitHub UI using our provided discussion templates. Please re-submit your discussion by navigating to the appropriate category and using the template provided.

This discussion has been closed because it was not submitted through the expected format. If you believe this was a mistake, please reach out to the maintainers.