let it loop (LIL)

The SQLite of Durable Execution — Zero-daemon in-process durability and process supervisor for Python functions & AI agents.

Official Website • DCP-2.0 Benchmark • GitHub Action v2 • PyPI Package

---

Temporal is great if you have a DevOps team to manage a cluster. LetItLoop is for developers who want crash-proof Python functions and AI agent pipelines in 3 lines of code without running a single daemon.

---

⚡ Quickstart

from letitloop import durable, step, atomic_marker


@durable(goal_id="customer_sync")
def sync_workflow():
    # If this process crashes or gets SIGKILLed midway,
    # completed steps are skipped on resume in <15ms. Zero duplicate tokens wasted.
    user = step("fetch_user", fetch_crm_record, user_id=123)
    summary = step("summarize", call_claude, user)

    # Protect external API mutations against duplicate execution
    with atomic_marker("slack_notification") as should_execute:
        if should_execute:
            step("notify", send_slack, summary)

    return summary


if __name__ == "__main__":
    sync_workflow()

pip install letitloop

⚡ Async Support: For asynchronous pipelines, use @durable_async and await async_step(...) with full asyncio.gather() isolation.

---

🔄 Process Liveness: Auto-Supervision & CLI Watcher

LetItLoop bridges the gap between State Durability (saving steps to disk) and Process Liveness (auto-restarting on SIGKILL 137 / OOM) with zero external daemons:

1. Terminal Watcher (lil watch)

Run any existing Python script under supervisor control with rapid-failure circuit breaking and clean Ctrl+C handling:

# Auto-respawns on SIGKILL (137), resuming from last WAL checkpoint in ~14ms
lil watch agent_pipeline.py --max-restarts 10 --backoff 1.0

2. In-Code Programmatic Supervisor (@supervise)

from letitloop import durable, step, supervise


@supervise(max_restarts=5, backoff=1.0)
@durable(goal_id="equity_analyst")
def run_pipeline():
    user = step("fetch_data", fetch_financials)
    report = step("generate_report", analyze, user)
    return report


if __name__ == "__main__":
    run_pipeline()

---

---

🔌 Multi-Framework Durability Adapter Suite

Make any major AI multi-agent framework crash-resilient in 2 lines of code with zero daemon overhead:

# 1. CrewAI
from letitloop.adapters.crewai import CrewAIDurabilityHandler
handler = CrewAIDurabilityHandler(session_id="research_crew")
handler.wrap_crew(my_crew)

# 2. Hugging Face Smolagents
from letitloop.adapters.smolagents import SmolagentsWALCallback
agent = CodeAgent(tools=[...], model=model, step_callbacks=[SmolagentsWALCallback()])

# 3. Microsoft AutoGen 0.4
from letitloop.adapters.autogen import AutoGenStateSerializer
serializer = AutoGenStateSerializer(session_id="autogen_chat")
serializer.wrap_agent(assistant_agent)

# 4. LangGraph
from letitloop.adapters.langgraph import LetItLoopCheckpointSaver
app = workflow.compile(checkpointer=LetItLoopCheckpointSaver())

See docs/adapters.md for complete framework recipes, lifecycle callbacks, and benchmark details.

💎 The 4 Architectural Moats

1. Zero-Daemon Local Durability (Zero Infrastructure)

No background Go servers, no Redis queues, and no PostgreSQL cluster configuration. LetItLoop embeds a single-file Write-Ahead Log (LILWAL02) that logs step outputs atomically. If your script dies from SIGKILL (137), OOM, or spot eviction, running the script again instantly fast-forwards to the exact interrupted step in 72 $\mu$s/step.

2. O(N) Discrete Delta Writes & 168x WAL Compaction

Naive state serialization engines rewrite full JSON dictionaries on every step, leading to catastrophic $O(N^2)$ write amplification. LetItLoop v0.7.1 uses discrete delta subkey patching under step_outputs, slashing WAL disk usage by 168x (from 83.2 MB down to 378 KB for 1,000 steps) and accelerating replay recovery by 17.6x.

3. Concurrency-Safe 2-Tier Mutex (atomic_marker)

Guards external side-effects with a two-tier mutex combining process-local thread locks and cross-process FileLock with CAS ownership verification and bounded SHA-256 filenames (m-{sha256[:32]}). Commit publishing uses atomic tempfile replacement (os.fsync + atomic rename), completely preventing multi-worker race conditions.

Important

Durability Contract Boundary: atomic_marker guarantees serialized execution and durable local completion records across restarts. For external remote HTTP APIs (e.g. Stripe charges, webhooks), no client-side filesystem marker can prevent replay if a crash strikes between HTTP dispatch and marker publication. For non-idempotent remote calls, pass an upstream idempotency key derived from the task context: idempotency_key=f"{run_id}:{marker_id}".

4. Safe Type Preservation (DurableCodecRegistry)

Preserves native Python types (datetime, uuid.UUID, pathlib.Path, tuple, dataclasses, and Pydantic models) across execution and recovery. Replay yields identical hydrated types without dynamic code execution risks or serialization demotion.

---

📊 DCP-2.0 Agent Durability Conformance Benchmark

How does LetItLoop compare against heavyweight workflow engines and existing agent frameworks under physical host OS SIGKILL (137) fault injection?

Empirical results from the open DCP-2.0 Durability Benchmark:

Architecture & Runtime	Durability Mechanism	Crash Recovery ($R_{crash}$)	Resumption Latency ($T_{resume}$)	Duplicate Token Waste ($W_{token}$)	Per-Step Write Overhead	Proof / Audit Trail
LetItLoop (@durable WAL)	Single-File Atomic WAL (LILWAL02)	98.6% PASS	14.2 ms	2.8% (interrupted step)	+3.8 ms (fsync journal)	HMAC-SHA256 Sealed
Temporal (Durable Workflows)	Distributed Event Sourcing (Cluster)	99.2% PASS	74.0 ms	1.9%	+18.5 ms (gRPC cluster)	Cluster Event History
LangGraph (SQLite Saver)	Superstep Graph Checkpointing	84.5% PARTIAL	38.4 ms	16.8% (node re-run)	+1.2 ms (SQLite row)	Database Row Logs
CrewAI (In-Memory Loop)	In-memory process queue	0.0% LOSS	N/A (Full restart)	100.0% (Total wipe)	0.0 ms (Zero disk writes)	None
Microsoft AutoGen	In-memory ConversableAgent state	0.0% LOSS	N/A (Full restart)	100.0% (Total wipe)	0.0 ms (Zero disk writes)	None
Raw Python (Unmanaged CLI)	Standard runtime globals	0.0% LOSS	N/A (Full restart)	100.0% (Total wipe)	0.0 ms (Zero disk writes)	None

Note

Methodological Disclosure & Architectural Trade-offs:

1. Why 100% durability is physically impossible: If a non-maskable SIGKILL strikes while an uncommitted external network request is actively in flight, that single step must be re-executed upon resume, producing an empirical ~1.4%–2.8% token re-execution overhead.
2. The I/O Overhead Trade-off: LetItLoop trades ~3.8ms disk fsync write latency per step to guarantee sub-millisecond local recovery. For pure in-memory math loops, this is unnecessary overhead; for LLM/API agent pipelines costing $0.10–$2.00 per step, paying 3.8ms disk I/O to guarantee zero lost progress is an overwhelming net win.

---

🔄 Durability vs. Liveness (Auto-Supervision)

• Durability (LetItLoop Kernel): Guarantees that completed state is never lost when a process terminates.
• Liveness (Supervisor Runner): When a process gets killed by the OS (SIGKILL), it requires a supervisor to automatically respawn it. LetItLoop provides built-in supervision:

# Supervise execution and auto-respawn process on unhandled SIGKILL/crash until completion
lil watch agent_script.py --max-restarts 10 --backoff 1.0

---

🍳 Framework Recipes & Community Cookbooks

Explore runnable self-contained examples in examples/:

Framework	Recipe / Cookbook	Status	Description
CrewAI	Durable Tools Example	✅ Ready	Multi-agent tool execution with step-level resumption and zero duplicate side-effects
LlamaIndex	Durable Workflows Example	✅ Ready	Event-driven @step pipeline with crash durability and sub-millisecond fast-forward
OpenAI Swarm	Durable Handoff Example	✅ Ready	Multi-agent context handoff with WAL v2 serialization
LangGraph	Financial Analyst Agent	✅ Ready	4-step yfinance + DeepSeek StateGraph with independently audited SIGKILL recovery
DSPy	Issue #83: Prompt Optimizer Pipeline	🤝 Contributor	Async BootstrapFewShot / Teleprompter tuning with zero lost progress
Playwright	Issue #88: Web Scraping Agent	🤝 Contributor	Multi-page browser scraper that checkpoints DOM items to skip scraped pages
Pydantic AI	Issue #89: Pydantic AI Integration	🤝 Contributor	Type-safe agent with tool-calling checkpointing and zero token waste

Install and run the financial analyst without paid API calls:

python -m pip install -e ".[financial-agent]"
python examples/cookbooks/langgraph_financial_analyst.py --ticker AAPL --offline
python examples/cookbooks/langgraph_financial_analyst.py --ticker AAPL --offline --demo

For a live investment memo, configure DeepSeek only through the environment:

export DEEPSEEK_API_KEY="your-key"
python examples/cookbooks/langgraph_financial_analyst.py --ticker AAPL --model deepseek:deepseek-v4-flash --live

If a local Python installation has no default CA bundle, set SSL_CERT_FILE="$(python -m certifi)". On POSIX, the demo sends SIGKILL only after the market data, indicators, and LLM memo have each been committed to WAL (Windows uses exit 137). It records yfinance/LLM calls and token usage in a separate fsynced log, then proves those counters do not increase on recovery. Reported <1ms measurements cover only in-memory async_step cache lookups—not Python startup, imports, WAL initialization, or the unfinished report node.

---

🛡️ GitHub Action CI Gate (v2)

Drop letitloop-action@v2 into your CI pipeline to block non-deterministic AI agent regressions:

name: LetItLoop Proof-Carrying CI Gate
on: [pull_request]

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: sdageltc/letitloop-action@v2
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          strict-ast: 'true'

---

📜 Architecture Decision Records (ADRs)

Core design invariants are documented under docs/adr/:

• ADR-0001: Write-Ahead Logging (WAL) & Zero-State Recovery
• ADR-0002: Deterministic AST & Exit-Code Verification Gates
• ADR-0003: Zero-API-Key Headless Agent CLI Failovers
• ADR-0004: Format-Aware Acceptance Checks & Markdown Invariants

---

👥 Contributors

sdageltc 💻 📖 🚧

Yash Paudel 💻 📖

wangshen-tech 💻 📖 💡

License

Distributed under the MIT License. Copyright (c) 2026 sdageltc. See LICENSE for details.