A distributed execution runtime built from first principles — custom TCP protocol, DAG scheduler, AOF-backed store, and agentic runtime in a single zero-dependency JAR of 129 KB.
5-Minute Quickstart · Full Documentation · How Titan Compares · Architecture
Titan is a distributed execution runtime with a custom DAG scheduler, binary wire protocol, AOF-backed KV store, and agentic execution engine. The core engine is a single JAR with zero external dependencies — it schedules, routes, and executes across a cluster with nothing else installed. The dashboard (Flask) and MCP server (mcp package) are optional extensions on top. It runs on a bare VM or laptop.
Submit jobs in YAML or Python. Titan resolves dependencies, routes to capable workers, streams logs back, and recovers from crashes via AOF replay. At the agentic tier, tasks can spawn new tasks mid-execution, share state across nodes through TitanStore, and pause for human approval before continuing.
It covers three capability tiers in one binary:
| Tier | What you run |
|---|---|
| T1 — Distributed Task Scheduler | Batch jobs, static DAGs, GPU/CPU-routed workloads, delayed execution |
| T2 — Service Orchestrator | Long-running APIs and daemons with auto-restart and port management |
| T3 — Agentic Runtime | Self-mutating DAGs, LLM-driven agents, multi-agent pipelines with HITL gates |
v1.0 research status. Single-Master topology (Raft on the v2 roadmap), process-level isolation (Docker on v2), no mTLS yet. Built to be understood, not to replace Kubernetes or Temporal in production today.
Titan Runtime, in 31 slides — the condensed version: the problem, the architecture, what is in the box, honest scoring against Airflow / Ray / Nomad / Temporal, and what v1 is not. Read it before the docs if you are deciding whether Titan is relevant to you.
Titan ships with a built-in Python Flask dashboard. Five views — four for your pipelines, two for the scheduler running them:
Real-time view of every connected worker — capability tag (GENERAL / GPU / HIGH_MEM), active job count, running services, and recent activity. Includes a + Launch Worker button to spin up new nodes from the browser.
Every pipeline submitted to the cluster — via CLI, SDK, YAML, or the visual Constructor — is automatically rendered as a live dependency graph. Node colours update in real-time as jobs move through PENDING → RUNNING → COMPLETED / FAILED. Click any node to stream stdout/stderr live.
Browser-based drag-and-drop DAG editor. Add task and service nodes, draw dependency edges, configure script, capability, priority, and HITL gates per node — then deploy directly to the cluster in one click. Auto-generates equivalent Python SDK and YAML as you build.
Groups all DAG stages that share an agent_run_id into a single timeline row. When an agent runs iteratively (PLAN → ITER → EVAL → SYNTH), each stage is a separate DAG submission — Agent Runs reconstructs the full lifecycle so you don't have to hunt through individual entries.
Every job dispatch on an absolute time axis. Each span records when a job became eligible and when it actually started, so time spent queued is shown separately from time spent running — the distinction a status view cannot make. Bars that overlap really ran at the same moment.
Gives you the critical path and its share of wall clock (high share means adding workers will not help), real parallelism, retries as separate attempts, and a failure reason that distinguishes a script failure from a readiness failure from a worker that never replied. Spans are kept in memory and written to disk for 7 days, so a post-mortem survives a Master restart.
The other views describe your pipelines; this one describes the system running them. Pending reasons in the scheduler's own words, capability dead ends with the command that fixes them, and four pre-dispatch lanes (delayed, blocked, ready, parked) that a single "pending" count cannot separate.
The dispatch loop is serialized, so its duration is the scheduler's throughput ceiling — it is broken down into routing, worker selection, bookkeeping, store writes and the network hand-off, with a share table. Plus saturation, failure rate, store read/write latency, per-node host CPU/memory/load, a live service roster with dialable addresses, and scaling history with the reason recorded for every scale-up and descale.
No metrics stack, no exporter, no Grafana — everything is sampled inside the Master and served from the same process.
| Method | Best for |
|---|---|
| YAML file | Repeatable, version-controlled pipelines. Commit to git and re-run any time. |
| Python SDK | Programmatic pipelines where shape is determined at runtime — agent loops, dynamic fan-out. |
| Visual Constructor | Building pipelines without writing code. Drag nodes, draw edges, deploy in one click. |
| MCP (natural language) | Controlling Titan from Claude Desktop or Cursor. Describe what you want — the agent writes the scripts and submits the DAG on your behalf. |
All four paths produce the same result: a tracked DAG in the visualizer with per-job logs, status, and workspace files.
Build a pipeline by dragging nodes and drawing edges — deploy to the cluster with one click.
Screen.Recording.2026-05-17.at.10.23.48.PM.mov
A DAG pauses at a checkpoint and waits for a human Approve/Reject before downstream jobs resume.
Screen.Recording.2026-05-17.at.10.25.48.PM.mov
HITL gate mid-execution on a multi-branch pipeline — shows how the visualizer reflects the paused state.
Screen.Recording.2026-05-17.at.10.29.38.PM.mov
A multi-stage agent loop — each stage is a separate DAG submission, grouped into one timeline in Agent Runs.
Screen.Recording.2026-05-17.at.11.22.29.PM.mov
More: Dynamic DAG Execution, Reactive Scaling, GPU Routing, Fanout
Control Plane: Dynamic DAG Execution
dynamic_dag.mp4
Reactive Worker Scaling
titan_load_scaling.mp4
GPU Affinity Routing
GPU_Affinity_yaml.mp4
Parallel Execution (Fanout)
fanout_yaml_dag.mp4
Full Load Cycle (Scale Up & Descale)
titan_load_descaling.mp4
# 1. Build the engine
mvn clean package -DskipTests
# 2. Start the cluster (Master + 2 workers + TitanStore + dashboard)
./titan-dev.sh up
# 3. Open the dashboard
open http://localhost:5000Submit your first job:
from titan_sdk.titan_sdk import TitanClient, TitanJob
client = TitanClient()
client.submit_dag("hello-world", [
TitanJob(job_id="hello", script_content="print('hello from Titan')")
])Full walkthrough: 5-Minute Quickstart
Three components:
- Control Plane (Master) — DAG scheduling, dependency resolution, capability routing, AOF-backed state recovery
- Workers — Capability-tagged execution nodes that self-register on startup and re-register after restart
- TitanStore (optional) — AOF-backed KV store for crash recovery and cross-job shared agent state. No external database required.
The wire protocol (TITAN_PROTO) is a custom fixed-header binary format over raw TCP — no JSON on the dispatch path.
Titan ships a built-in MCP server. Connect Claude Desktop or Cursor and control your cluster in natural language:
"Research three approaches to distributed ML scheduling. Analyze gaps, methodology, and open problems in parallel. Synthesize into a report."
Titan executes the parallel jobs, fans results into a synthesis job, and returns the output — all from a single prompt. No terminal, no code.
Execution
- Static DAGs via YAML or Python SDK
- Dynamic DAG mutation at runtime — tasks can spawn new tasks mid-execution
- Long-running services alongside batch jobs in one runtime
Routing & Scaling
- Capability-based routing — tag workers
GPU,HIGH_MEM, or custom; jobs are held until a matching node is free - Affinity routing — pin jobs to specific workers by tag
- Least-loaded dispatch across available workers, by concurrent job count
- Reactive auto-scaling — when a worker's queue saturates, it spawns child worker processes on the same machine to absorb the spike; idle burst workers decommission automatically after 45 seconds
Worker Lifecycle
- Permanent vs. ephemeral workers — mark nodes as permanent so they stay alive across job completions; burst workers exit when idle
- Workers re-register with the Master on reconnect — the cluster heals through a Master restart without manual intervention
- Orphan cleanup on startup — workers scan for and kill leftover processes from a previous crash before accepting new work
Resilience
- AOF crash recovery — Master replays state on restart, resumes in-flight DAGs
- Worker re-registration — cluster recovers through a Master restart without manual intervention
- Callback retry with exponential backoff
- In-flight job recovery — when a worker fails a heartbeat, the jobs it was holding are closed with a reason and re-queued through the normal retry path, so a dead node cannot strand work or block dependent jobs
- Orphan reconciliation — a periodic check enforces the invariant that no job is counted running on a worker that has left the fleet, covering scaler reclaim and decommission as well as crashes
Observability
- Live DAG visualizer with per-node status, and real-time log streaming from any job
- Agent Runs timeline for multi-stage agent workflows
- One span per dispatch attempt — eligible/start/end, worker, attempt, priority, parents — so queue wait is attributed per job
- Critical path and parallelism computed from the graph
- Dispatch loop broken down by phase (route / select / record / store / send) with a share table
- Saturation, failure rate, throughput, store read and write latency, heartbeat round-trip per node
- Host CPU / memory / load average per worker, reported on the heartbeat
- Live service roster with dialable addresses and uptime
- Queue wait by declared priority, and pipeline duration trend across runs
- Span history persisted to day-partitioned JSONL (7 days), surviving Master restarts
- Markdown / JSON metric export, and a built-in demo runner with seven preset workflows
Human-in-the-Loop
- Native HITL gates — pause a DAG at any checkpoint, Approve/Reject from the dashboard
- Configurable timeout (default 48 hours)
- Automatic gate injection via SDK
Titan runs locally out of the box. When you're ready to move to the cloud, package_cloud.sh builds two deployment bundles:
./package_cloud.sh
# → titan-master-bundle.zip (~2.3 MB) — everything needed on the Master VM
# → titan-worker-bundle.zip (~120 KB) — Worker.jar + titan_sdk for remote workers- Multi-VM Setup — permanent cluster on GCP / AWS / Azure
- Remote GPU via SSH Tunnel — keep your local machine as Master, tunnel a RunPod or cloud VM as a worker with no open ports
| Example | What it shows |
|---|---|
| Build Your First Agent | Writer → Critic loop — simplest agentic pattern in ~60 lines |
| Human-in-the-Loop Pipeline | ML pipeline that pauses for human Approve/Reject before training |
| Multi-Agent Research Pipeline | Parallel agents + HITL gate + synthesis fan-in |
| Static YAML Pipelines | Diamond patterns, GPU routing, parallel fan-out |
| LangChain Integration | Wrap the Titan SDK as LangChain tools — no MCP needed |
Titan is an experimental runtime built from first principles by a single developer. Bug reports, edge case findings, and contributions are encouraged.
Report an Issue · How to Contribute
Licensed under the Apache License 2.0. © 2026 Ram Narayanan A S.