rm-core

The contracts that connect ai-rm components

English · 한국어

Quick start · ai-rm integration · Architecture · Protocol · Roadmap

---

rm-core is the contracts package of ai-rm. It defines how components talk to each other — accelerators (L1), memory tiers and KV transport (L2), workloads and placement (L3–L4) and the shared metric schema — so that components never import each other. It holds types, interfaces, reference implementations and conformance checks only: standard library, Python 3.9+, no runtime dependencies. The normative specification is PROTOCOL.md (v0.1); this is release 0.1.1.

✨ Highlights

Typed contracts per layer Accelerator / AcceleratorProvider (L1), MemoryTier / KVLayout / KVTransport (L2), Workload / Placement (L3–L4), with validation that raises ContractError.	Selected by name Components register adapters as Python entry points and runtimes load them with rm_core.registry.load(group, name), so switching an implementation is a configuration change.	Checked by conformance python -m rm_core.conformance runs the contract checks for every registered implementation and reports PASS, SKIP (no hardware) or FAIL.
One metric language rm_core.metrics.METRICS fixes names, units and tags (TTFT, TPOT, tok/s, power, MFU, KV transfer …); Prometheus series are named airm_<name>.	Kubernetes representation The AcceleratorProfile CRD (airm.ai-computing.io/v1alpha1) and node labels, built and read back by rm_core.k8s.	Reference implementations sim, loopback and greedy need no hardware. They define the expected behavior and are the shape to copy when writing a real adapter.

🚀 Quick start

pip install "rm-core @ git+https://github.com/ai-computing/rm-core@v0.1.1"
python -m rm_core.conformance            # check every registered implementation

With only rm-core installed, the reference implementations are checked:

PASS  airm.accelerator_providers:sim
PASS  airm.kv_transports:loopback
PASS  airm.placements:greedy

Check one group or one implementation: python -m rm_core.conformance <group> [name ...]. The command exits non-zero if any implementation fails.

🔌 ai-rm integration

Components implement a contract in a small adapter module inside their own repository and register it under an entry-point group:

Entry-point group	Contract	Registered by	Reference
airm.accelerator_providers	AcceleratorProvider	L1 (device discovery)	sim
airm.kv_transports	KVTransport	L2 (data movement)	loopback
airm.placements	Placement	L4 (placement policy)	greedy
airm.workloads	workload runner (§5.3)	L3 (serving, training)	—

Implementing a contract

# your_pkg/airm.py
from rm_core import KVTransport, TransferPlan, TransferHandle, TransferResult, TransportError

class P2PTransport:
    name = "p2p"
    def supports(self, src, dst): ...
    def plan(self, request_id, src, dst, src_layout, dst_layout, tokens): ...
    def submit(self, plan): ...
    def wait(self, handle, timeout_s): ...
    def close(self): ...

# your pyproject.toml
[project.entry-points."airm.kv_transports"]
p2p = "your_pkg.airm:P2PTransport"

python -m rm_core.conformance airm.kv_transports p2p

Runtimes then select it by name from configuration (kvTransport: p2p):

from rm_core import registry
transport = registry.load("airm.kv_transports", "p2p")

Reference implementations to copy from: rm_core.reference (SimAcceleratorProvider, LoopbackTransport, GreedyPlacement). Components depend on rm-core with a compatible pin, e.g. rm-core>=0.1,<0.2 (§1).

Key rules from the protocol

• Accelerators — id is <node>/<vendor>:<index>, stable across restarts. peak_tflops is dense peak per precision and needs a peak_source; unknown values are omitted, never zero. discover() returns [] (not an exception) when no device is present.
• KV transport — plan() validates its inputs and raises TransportError; wait() returns only when the data is visible at the destination; a transport never silently falls back to another path. Transports that need real devices implement conformance_case() and raise rm_core.conformance.NoHardware when they are absent (reported as SKIP).
• Placement — returns only ids from the pool, is deterministic, and returns an empty decision with a reason (not an exception) when nothing fits.
• Metrics — the quantities in METRICS are reported under exactly those names and units; tags come from TAG_KEYS; mfu_pct carries peak_source, power_w carries scope=card|node.
• Security — contracts, profiles, metrics and logs never carry internal addresses, credentials, tokens or keys; site values come from the environment.

Full text: PROTOCOL.md.

Shared metrics (rm_core.metrics.METRICS)

Name	Unit	Meaning
ttft_p50_ms · ttft_p90_ms · ttft_p99_ms	ms	time to first token
tpot_p50_ms · tpot_p90_ms · tpot_p99_ms	ms	time per output token
e2e_p90_ms	ms	end-to-end request latency, p90
tok_s	tokens/s	generated tokens per second, whole system under test
req_s	requests/s	completed requests per second
power_w	W	mean power over the measured window (tag scope=card|node)
energy_j	J	energy over the measured window
gflops_per_w	GFLOPS/W	power efficiency
mfu_pct	%	model FLOPs utilization (tag peak_source)
kv_transfer_ms	ms	KV handoff time per request
kv_transfer_gbps	GB/s	KV transfer bandwidth
accuracy_pct	%	task accuracy (tag dataset)
slo_attainment_pct	%	requests meeting the SLO
accelerators	count	accelerators in use (tag vendor/model)

Prometheus series: airm_<name> (rm_core.metrics.prometheus_name). Tag keys: component, model, accelerator, vendor, scope, peak_source, dataset, workload, kv_transport, run_id, commit.

🏗 Architecture

src/rm_core/
├── accelerator.py   # L1: Accelerator, AcceleratorProvider; vendor, memory, precision, interconnect enums
├── memory.py        # L2: MemoryTier, KVLayout, TransferPlan/Handle/Result, KVTransport
├── workload.py      # L3–L4: Workload, PlacementDecision, Placement; workload kinds, SLO keys
├── placement.py     # L4: airm-placement document (decisions.json): model, validate(), build()
├── metrics.py       # METRICS, TAG_KEYS, Measurement, prometheus_name
├── k8s.py           # AcceleratorProfile objects and node labels; from_profile()
├── registry.py      # entry-point groups: available(), load(), groups()
├── reference.py     # sim / loopback / greedy (no hardware)
├── conformance.py   # python -m rm_core.conformance
└── errors.py        # ContractError, TransportError, NoHardware
k8s/acceleratorprofile.crd.yaml   # AcceleratorProfile CRD (airm.ai-computing.io/v1alpha1)
k8s/airm-placement.schema.json    # JSON Schema of the airm-placement document (PROTOCOL §5.4)

A node daemon publishes each device as a cluster-scoped AcceleratorProfile plus summary node labels airm.ai-computing.io/<vendor>.<model>=<count> and airm.ai-computing.io/managed=true; rm_core.k8s builds both and reads them back. Profiles never contain addresses or credentials.

🧪 Development

pip install -e .
python -m unittest discover -s tests -v

CI runs the tests on Python 3.9–3.12, then python -m rm_core.conformance and python tools/check_sensitive.py . (workflow).

🗺 Roadmap

• Protocol v0.1: contracts, AcceleratorProfile CRD, reference implementations, conformance checks
• 0.1.1: AcceleratorProfile keeps extra, NoHardware reported as SKIP under python -m, metric migration rule
• 0.1.2 (unreleased): airm-placement document normative (§5.4), rm_core.placement validator, JSON Schema, placement conformance validates the document
• Protocol v0.2: MemoryRegion (owner, fd/offset, size) in transfer plans, so device-to-device transports can bind live buffers and pass conformance with a hardware fixture
• Protocol v0.2: formalized workload-runner interface (airm.workloads)

Change log: PROTOCOL.md § Change log.

📚 Documentation

Specification	PROTOCOL.md · AcceleratorProfile CRD · placement schema
ai-rm	Umbrella · Interfaces · Integration guide

📄 License

BSD-3-Clause.

_{Part of ai-rm · ai-computing team · ETRI}