feat(profiling): emit lucegraph concurrency capture in-tree

[Graffioh]

Why

Lucebox Forge (Luce-Org/luce_box#259) profiles serving performance with LuceGraph. Its JSONL capture producer (lucebox.concurrency.v1) has lived only as an out-of-tree patch in a skill asset. That patch sits 509 commits behind main, conflicts in 11 files, and still uses the pre-rename DFLASH_* variables. Main's ROCTX markers also carry no round_id, so a kernel trace cannot be joined to scheduler rounds. This PR brings the producer in-tree, off by default.

What changes

• Capture emitter, in server/src/common/observability/:

  • inference_profile covers a round's lanes and 14 phases.
  • concurrency_capture handles bounded keep-first retention, JSONL export, and atomic checkpoints.
  • It is enabled with LUCE_PROF=concurrency (or 1; the value is a comma list, as the existing LUCE_PROF already is).
  • The other settings are LUCE_PROF_OUT, LUCE_PROF_MAX_ROUNDS, LUCE_PROF_MAX_REQUESTS, LUCE_PROF_MAX_TOKEN_BURSTS and LUCE_PROF_CHECKPOINT_EVERY.

• One round id per engine step, owned by the scheduler. The capture step and these ROCTX ranges all carry the same round_id:

  • Qwen: service_round, concurrent_step and graph_compute;
  • DeepSeek4: paged_gathered_step.

• Chain speculation rounds get their own concurrent_step range and the step path chain. They used to be unknown, because step_chain_spec returned before a step range was opened. The consumer schema accepts chain in Dflash issue on V100 build #259.

• Off by default. When LUCE_PROF is unset:

  • the engines receive a null profile pointer;
  • phase scopes read no clock;
  • the capture allocates nothing and writes no file.

  What remains is one branch on a cached bool and a counter increment per round.

• Not ported: the old patch's HTTP observability endpoint and HTML page.

Validation

• Built luce_server and the unit tests for HIP gfx1201 (Ninja, Release) with no new warnings in the touched files.
• These tests pass:
  • test_concurrency_capture (new). It writes a real capture through the scheduler/engine API and checks record order, required fields and types, enums, escaping, drop counts and checkpoints.
  • test_qwen35_roctx and test_deepseek4_roctx, both extended with round_id.
  • test_seq_engine_contract, test_seq_slot_manager and test_parallel_prefix_txn.
  • 21 scheduler and concurrency tests in test_server_unit, run with LUCE_PROF both unset and set to 1.
• The LuceGraph loader and report builders in Dflash issue on V100 build #259 accept both the test's capture and one written by the real scheduler loop with a fake engine.

Not verified

• No GPU run yet: live marker placement around draft and target kernels, a complete zero-drop capture, and a rocprofv3 join are still pending.
• The CUDA build was not compiled; the changes are host-only C++.
• NVTX ranges are not added, because the tree has no NVTX dependency.
• DeepSeek4 steps report path unknown with scheduler-level fields only.
• git_sha is fixed at CMake configure time.

🤖 Generated with Claude Code