jev-doom

Overview

An LLM-driven agent (Jev / TypeSafe's MODEL=jev-latest) that plays VizDoom's defend_the_center scenario. The agent makes one structured decision per tick — indexed target + shoot/hold + danger score — without using pixels. Everything is numeric/textual: enemy positions, bearings, distances, player state. A geometry-only heuristic baseline (no API key) is included for comparison.

Technologies

Key Features

• Structured LLM Decisions - Jev outputs typed choices: target: Choice(idx.../none), fire: Choice(shoot/hold), danger: Score(0–2) — no free-form text.
• 1-Flight Async Pipeline - While the next decide() runs in a background thread, the agent holds the last safe action. The ~10-tic latency gap is spent tracking the picked target (re-aim at 4-tic cadence, re-fire only after a shoot pick).
• Latency-Gap Compensation - Snapshots carry a latency lead (lead_tics, EMA of gap length): enemies dead-reckoned along velocity, facing advanced along the held turn.
• Geometry-Only Baseline - Synchronous heuristic: nearest visible enemy by distance, fire within ±8°, else turn toward it. No API key needed, serves as honest comparison.
• Deterministic Evaluation Suite - game.set_seed(seed) before each episode; seeds 1–5 reported with mean±std. Logs to runs/*.jsonl + .summary.json (final hp/ammo/kills).
• Live Visualizer - doom.watch shows game on left, Jev's mind on right: enemy context list, decision with confidences, latency/tokens/hp/ammo/kills.

Results

Jev (indexed target + latency-gap tracking): 10.20 ± 2.04 kills (seeds 1–5)

Heuristic baseline (geometry only, no API): 14.80 ± 1.94 kills

The honest baseline out-kills Jev — it never hesitates, never mis-picks, fires the instant something centers. Jev's cost is the experiment itself: API latency (~10 tics/decision) plus occasional wrong picks. What Jev buys is judgment the baseline lacks — target lock (finish one enemy before switching), sustained chained bursts, and flinch-turns toward off-screen shooters.

Commands

# Jev on defend (the 10.20±2.04 setup)
uv run python -m doom.play --brain jev --suite

# Watch it play
uv run python -m doom.play --brain jev --seed 1 --visible

# No-API baseline for comparison
uv run python -m doom.play --brain heuristic --suite

# Scoreboard (seeds 1..5)
uv run python -m doom.compare

# Live visualizer: game + Jev's mind
uv run python -m doom.watch --brain jev --seed 1

# Play yourself
uv run python -m doom.human

Links