pepa-pi-bot/docs/runtime.md

Runtime — hybrid script + LLM-on-demand

Status: active. This is the recommended way to run pepa-pi-bot since 2026-05-25. The pure Pi runtime (pi from repo root) still works and is documented as a fallback at the bottom of this file.

Phase 0 product pivot (2026-05-25). MC chat is now dialog-only for everyone, including OPERATOR_USERNAMES. The reflex loop no longer takes commands from chat. Local control lives in the TUI (p/s hotkeys); long-term control lives in the repo. See plans/autonomous-survival-bot-prd.md for the survival-bot pivot.

Why a hybrid runtime?

The original design ran every tick inside Pi — the LLM saw the world, picked one tool, executed it, looped. That gave full self-extension out of the box, but had three problems in practice:

1. Slow. A "look around → defend yourself" round-trip took 20–60 seconds because the LLM was in the hot path.
2. Expensive. Hostile mob at 4 m? Cost of evasion = one full reasoning pass. Hungry? Same. Idle? Same.
3. Invisible. With Pi as the only frontend, you had to tmux capture-pane to know what the bot was doing.

The hybrid runtime splits the bot into a script-driven layer that handles fast, well-understood things on its own, and a Pi (or Codex) headless escalation that's only invoked when the script gets stuck or needs to write new code for itself.

Architecture

┌────────────────────────────────────────────────────────────────────┐
│  operator                                                          │
│  ├── repo edits (.env, skills/, runtime/)                          │
│  ├── TUI (Ink) — see status, send chat, press [a] to escalate      │
│  └── (future) Telegram bridge                                      │
└─────────────┬────────────────────────────────────────────┬─────────┘
              │ Unix socket (newline-JSON)                 │ git
              ▼                                            ▼
┌────────────────────────────────────────────────────────────────────┐
│  runtime/bot.js — single long-running Node process                 │
│                                                                    │
│  ┌──────────────────┐  ┌──────────────────────┐  ┌────────────────┐│
│  │ Mineflayer       │  │ Reflex loop          │  │ IPC server     ││
│  │ - MC TCP         │  │ - tick every N sec   │  │ - Unix socket  ││
│  │ - AuthMe handler │◀─│ - priority order:    │─▶│ - broadcasts   ││
│  │ - chat / events  │  │   defend > eat       │  │   status/log/  ││
│  │ (dialog-only)    │  │   > sleep > tech     │  │   chat events  ││
│  │                  │  │   > autonomous       │  │ - accepts      ││
│  │                  │  │ - NO LLM in path     │  │   commands     ││
│  └──────────────────┘  └─────────┬────────────┘  └────────────────┘│
│                                  │                                 │
│                                  ▼ on stuck / new scenario         │
│                        ┌──────────────────────┐                    │
│                        │ pi-bridge.js         │                    │
│                        │ spawn `pi -p`        │                    │
│                        │ stream stdout to IPC │                    │
│                        └──────────────────────┘                    │
└────────────────────────────────────────────────────────────────────┘
              │
              │ TCP 25565
              ▼
       Minecraft server

The bot is one process. The TUI is a separate process you can connect and disconnect at will — the bot keeps running. Multiple TUI clients can attach to the same bot simultaneously.

Quickstart

# Once
cd ~/Projects/pepa-pi-bot
npm install

# Terminal 1 — the bot daemon
npm run bot
# Logs go to stdout AND state/<host>/logs/<YYYY-MM-DD>.log

# Terminal 2 — the dashboard
npm run tui

The TUI auto-reconnects to the bot if you restart it. Press q to leave the TUI; the bot is unaffected.

TUI hotkeys

Key	Effect
p	Pause / resume the reflex loop (MC connection stays).
s	Stop the bot process gracefully (disconnect + cleanup + exit).
r	Force-broadcast a status snapshot now.
c	Enter chat mode — type a message, Enter sends it into MC chat.
a	Enter ask-Pi mode — type a prompt, Enter spawns pi -p and streams output into the Pi panel.
k	Enter run-skill mode — type skill.id plus optional JSON args; dispatches once through cmd:run-skill.
v	Capture a headless viewer screenshot into state/<host>/screenshots/ when viewer support is available.
!	Force a stuck incident proposal through the critic/auto-improve path for diagnostics.
y	Open the latest pending proposal. In the proposal panel: y approves, n/Esc closes.
q	Quit TUI only. Bot keeps running.

Enter submits, blank submit cancels. The status bar shows [proposals N, press y] when there's something pending.

What the reflex loop does today

The chain (highest priority first), wired and dispatching real Mineflayer actions:

1. Modes — Mindcraft-style interrupts run first: self_preservation, hunger, night_shelter. They dispatch registered skills (survive.flee, survive.eat, survive.sleep) and therefore feed metrics, scenario memory and current-task state.
2. defendReflex — closest hostile within 4 m → attackNearest (equips best melee). Low HP / close hostile → flee.
3. eatReflex — food < 16 and edible item is carried → eatBestFood. No-food states fall through to the curriculum instead of eat-spamming.
4. sleepReflex — night + no hostile within 8 m + bed available → sleepInBed. Impossible states are skipped before dispatch.
5. curriculumReflex — dispatches snapshot.curriculum.plan.skillId through runSkill. Inventory pressure can insert village.deposit-surplus before continuing.
6. idleReflex — every 20th tick, log heartbeat (HP / food / pos).

There is no operator-goal reflex anymore. MC chat does not create movement/build/mining tasks (Phase 0 of plans/autonomous-survival-bot-prd.md).

Adding a new reflex = a function (ctx) => { action, ... } in runtime/reflex.js, inserted at the right priority. Actions live in runtime/actions.js. Both files trigger a supervisor hot-restart when saved (see "Self-improvement" below).

When the bot calls Pi

Two escalation paths:

1. Manual — operator presses a in the TUI, types a question, the bot spawns pi -p "<question>" and streams its stdout into the Pi panel.

2. Automatic — every tick where the entire reflex chain returns noop (no hostiles in reach, food fine, day or no bed, nothing to craft, nowhere to wander) increments a counter. When the counter hits ESCALATE_AFTER_NOOPS = 20 (≈1 min at tick=3s), the bot fires askPi with the current snapshot and a fixed system prompt telling Pi to suggest one next action. 10 min cooldown so a permanently-idle bot doesn't run the LLM dry.

The auto-escalation prompt explicitly bans code-change proposals — Pi should only suggest what to do with the existing tools. If a deeper problem is happening, the failure-tracker (see Self-improvement) will file a proposal instead.

Observability (Phase 1 — survival-bot pivot)

Every STATUS snapshot now carries fields the TUI uses to answer "what is the bot doing and why isn't it doing more?" without parsing the log stream:

Field	Meaning
runtimeState	finite-state classification: emergency / working / recovering / planning / social / idle (see runtime/state.js).
activeSkill	current dispatched action label, or the last one if idle.
currentMilestone	first uncompleted line from state/<host>/plan.md (cached 30 s).
lastResult	{ label, ok, code, detail, ts } of the most recent dispatched action.
noProgressReason	one of waiting_for_day, night_hostile_nearby, no_food_source, inventory_full, no_reachable_target, planner_empty, awaiting_action_cooldown, … emitted when position + inventory have not changed for ≥60 s (see runtime/no-progress.js).
failuresByCode	rolling counts of recent failures grouped by class (bug / timeout / feature-gap / other).
lastEscalation	{ ts, ageMs } of the most recent Pi auto-escalation.
reflexPaused	mirror of the local pause flag (so TUI shows the right state immediately).

Skill substrate (Phase 2)

Lives under runtime/skills/. A skill is a small composable unit of survival behaviour with a uniform contract:

export const skill = {
  id: "namespace.action",
  title: "Human label",
  timeoutMs: 45_000,
  preconditions(ctx)       -> { ok, code?, detail? }
  async execute(ctx, args) -> { ok, code, detail, worldDelta }
  validate?(ctx, result)   -> boolean       // optional
  recover?(ctx, result)    -> any | null    // optional
}

Registered skills are dispatched via runSkill(id, ctx, args) from runtime/skills/index.js. The runner enforces the timeout, normalises the result shape, runs validate() and calls recover() on failure so the scheduler can act on the hint (e.g. "switch to wander"). Stable failure codes the runner itself emits live in RUNNER_CODES (unknown_skill, precondition_failed, timeout, threw, validation_failed, done).

Item/block groups are dynamic: runtime/skills/groups.js exposes logs(bot), planks(bot), sticks(bot), beds(bot), foods(bot), axes(bot), pickaxes(bot), swords(bot) — every group is derived from bot.registry, so a version-sensitive item that doesn't exist on the connected server simply doesn't appear in the set and skills return code: "unsupported_version" instead of crashing.

Reference skills shipped today include gather.logs, gather.stone, gather.wool, survive.eat, survive.flee, survive.sleep, survive.acquire-food, explore.wander, explore.far, village.choose-base, village.place-chest, village.deposit-surplus, village.build-shelter, farm.wheat, and the craft.* progression. Some low-level action primitives still live in runtime/actions.js, but they are increasingly called behind skill contracts so metrics and self-improvement see them.

Run the contract + groups + curriculum tests:

npm test

Survival curriculum (Phase 3)

runtime/curriculum.js exposes a deterministic early-game progression:

wood.16 → wood.planks-and-sticks → wood.tools →
survive.bed → stone.32 → stone.tools → food.basic →
storage.chest → shelter.torch → village.base-site → village.shelter

Each milestone has an isDone(inventory, snapshot) predicate and a suggest(inventory, snapshot) function that names the next skill to dispatch. isDone is stateful in the sense that reaching a later tier implies all earlier "gather" milestones are complete (so the bot doesn't loop back to "gather 16 logs" after crafting them into planks).

The current curriculum result is on every snapshot as snapshot.curriculum = { milestone, plan, inventoryFull } so the TUI can show what the bot is working on and which skill should drive it. The scheduler now calls runSkill(plan.skillId, …) directly from curriculumReflex; no LLM is in the hot path.

Inventory pressure: isInventoryFull(snapshot) is exposed on every curriculum result; the TUI surfaces [inventory full], and when a known/nearby chest exists the scheduler tries village.deposit-surplus before the next milestone.

Optional: prismarine-viewer

Set VIEWER_PORT=<port> in .env to launch prismarine-viewer in-process. The package is not a default dep — install it explicitly (npm i prismarine-viewer) before enabling. If missing, the runtime logs a warning and continues.

Memory: world-journal + scenario-memory (2026-05-26)

Two persistent stores under state/<host>/:

• world-journal.jsonl — append-only log of discovered points ({kind, name, at:{x,y,z}, ts}). Skills feed it automatically via worldDelta on each successful dispatch — chops, mines, placements, base/shelter location, planted/harvested crops, plus dead_end markers on no_target / silent_dig_failure. Indexed by a 16-block spatial grid so nearest({kind, x, z, radius}) is O(neighbors). Pruned at 6 h age + 10k line ceiling. leanestQuadrant({x, z}) returns the cardinal quadrant the bot has explored LEAST — used by explore.far to circle rather than retread the same patch.
• skill-metrics.json — persistent per-skill ok/fail counters, last code and duration. snapshot.skillMetrics exposes the loaded totals so proposals learn from previous runs, not only the current process lifetime.
• scenarios.jsonl — sliding window of (skillId, situationHash, code, ok, detail, ts) tuples. situationHash is a coarse fingerprint of where + how the bot was (16-cell + 8y bucket, day/night, food bucket, hp bucket, biome, nearby block groups, inventory key set, closest hostile name). The curriculum reflex calls memory.shouldSkip({skillId, situation}) — ≥3 failures of the same (skill, situation) within 30 min and the reflex auto-converts into a wander hint instead of re-dispatching the failing skill. A subsequent success in the same situation un-locks it.

Both stores feed stuck-incident proposal bodies: when the LLM is asked to patch a stuck state, it sees byKind journal counts AND the last 12 scenario-memory entries, so it can write a structural fix based on what's actually been tried, not just one snapshot.

Scheduler driven by the curriculum (2026-05-26)

The reflex chain is now: defend → eat → sleep → curriculum → idle. Modes run before that chain and dispatch their own skills immediately.

curriculumReflex reads snapshot.curriculum.plan.skillId (populated by runtime/curriculum.js each tick) and dispatches it via runSkill(skillId, ctx). Recovery hints flow back through ctx.skillBackoff:

• If the skill's recover() returns { hint: "wander" } (e.g. gather.logs returns code: "no_target" because there's no tree within 32m), the curriculum reflex swaps to wander for ~60 s.
• If the result code is missing_tool / missing_material / no_target / no_food_source / unsupported_version / storage blockers, that specific skill backs off for 60 s instead of retrying every tick.
• Unknown skillId (curriculum suggested something that isn't registered yet) falls through to wander — useful while we wire future skills like village.build-shelter.

The old techTreeReflex and autonomousReflex were removed — the curriculum + craft skills cover their territory, and unit tests in runtime/reflex.test.js exercise the new dispatch paths.

Chat banter escalation to Pi (2026-05-26)

When social/intent.js classifies an inbound line as ADDRESSED_BANTER and templates can't answer, bot.js spawns a one-shot askPi with the bot's runtime state + the last 5 lines from that speaker (redacted via chatMemory). The reply is capped at 200 chars and sent as a single chat line.

Hard rate limit so banter can't drain the LLM budget: 6 calls per hour, minimum 90 s between calls. Suppressed escalations log once and silently drop.

Base-site scoring + locations (2026-05-26)

• runtime/locations.js — atomic JSON store at state/<host>/locations.json. setLocation(name, {x,y,z,…}), getLocation(name), nearestLocation({x,z}).
• runtime/base-site.js — scoreCurrentPosition(bot) returns {score, reasons, position} based on wood/stone/water proximity, surface flatness, distance to other players and absence of foreign builds (man-made blocks not in the owned-blocks ledger).
• runtime/skills/choose-base.js — village.choose-base scores the current spot; if score ≥ 8 it writes locations.base, otherwise emits code: "too_weak" with a wander recover hint.
• The curriculum has a new final milestone village.base-site that fires village.choose-base until a base is established.

Compatibility hardening (Phase 7)

Several modules now guard against the live regressions PRD §7 Phase 7 explicitly calls out:

• runtime/movement-profiles.js — named profiles (GATHER, TRAVEL, FLEE, BUILD, RETURN_TO_BASE) as pure descriptors; applyProfile(profile, bot) writes a fresh Movements to pathfinder. Stops one skill's canDig=false from leaking into the next skill's path.
• runtime/owned-blocks.js — JSONL ledger of blocks this bot placed (and removed). isOwned({x,y,z}) for O(1) lookups. ensureDir() makes the dir lazily so first-write doesn't fail.
• runtime/claim-avoidance.js — classifyArea({blocks, isOwned}) returns player_build / natural_or_owned / insufficient_data based on man-made block density vs ownership ratio. Designed for the gather/place skills to call before touching a contested area.
• runtime/skills/compat.test.js — runs groups.js against real minecraft-data registries for 1.18.2, 1.20.4, 1.21.5; verifies that version-sensitive blocks (e.g. pale_oak_log) only appear where they should.

Self-improvement v2 (Phase 6)

Two classes of proposals now land in state/<host>/proposals/:

1. Bug-class failures — same-label action returns {ok: false} 5× in a row, dominated by bug (TypeError, "Cannot read properties") or persistent timeout. Handled by the older tracker in bot.js.
2. Stuck incidents — noProgressReason stays the same for ≥5 min without a productive dispatch. Handled by runtime/stuck-incident.js. The proposal body includes the runtimeState, milestone, suggested skill, slim snapshot, last result, and per-skill success/failure metrics.

Both kinds now persist an editScope in their frontmatter — an array of repo-relative path prefixes the auto-patcher is allowed to modify. scripts/auto-patch.js enforces that scope, runs the cheap scripts/lint-patch.js gate, then runs npm test before cherry-pick.

Learning speed is configurable:

• PEPA_LEARNING_MODE=fast or dev lowers stuck detection and auto-improve cooldowns for active training sessions.
• PEPA_STUCK_THRESHOLD_SECONDS, PEPA_STUCK_COOLDOWN_SECONDS, PEPA_AUTO_IMPROVE_COOLDOWN_SECONDS, PEPA_AUTO_IMPROVE_MAX_PER_HOUR override those defaults directly.

Social layer (Phase 5)

Inbound MC chat is classified via runtime/social/intent.js into one of GREETING / STATUS_QUESTION / ADDRESSED_BANTER / COMMAND_LIKE / UNSAFE_REQUEST / AMBIENT. The classifier uses Unicode-aware boundaries so "Привет всем" lands as GREETING while "build me a tower" stays AMBIENT until the bot is addressed.

runtime/social/reply.js turns an intent into a short reply:

• GREETING → one of a small picked-randomly set ("yo" / "привет" / …).
• STATUS_QUESTION → live snapshot summary: active skill, current milestone, hp/food/position, no-progress reason, last diary line.
• COMMAND_LIKE → dialog-only notice (per Phase 0).
• UNSAFE_REQUEST → terse "logged for operator review", plus an entry in state/<host>/escalations.jsonl.
• ADDRESSED_BANTER → templates can't reliably answer, so the generator returns escalate: true. Today bot.js does NOT spawn Pi from this path (keeps the LLM out of the hot path); a rate-limited escalation lands in Phase 6.

runtime/social/memory.js maintains an LRU per-speaker buffer of recent lines (default 8 per speaker, 16 speakers max) and redacts password / api-key / JWT-shaped tokens before they ever exit the runtime.

In-game chat (dialog-only)

As of the Phase 0 survival-bot pivot, MC chat does not drive bot actions for anyone, including names listed in OPERATOR_USERNAMES. The bot will:

• reply to greetings (hi, привет, etc.) and to being addressed by name, rate-limited;
• answer status questions (pepa_bot status, как дела, what are you doing?) from the live snapshot;
• detect command-like verbs (come, follow, build, pause, stop, give, …) when addressed, record them in the diary, and reply once per cooldown that MC chat is dialog-only.

Local control of the bot (pause/resume/stop, sending chat manually, escalating to Pi) lives in the TUI. Long-term control (skills, runtime code, proposals) lives in the repo. OPERATOR_USERNAMES is still used to label speakers in logs (operator <name> vs player <name>), and remains the right place to plug a future trusted control channel (e.g. Telegram bridge).

IPC protocol

Socket: state/<MC_HOST>_<MC_PORT>/bot.sock (permissions 0600, removed on shutdown). Framing: one JSON object per line.

Server → client events (see runtime/ipc-protocol.js):

Type	Payload
hello	{ snapshot, recentLogs } — sent on connect.
status	full snapshot from perceive.js.
log	{ ts, level, source, text, details } — every log line.
chat	{ from, text, kind: "player" | "system" }.
death	{ reason, position }.
error	{ source, text }.
ask-pi-chunk	{ stream: "stdout" | "stderr", text }.
ask-pi-done	{ code, durationMs }.

Client → server commands:

Type	Payload	Effect
cmd:pause	{}	Reflex loop stops ticking.
cmd:resume	{}	Reflex loop resumes.
cmd:stop	{}	Graceful shutdown of the bot.
cmd:chat	{ text }	Sends text into MC chat (rate-limited).
cmd:ask-pi	{ prompt }	Spawns pi -p "<prompt>".
cmd:snapshot	{}	Force a status event now.
cmd:run-skill	{ skillId, args? }	Pauses the reflex loop long enough to dispatch one registered skill.
cmd:screenshot	{ reason?, frames? }	Captures a headless viewer screenshot for visual debugging.
cmd:force-incident	{ kind?, reason? }	Forces a critic-backed proposal, useful for testing self-improvement.

The protocol is intentionally tiny — anyone can write a second client (a Telegram bridge, a web UI, a one-shot CLI) by reading runtime/ipc-protocol.js.

Self-improvement loop (fully autonomous)

End-to-end, no operator-in-the-loop. The bot writes proposals when it spots a real bug, applies them with Pi headless, and rolls them back if they break things. The flow:

1. reflex dispatches action → action returns { ok: false, detail }
2. bot.js failure tracker classifies the detail:
     bug          → TypeError / Cannot read / is not defined …
     timeout      → "timed out after Ns"
     feature-gap  → "no reachable log", "no food", "no bed" …
     other        → anything else
3. 5 consecutive failures with the SAME label, where the run is dominated
   by 'bug' or all 'timeout' → writeProposal()
   (feature gaps are SKIPPED — reflex routing solves those, not the LLM)
4. runtime/auto-improve.js watcher (poll 2s) sees the new file,
   debounces 10s, then spawns scripts/auto-patch.js detached
5. auto-patch.js:
     - refuses on dirty tree
     - moves proposal pending → approved/ (audit trail)
     - creates branch auto/<slug> off main
     - runs `pi -p "<patch prompt>"` with 10-min timeout
     - if Pi committed AND changed only the proposal editScope
       (+ runtime/**/*.test.js) AND lint/npm-test pass → cherry-pick onto main
     - else → discard branch, exit non-zero
6. supervisor's runtime/*.js watcher fires the moment the cherry-pick
   lands → child restarts on the new code
7. if the new code crashes >5 times in 60s AND the last commit on main
   is younger than 15 min AND it touched runtime/ → supervisor
   `git reset --hard HEAD~1` and restarts. Up to MAX_ROLLBACKS times
   per supervisor lifetime, then bails out for manual investigation.

Rate limits

• Proposal cooldown: 30 min between proposal files of any kind.
• Auto-improve cooldown: default 15 min between finished auto-patch.js runs; PEPA_LEARNING_MODE=fast|dev lowers this to 5 min unless overridden.
• Hourly cap: default max 4 auto-patches per hour; fast/dev default is 8.
• Rollback cap: 3 rollbacks per supervisor lifetime; after that the supervisor exits and waits for human review.

What counts as a bug

runtime/bot.js ships two whitelists (NORMAL_FAILURE_SUBSTRINGS and BUG_FAILURE_SUBSTRINGS). The proposal trigger fires only when:

• the trailing run of same-label failures contains at least one bug (TypeError / ReferenceError / "Cannot read properties" / etc.),
• OR every failure in the run is a timeout (and they happened on the same operation, so it's probably broken not just unreachable).

Feature gaps like "no reachable log within 32 blocks" are not a bug — the autonomous reflex sees that result, sets noTreesUntil and switches to wander. If the script can't solve it via reflex routing, that's a design issue the operator fixes by editing runtime/reflex.js directly — not by asking Pi to patch around it.

Manual escape hatches

These still work but should rarely be needed:

• TUI hotkey y opens the latest pending proposal for inspection.
• npm run propose:apply <filename> runs the attended version of the patcher — leaves the result on a feat/proposal-<slug> branch without cherry-picking, so the operator can review the diff manually.
• npm run stop kills everything and clears lock/socket.

File layout

runtime/
  supervisor.js       forks bot.js, watches runtime/*.js, restart-on-change
  bot.js              entrypoint — owns MC + tick + IPC + reconnect
  config.js           reads .env, exposes frozen config + redacted view
  log.js              ring buffer + stdout + daily file + IPC fan-out
  perceive.js         snapshot(bot) → JSON
  reflex.js           priority chain (operator > defend > eat > sleep > idle)
  actions.js          attackNearest / fleeFrom / eatBestFood / sleepInBed / goTo
  state-store.js      current-task / diary / proposals on disk
  ipc-server.js       Unix-socket server
  ipc-protocol.js     shared contract (event types, command types, framer)
  pi-bridge.js        spawn `pi -p`, stream stdout

tui/
  tui.tsx             Ink dashboard (React)
  ipc-client.js       socket client → EventEmitter

scripts/
  propose-apply.js    approved-proposal → feat-branch + `pi -p` patcher

Per-server state stays under state/<MC_HOST>_<MC_PORT>/, gitignored:

state/play.xmatic.team_25565/
  bot.sock                   Unix-domain socket (perms 0600, ephemeral)
  joined-before.flag         AuthMe /register vs /login marker
  current-task.json          resume anchor — what the bot was doing
  goal.md                    long-term ambition (operator-seeded)
  diary/YYYY-MM-DD.md        daily journal (one line per milestone)
  proposals/                 pending self-improvement proposals
  proposals/approved/        approved, waiting on propose:apply
  logs/YYYY-MM-DD.log        full runtime log mirror

Pi-only fallback

The original Pi-driven runtime still works if you prefer the single-process model — npm run agent from repo root loads AGENTS.md and the existing extensions in extensions/. The two runtimes share the .env, the mineflayer deps, and the state/ directory. They MUST NOT run simultaneously — both will try to claim the same MC nickname and the server will kick one of them.

If you switch between them frequently, kill one before starting the other:

# stop hybrid
# (in TUI press 's', or just kill `npm run bot`)

# start Pi
npm run agent