diff --git a/README.md b/README.md index da707c8..dd27a56 100644 --- a/README.md +++ b/README.md @@ -228,7 +228,7 @@ These are mirrored in `AGENTS.md` and re-stated at the top of any system prompt 🌿 **Survival-bot pivot (2026-05-25)** — the bot is becoming a self-sufficient survival resident of the configured server. **MC chat is dialog-only**; operator/player chat commands are recorded but not dispatched (TUI is the only local control plane). The hybrid runtime now has enriched perception, priority modes, a skill-driven curriculum, food acquisition, bed/sleep, base/chest/shelter/farm skills, persistent skill metrics, scenario memory, and a scoped auto-patch loop with `npm test` smoke gating. Full plan: `plans/autonomous-survival-bot-prd.md` (local-only, gitignored). -🌿 **v0.2.0 — self-learning agent (2026-05-27).** SQLite-backed knowledge base at `state//knowledge.db` with seeded recipes / mob intel / block intel / 12 starter lessons. Every death is captured with full context into a `deaths` table; a periodic Pi-coach loop (≤3 calls/hour, 12-min cooldown) batches unanalysed deaths and extracts generalised lessons. The reflex chain consults `coach/advice` before every dispatch — high-confidence lessons can override (`attack creeper` → `survive.flee`) or back-off. Bot narrates major events in Russian MC chat (≤8 lines/hour, ≥75 s gap). Auto-patch now opens a **PR for operator review** instead of cherry-picking onto `main` — `main` is protected, the operator is the only approver. Design: [`docs/v0.2.0-self-learning.md`](./docs/v0.2.0-self-learning.md). +🌿 **v0.2.0 — self-learning agent (2026-05-27).** SQLite-backed knowledge base at `state//knowledge.db` with seeded recipes / mob intel / block intel / 12 starter lessons. Every death is captured with full context into a `deaths` table; a periodic Pi-coach loop (≤3 calls/hour, 12-min cooldown) batches unanalysed deaths and extracts generalised lessons. A 30-min self-reflection loop asks Pi *"are you in a loop or making progress?"* and records the verdict + lessons to `state//reflections/`. The reflex chain consults `coach/advice` before every dispatch (including fallback paths) — high-confidence lessons can override (`attack creeper` → `survive.flee`) or back-off. Bot narrates major events in Russian MC chat (≤8 lines/hour, ≥75 s gap). New escape skill `survive.pillar-up` for pit terrain; wedged-emergency reflex fires it after 60s of no horizontal progress. Auto-patch now opens a **PR for operator review** instead of cherry-picking onto `main` — `main` is protected, the operator is the only approver. Currently shipped: `rc.1` (substrate) + `rc.2` (P0 hardening) + `rc.3` (escape + advice-in-fallback). Current state, next steps, and known issues: [`dev/v0.2.0/STATUS.md`](./dev/v0.2.0/STATUS.md). Original design: [`docs/v0.2.0-self-learning.md`](./docs/v0.2.0-self-learning.md). Full plan: [`docs/roadmap.md`](./docs/roadmap.md). Memory layout: [`docs/memory-model.md`](./docs/memory-model.md). Day-to-day judgement: "Operating principles" in [`AGENTS.md`](./AGENTS.md). diff --git a/dev/v0.2.0/STATUS.md b/dev/v0.2.0/STATUS.md new file mode 100644 index 0000000..0a84c11 --- /dev/null +++ b/dev/v0.2.0/STATUS.md @@ -0,0 +1,236 @@ +# pepa v0.2.0 — status + +> **Read this first** if you're continuing the iteration in a new session. +> It's the single source of truth for "where we are, what shipped, what's next". +> +> Live state snapshot lives in `state//` (gitignored). Code lives on `main`. +> Anything in this folder is shared knowledge for future contributors / sessions. + +**Last update**: 2026-05-27, after rc.3 merged (PR #22). + +**Current code**: `0.2.0-rc.3`. Live bot running on this version since +2026-05-27 15:32 (supervisor PID 31432, child PID rotates on hot-restart). + +--- + +## What v0.2.0 is + +Major iteration over v0.1.x. The bot acquires a **self-learning substrate**: +structured knowledge in SQLite, post-mortem analysis of every death by Pi, +Russian chat narration, retrieval-augmented dispatch (lessons override +planned actions), and a 30-min self-reflection loop where Pi asks the bot +*"are you in a loop?"*. + +It also tightens the self-improvement pipeline: auto-patch now **opens a +PR for operator review** instead of cherry-picking onto `main` (branch +protection enforces — only operator can merge). + +See [`docs/v0.2.0-self-learning.md`](../../docs/v0.2.0-self-learning.md) +for the original design doc. + +--- + +## Shipped (rc.1 → rc.3) + +### rc.1 — substrate ([PR #?]/9c2c56f) +- `runtime/knowledge/{schema.sql,store.js,seed.js,lessons.js,index.js}` + — SQLite knowledge base at `state//knowledge.db`. 9 tables + (recipes, mob_intel, block_intel, lessons, deaths, postmortems, poi, + wiki_pages, chat_log, code_changes). Seeded from + `docs/minecraft-recipes.json` + inline starter intel: 38 recipes, + 15 mobs (creeper/zombie/etc. with `verdict_no_weapon`), 30 blocks, + 12 starter lessons (creeper rule, "first wood→pickaxe→sword", etc.). +- `runtime/coach/postmortem.js` — `bot.on('death')` captures context + (last skill, hostile, nearby journal, scenarios tail) → `deaths` + table. Periodic drain (5min, ≤3 Pi calls/h) batches up to 8 + unanalysed deaths and asks Pi for generalised lessons. Pi reply + parsed into `postmortems` + `lessons` rows. +- `runtime/coach/advice.js` + reflex hooks — `consultAdvice()` runs + before dispatch in `curriculumReflex` and `defendReflex`. Lessons + with `confidence ≥ 0.6` and matching `avoid_skill` swap the planned + skill with `prefer_skill` (if it's in `SAFE_OVERRIDES`). +- `runtime/persona/chatter.js` — Russian narration in MC chat on skill + transitions, threat spotted, day/night, respawn, milestone done. + Rate-limited 8/h, min 75s gap, no-dup suppression. +- `auto-patch.js` switched to PR-open (no more direct cherry-pick to + main). `PEPA_AUTO_PATCH_MERGE=cherry-pick` fallback for emergencies. +- Branch protection on `main`: 1 required approver, `enforce_admins:false`. + +### rc.2 — P0 hardening ([PR #21]/e84148d) +- **PEPA_HEADLESS=1** — `pi -p` subprocesses (banter/coach/planner) no + longer open a second MC connection. + [extensions/mineflayer-bridge.ts](../../extensions/mineflayer-bridge.ts): + `session_start` skips `connect("startup")` when env is set; pi-bridge + sets it on every spawn. +- **Test state isolation** — `runtime/config.js` detects node test + runner / `PEPA_STATE_DIR` and redirects `stateDir` to + `/tmp/pepa-test-state-/`. `npm test` no longer pollutes live + `scenarios.jsonl` / `world-journal.jsonl` / daily log. +- **defendReflex outcome bug fix** — `reportAdviceOutcome` was called + with `succeeded:false` *before* the flee skill returned. Now wired + through `dispatchDefendFlee` onComplete. +- **Mode-name → skill-id translation** in `coach/advice.js`. Pi-coach + prefer values like `"night_shelter"`, `"self_preservation"`, + `"hunger"` now map to `survive.sleep / survive.flee / survive.eat`. +- **Self-reflection loop** (`runtime/coach/reflect.js`). Every 30 min + Pi gets asked *"are you making progress, or in a loop?"*. Verdict + + summary + new lessons. Writes to `state//reflections/.md`, + lessons land in DB with `source="pi-reflect"`. Rate-limited 2/h. + +### rc.3 — escape mechanics ([PR #22]/865aae1) +- **`survive.pillar-up`** — vertical escape skill. Places dirt/cobble/ + planks under self, jumps onto it. Up to 8 steps per dispatch. NO + pickaxe required. +- **Wedged-emergency reflex** — at top of `curriculumReflex`: if no + hostile <6m AND position hasn't moved 16+ blocks in 60s AND placeable + block in inv → force pillar-up. 2-min cooldown. +- **consult() in fallback path** — `curriculumReflex` wander/explore.far + fallback now consults advice too. This closes the gap where Pi-coach + lessons fired but the dispatcher's fallback bypassed them. +- **POI on death** — `coach/postmortem.js` calls + `recordPOI({kind:"danger"})` on every death. 6h expiry. POI table now + populates as bot plays. +- **SAFE_OVERRIDES** extended: + `survive.pillar-up`, + + `village.choose-base`. + +--- + +## Current state (2026-05-27 ~15:30, ~1.5h after rc.3) + +Live DB: +``` +deaths 28 +postmortems 22 (coach analyzed all but 6 — backlog draining at 5min/batch) +lessons 34 (12 starter + 22 Pi-extracted) +lessons_applied 1 ← still very low; advice consult fires but most matches fail +poi 22 (rc.3 fired recordPOI on every death since 14:17 deploy) +chat_log 0 (chatter narrates in MC chat but doesn't write to DB table yet) +reflections 2 (14:51, 15:22 — 30-min interval working) +``` + +The bot is **alive but still struggling**. Coach is producing high-quality +lessons (see `state//reflections/2026-05-27T12-21-58-656Z.md`: +"Я не продвинулся к цели… выбрать новое безопасное дневное место"), but: + +1. Pi-coach + Pi-reflect routinely invent `prefer_skill` values that + aren't registered skill ids (e.g. `choose.safe.surface`, + `relocate.daylight`). `normalisePreferSkill` only knows mode names, + not these. So even the right advice can't dispatch the right action. +2. The bot's home terrain (~608, 90) is genuinely bad — stone walls, no + pickaxe to break out, hostiles spawn nearby. pillar-up *can* help but + it needs a placeable block AND open sky above. In a closed cave it + places dirt and hits a ceiling. +3. `gather.stone` returns `missing_tool` correctly (rc.0 was already + fine here), but curriculum doesn't auto-route to `craft.wooden-pickaxe` + — it just backs off and tries the same thing. + +--- + +## Known issues / followups (rc.4 candidates) + +Ranked roughly by impact. + +### 1. Pi-suggested `prefer_skill` is often a hallucinated action name +**Symptom**: reflections produce `prefer: choose.safe.surface`, +`prefer: relocate.daylight`, `prefer: defend.self` — none are +registered skills. `SAFE_OVERRIDES` check fails, lesson falls back +to bare `avoid` (which dispatches nothing). + +**Fix options**: +- (a) Include the full registered-skill list in the Pi prompt so it + picks from valid IDs. +- (b) Add a fuzzy mapper: "choose|find|new + base|spot|place" → + `village.choose-base`. "relocate|move + safe|day" → `explore.far`. +- (c) Both. (b) is a fallback safety net. + +**Where**: `runtime/coach/{postmortem.js, reflect.js}` prompt builder + +`runtime/coach/advice.js` `normalisePreferSkill`. + +### 2. Tool-progression auto-craft +**Symptom**: `gather.stone` returns `missing_tool` → curriculum backs +off → next tick tries `gather.stone` again → loop. Should auto-suggest +`craft.wooden-pickaxe`. Same pattern for axe-required gathering. + +**Fix**: In curriculum or in a new "tool-need" reflex, when a gather +skill returns `missing_tool`, look up the tier requirement (already in +`block_intel.required_tool`) and route to the matching craft skill if +materials are available, else to the prerequisite gather skill. + +**Where**: `runtime/reflex.js` or new `runtime/coach/tool-progression.js`. + +### 3. `village.choose-base` should penalise danger POI +**Symptom**: bot may pick a new base that's adjacent to a danger POI +(a recent death site). rc.3 records the POIs but `choose-base` scoring +doesn't read them. + +**Fix**: In `runtime/base-site.js` `scoreCurrentPosition`, query +`poiNearby({kind:"danger", x, z, radius: 32})` and subtract a penalty +proportional to recent danger count. + +### 4. Pillar-up perimeter sense +**Symptom**: in a closed cave, pillar-up places a block and hits the +ceiling block above. Useless. + +**Fix**: before starting pillar-up, scan a 3×3 above current head; if +solid blocks in the column path, abort and request `recovery.tunnel-out` +instead. Or walk to a column-clear cell within reach first. + +### 5. `chat_log` table not populated +**Symptom**: 0 rows in `chat_log` despite bot narrating ~8 lines/hour. + +**Fix**: `runtime/persona/chatter.js` `sendChat()` calls `logChat()` +from knowledge index. Also wire the inbound side in `bot.js` +`bot.on("chat")` handler. + +### 6. Reflections don't write lessons to the DB until they parse +**Symptom**: confirmed working; just noting that the verdict + summary +themselves aren't queryable. Currently only `lessons` table benefits +from reflection. + +**Possible fix**: add a `reflections` table mirroring the JSON output. +Or run lessons-from-reflection through the same advice path so they +get `applied_count`. + +### 7. Re-narration spam +**Symptom**: persona narrated `за дровами` twice in 6 seconds after +hot-restart. Cooldown is per-process state, lost on restart. + +**Fix**: persist `_lastNarrationAt` and `_narrationTimes` in a small +file in `state//` so restarts inherit the cooldown. + +--- + +## Next session — quick start + +1. **Read this file first.** +2. **Check live DB** for the latest counters: + ```bash + sqlite3 state/play.xmatic.team_25565/knowledge.db \ + "SELECT 'deaths', COUNT(*) FROM deaths + UNION SELECT 'lessons', COUNT(*) FROM lessons + UNION SELECT 'lessons_applied', COUNT(*) FROM lessons WHERE applied_count > 0 + UNION SELECT 'poi', COUNT(*) FROM poi;" + ``` +3. **Read the latest reflection** for current Pi assessment: + ```bash + ls -t state/play.xmatic.team_25565/reflections/ | head -1 + ``` +4. **Check bot is alive**: `ps aux | grep "runtime/(bot|supervisor)"`. +5. **Pick a followup from §"Known issues"** (1 and 2 are the highest + impact: actually-applying-lessons + tool progression). + +--- + +## Workflow notes for next session + +- `main` is protected — only the operator merges PRs. Auto-patch from + the bot itself opens PRs via `gh pr create`. +- Tests: `npm test` (255+ green at last check). Tests run isolated under + `/tmp/pepa-test-state-/` and don't pollute live state. +- The bot supervisor watches `runtime/**/*.js` and hot-restarts on + file change. After merging a PR, `git pull` locally and the + supervisor picks up the new code. +- Stop bot: `bash scripts/stop.sh`. Start: `npm run bot` (inside + tmux is fine). +- Memory in `state//` is server-specific and gitignored. +- Don't bypass branch protection — the auto-mode classifier will block it. diff --git a/docs/v0.2.0-self-learning.md b/docs/v0.2.0-self-learning.md index ac7601e..d9930a2 100644 --- a/docs/v0.2.0-self-learning.md +++ b/docs/v0.2.0-self-learning.md @@ -153,37 +153,74 @@ This is the loop that closes "I died yesterday in this exact spot attacking a creeper with fists" → "tonight I avoid attack and dig shelter instead". -## Phases — what ships in which rc +## Phases — what shipped -### v0.2.0-rc.1 (this branch) +> Live status, observed metrics, and rc.4 followups: [`dev/v0.2.0/STATUS.md`](../dev/v0.2.0/STATUS.md). + +### v0.2.0-rc.1 — substrate (merged 2026-05-27) - [x] WIP baseline commit (pathfinder/reflex/metrics fixes). -- [ ] This design doc. -- [ ] `better-sqlite3` dep + `runtime/knowledge/{store,schema,seed,index,lessons}.js`. -- [ ] Seed knowledge DB from `docs/minecraft-recipes.json` + the - mob/biome/food/recipe tables in `docs/minecraft-knowledge.md`. -- [ ] `runtime/coach/postmortem.js` — death event capture, deaths table, - Pi-extracted lessons (initial version uses async one-shot, no batch). -- [ ] `runtime/persona/chatter.js` — narration on death/respawn, - gather-start, threat-spotted (templates only, no Pi yet). -- [ ] Wire-in: `bot.js` calls `coach.attach(bot)` and - `persona.attach(bot)` once on connect. -- [ ] Tests for store, lessons, postmortem context capture, persona - cooldowns. +- [x] This design doc. +- [x] `better-sqlite3` dep + `runtime/knowledge/{store,schema,seed,index,lessons}.js`. +- [x] Seed knowledge DB from `docs/minecraft-recipes.json` + inline + starter tables: 38 recipes, 15 mobs, 30 blocks, 12 starter lessons. +- [x] `runtime/coach/postmortem.js` — death event capture, deaths table, + Pi-extracted lessons (5 min drain, ≤3 Pi calls/h, 12 min cooldown). +- [x] `runtime/persona/chatter.js` — narration on death/respawn, gather + start, threat spotted, dusk/dawn, milestone done (templates). +- [x] Wire-in: `bot.js` calls `coach.attach`, `reflect.attach`, + `persona.attach` once on connect. +- [x] Retrieval-augmented dispatch via `runtime/coach/advice.js` — + `consult()` before curriculum + defend reflexes. SAFE_OVERRIDES + whitelist guards what skills lessons may dispatch into. +- [x] Auto-patch loop switched to PR-on-merge instead of direct + cherry-pick. Branch protection on `main`: 1 required approver. +- [x] Tests for store, lessons, postmortem context, advice + consultation, persona cooldowns. -### v0.2.0-rc.2 +### v0.2.0-rc.2 — P0 hardening (merged 2026-05-27) -- [ ] Wiki crawler with cache + parser. -- [ ] Retrieval-augmented dispatch (curriculum + dispatch hooks). -- [ ] `runtime/persona/look.js` — head movement. -- [ ] `runtime/coach/index.js` — periodic batched Pi pass. +- [x] `PEPA_HEADLESS=1` guard in `extensions/mineflayer-bridge.ts` so + `pi -p` subprocesses don't open a second MC connection. +- [x] Test state isolation — `runtime/config.js` redirects `stateDir` + to `/tmp/pepa-test-state-/` under node test runner. +- [x] `defendReflex` outcome bug — `reportAdviceOutcome` now fires + AFTER the flee skill returns, not before. +- [x] Mode-name → skill-id translation (`night_shelter` → + `survive.sleep`, etc.) in `coach/advice.js`. +- [x] **Self-reflection loop** (`runtime/coach/reflect.js`) — every + 30 min Pi gets asked *"are you in a loop?"*. Verdict + summary + + new lessons. Reflection written to + `state//reflections/.md`. -### v0.2.0 (final) +### v0.2.0-rc.3 — escape mechanics (merged 2026-05-27) -- [ ] Tune rate limits + thresholds with real data from the prior weeks. -- [ ] Cross-server lesson abstraction (raw → committed `skills/.md` - when applied repeatedly across situations). -- [ ] Roadmap update. +- [x] `survive.pillar-up` — vertical pit escape, NO pickaxe required. +- [x] Wedged-emergency reflex — auto-fires pillar-up after 60s of no + horizontal progress + no hostile + placeable block. +- [x] `consult()` runs on curriculum's wander/explore.far fallback + path (closes the gap where Pi-coach lessons fired but were + ignored by the dispatcher's fallback). +- [x] `recordPOI(kind:"danger")` on every death — spatial memory. +- [x] `SAFE_OVERRIDES` extended with `survive.pillar-up`, + `village.choose-base`. + +### rc.4 candidates — see [`dev/v0.2.0/STATUS.md §"Known issues"`](../dev/v0.2.0/STATUS.md) + +1. Fix hallucinated `prefer_skill` from Pi (e.g. `choose.safe.surface`). +2. Tool-progression auto-craft: `gather.stone → missing_tool → + craft.wooden-pickaxe`. +3. `village.choose-base` ranking should penalise nearby danger POI. +4. Pillar-up perimeter sense — abort if ceiling above. +5. Persist persona cooldowns across restarts. +6. Populate `chat_log` table on every chat in/out. + +### Wiki crawler — pushed to v0.2.1 + +`runtime/knowledge/wiki.js` still planned but isn't on the critical +path while the seeded intel + Pi-extracted lessons are sufficient. +Will land when the bot has stabilised enough that "learning new things +from the wiki" beats "applying things it already learned". ## Out of scope for v0.2.0