pepa-pi-bot/docs/v0.2.0-self-learning.md

# pepa v0.2.0 — self-learning agent

> Status: **active** (started 2026-05-27). Carries forward the WIP that landed
> in the baseline commit of this branch (pathfinder watchdog, reflex
> metric-recovery, persistent skill metrics, new skills like `flee`,
> `acquire-food`, `place-chest`, `sleep`).
> Local-only repo — this file ships in the working tree but is not pushed.

## Why v0.2.0

The v0.1.x bot survives in the abstract — reflex chain, curriculum, modes,
auto-improve — but on the live server it spent ~15 hours overnight in a tight
death spiral (≈300 deaths, never moved off spawn, no tools, no progress).
Three failure modes dominate:

1. **No persistent knowledge.** Each tick reasons from scratch. The bot has
   no structured memory of "what I tried", "what worked", "what the recipe
   for X is", "what kills this mob". `scenarios.jsonl` is a forgetful
   sliding window keyed by a coarse situation hash, not a queryable store.
2. **No coaching from failure.** A death is just a respawn event. Pi is
   never asked "why did I die, what should I do differently next time".
   Lessons aren't extracted and aren't recalled before the next attempt.
3. **No outside knowledge.** The bot doesn't read the wiki, doesn't know
   that creepers need 4-block kite distance, that fists vs hostile at
   night is suicide, that bread is the cheapest renewable food.

v0.2.0 fixes the substrate so future iterations get steeper progress per
hour-of-runtime. The bot becomes Voyager-shaped: structured skill library +
persistent learned-skill memory + post-mortem-driven self-improvement +
retrieval-augmented decision making + free chat persona.

## Pillars

### 1. Structured knowledge base (SQLite)

A single per-server `state/<host>/knowledge.db` (gitignored) with these
tables:

| Table | Purpose |
|---|---|
| `lessons` | Generalised lessons learned (one row = "don't attack hostiles at night without armor"). Pi extracts these from post-mortems and explicit incidents. Indexed by tag + situation key. Each lesson has `confidence` and `applied_count`. |
| `recipes` | Crafting recipes (seeded from `docs/minecraft-recipes.json`, augmented by wiki). |
| `mob_intel` | Per-mob behaviour: aggro range, drops, weakness, light requirement, "should we fight or flee" verdict. |
| `block_intel` | Per-block: required tool tier, drops, light emission, walkable. |
| `deaths` | One row per death event: ts, position, last skill, last hostile, hp/food at death, inventory loss summary, raw context blob. |
| `postmortems` | One row per Pi-analysed death event: links to `deaths`, contains structured Pi output (`cause`, `lesson`, `next_action_suggestion`). |
| `poi` | Points of interest: discovered ore vein, water, foreign build, danger zone. Spatial-grid indexed. Supersedes scatter in `world-journal.jsonl` for queryable retrieval; the journal stays as append-only event log. |
| `wiki_pages` | Cached `minecraft.wiki` pages with TTL. `etag`, `body`, `fetched_at`. |
| `chat_log` | Full inbound + outbound chat, structured (speaker, text, intent, replied_with). Supersedes the per-speaker LRU for long-tail player memory while keeping it as the fast in-process buffer. |
| `code_changes` | Audit of every auto-patch the bot applied to its own code: file, diff hash, proposal slug, success/rollback. |

SQLite chosen because: zero-config, perfect for single-process workloads,
embeddable, queryable by humans (`sqlite3 state/.../knowledge.db ".tables"`).
Migration: bundled `schema.sql` is applied idempotently on every boot;
schema version row gates additive changes.

**Graceful degradation:** if `better-sqlite3` isn't installed yet (fresh
`git pull` without `npm install`), `runtime/knowledge/store.js` logs one
warning and the knowledge subsystem becomes a no-op. The rest of the bot
keeps working.

### 2. Death post-mortem coach

`runtime/coach/postmortem.js`:

1. Hooks `bot.on('death')` (event already exists; the runtime currently
   only logs it).
2. Captures the death-context bundle: last 30s of log/scenarios, last
   dispatched skill, last hostile name + position, inventory delta (lost),
   plan.md current line, recent world-journal events nearby.
3. Inserts a row into `deaths`.
4. Enqueues a Pi job (rate-limited, max N per hour) that asks:
   > Look at this death event. In ≤120 words: what was the root cause,
   > what generalised lesson should I remember next time I see this
   > situation, and what skill should I dispatch differently?
5. Parses Pi's JSON reply into `postmortems` and `lessons`.

Lessons are tagged by situation: hostile name, biome, day/night, HP bucket,
inventory presence (weapon? armor?). At dispatch time, the curriculum
reflex calls `lessons.recall({skill, situationHash, hostile})` and adjusts
its plan if a high-confidence lesson is on point.

### 3. Wiki crawler (Phase 2 of v0.2.0)

`runtime/knowledge/wiki.js` — on-demand fetch of `minecraft.wiki/w/<Page>`
pages with the global fetch API. Parses recipe + mob + block info heuristically
(table-row scrape + targeted regex for hardness/light/drops). Respects
robots.txt and adds `User-Agent: pepa-pi-bot (https://github.com/...)`.
Pages cached in `wiki_pages` with 7-day TTL.

Used by `recipes`, `mob_intel`, `block_intel` when the local DB misses.
Triggered also by an explicit "go-learn" Pi suggestion ("look up phantom
behaviour before you sleep tonight").

### 4. Coach loop

`runtime/coach/index.js` — a periodic out-of-band coach pass that runs
every 30 minutes (configurable) and:

- Reads the last N deaths, last N skill failures, last N hours of the
  world-journal, the active plan.md.
- Asks Pi (one job, batched) to produce: lessons + plan adjustments +
  proposal hints.
- Writes lessons to the DB.
- If Pi recommends a plan change → patches plan.md (with a backup) and
  writes a diary entry explaining the change.
- If Pi recommends code surgery → writes a `state/<host>/proposals/<slug>.md`
  file in the existing self-improvement format, so the auto-patch pipeline
  picks it up.

This is the *proactive* counterpart to the reactive stuck-incident
proposal. The bot is no longer waiting to be 5×-broken; it's reflecting
every 30 minutes regardless.

### 5. Persona — narration & lifelike chat

`runtime/persona/chatter.js` — when the bot:

- starts a major skill (gather, build, travel ≥ 50 blocks)
- has a milestone success
- spots a notable threat ("4 креперов рядом")
- gets killed and respawns
- spots a player nearby

...it occasionally drops a single short Russian line in MC chat ("пошёл за
деревом", "ох, креперы, прячусь", "доброе утро"). Heavily rate-limited
(min 60s between lines, max 8/hour). Driven by simple templates + Pi
fallback for novelty.

`runtime/persona/look.js` (Phase 2 of v0.2.0) — `bot.look(yaw, pitch)`
during idle to simulate a living player: glance at sky, look at speakers,
look at target block before mining. Implemented as a low-priority
"animation" reflex that fires only when no real action is dispatched.

### 6. Retrieval-augmented dispatch

Before dispatching a skill, the curriculum reflex now queries:

```js
const advice = knowledge.recall({
  skillId,
  situation: snapshot.situationHash,
  hostile: snapshot.threats[0]?.name,
});
```

If advice contains a lesson with `confidence ≥ 0.6` and an explicit
`avoid: skillId` directive, the skill is backed off (and the lesson's
`applied_count` is incremented). If the lesson contains a `prefer:
alternative_skill_id`, that alternative is dispatched instead.

This is the loop that closes "I died yesterday in this exact spot
attacking a creeper with fists" → "tonight I avoid attack and dig
shelter instead".

## Phases — what shipped

> Live status, observed metrics, and rc.4 followups: [`dev/v0.2.0/STATUS.md`](../dev/v0.2.0/STATUS.md).

### v0.2.0-rc.1 — substrate (merged 2026-05-27)

- [x] WIP baseline commit (pathfinder/reflex/metrics fixes).
- [x] This design doc.
- [x] `better-sqlite3` dep + `runtime/knowledge/{store,schema,seed,index,lessons}.js`.
- [x] Seed knowledge DB from `docs/minecraft-recipes.json` + inline
      starter tables: 38 recipes, 15 mobs, 30 blocks, 12 starter lessons.
- [x] `runtime/coach/postmortem.js` — death event capture, deaths table,
      Pi-extracted lessons (5 min drain, ≤3 Pi calls/h, 12 min cooldown).
- [x] `runtime/persona/chatter.js` — narration on death/respawn, gather
      start, threat spotted, dusk/dawn, milestone done (templates).
- [x] Wire-in: `bot.js` calls `coach.attach`, `reflect.attach`,
      `persona.attach` once on connect.
- [x] Retrieval-augmented dispatch via `runtime/coach/advice.js` —
      `consult()` before curriculum + defend reflexes. SAFE_OVERRIDES
      whitelist guards what skills lessons may dispatch into.
- [x] Auto-patch loop switched to PR-on-merge instead of direct
      cherry-pick. Branch protection on `main`: 1 required approver.
- [x] Tests for store, lessons, postmortem context, advice
      consultation, persona cooldowns.

### v0.2.0-rc.2 — P0 hardening (merged 2026-05-27)

- [x] `PEPA_HEADLESS=1` guard in `extensions/mineflayer-bridge.ts` so
      `pi -p` subprocesses don't open a second MC connection.
- [x] Test state isolation — `runtime/config.js` redirects `stateDir`
      to `/tmp/pepa-test-state-<pid>/` under node test runner.
- [x] `defendReflex` outcome bug — `reportAdviceOutcome` now fires
      AFTER the flee skill returns, not before.
- [x] Mode-name → skill-id translation (`night_shelter` →
      `survive.sleep`, etc.) in `coach/advice.js`.
- [x] **Self-reflection loop** (`runtime/coach/reflect.js`) — every
      30 min Pi gets asked *"are you in a loop?"*. Verdict + summary
      + new lessons. Reflection written to
      `state/<host>/reflections/<ts>.md`.

### v0.2.0-rc.3 — escape mechanics (merged 2026-05-27)

- [x] `survive.pillar-up` — vertical pit escape, NO pickaxe required.
- [x] Wedged-emergency reflex — auto-fires pillar-up after 60s of no
      horizontal progress + no hostile + placeable block.
- [x] `consult()` runs on curriculum's wander/explore.far fallback
      path (closes the gap where Pi-coach lessons fired but were
      ignored by the dispatcher's fallback).
- [x] `recordPOI(kind:"danger")` on every death — spatial memory.
- [x] `SAFE_OVERRIDES` extended with `survive.pillar-up`,
      `village.choose-base`.

### rc.4 candidates — see [`dev/v0.2.0/STATUS.md §"Known issues"`](../dev/v0.2.0/STATUS.md)

1. Fix hallucinated `prefer_skill` from Pi (e.g. `choose.safe.surface`).
2. Tool-progression auto-craft: `gather.stone → missing_tool →
   craft.wooden-pickaxe`.
3. `village.choose-base` ranking should penalise nearby danger POI.
4. Pillar-up perimeter sense — abort if ceiling above.
5. Persist persona cooldowns across restarts.
6. Populate `chat_log` table on every chat in/out.

### Wiki crawler — pushed to v0.2.1

`runtime/knowledge/wiki.js` still planned but isn't on the critical
path while the seeded intel + Pi-extracted lessons are sufficient.
Will land when the bot has stabilised enough that "learning new things
from the wiki" beats "applying things it already learned".

## Out of scope for v0.2.0

- Cross-server identity (still per-host).
- Telegram bridge (Phase 4 in roadmap).
- Voice/avatar mimicry beyond chat narration.
- PvP/grief tooling.

## File layout (new)

```
plans/
  v0.2.0-self-learning.md         (this file)

runtime/knowledge/
  store.js                        better-sqlite3 gateway, schema bootstrap
  schema.sql                      DDL (versioned)
  seed.js                         load docs/minecraft-recipes.json + mob_intel rows
  index.js                        public API: recall(), record(), lookupRecipe(), …
  lessons.js                      lesson tag-matching, scoring, retrieval
  wiki.js                         (rc.2) on-demand wiki fetcher

runtime/coach/
  postmortem.js                   on death → capture → DB → enqueue Pi
  index.js                        (rc.2) periodic batched coach pass

runtime/persona/
  chatter.js                      proactive narration
  look.js                         (rc.2) head movement
```

## Compatibility / safety

- All new modules: side-effect-free import (no top-level network, no DB
  open). Attach via explicit `.attach(bot)` calls in `bot.js`.
- DB writes are best-effort (try/catch). A broken DB never crashes the
  reflex loop.
- Pi calls from the coach respect the existing 6-per-hour banter limit
  via a shared rate-limiter (`runtime/pi-budget.js` — new tiny module).
- Chat narration respects existing `cmd:chat` rate limits + dialog-only
  policy: it's `bot.chat()` calls, no command verbs.
- All new tables in `state/<host>/knowledge.db` are gitignored via the
  existing `state/` rule.