pepa-pi-bot/docs/v0.2.0-self-learning.md

# pepa v0.2.0 — self-learning agent

> Status: **active** (started 2026-05-27). Carries forward the WIP that landed
> in the baseline commit of this branch (pathfinder watchdog, reflex
> metric-recovery, persistent skill metrics, new skills like `flee`,
> `acquire-food`, `place-chest`, `sleep`).
> Local-only repo — this file ships in the working tree but is not pushed.

## Why v0.2.0

The v0.1.x bot survives in the abstract — reflex chain, curriculum, modes,
auto-improve — but on the live server it spent ~15 hours overnight in a tight
death spiral (≈300 deaths, never moved off spawn, no tools, no progress).
Three failure modes dominate:

1. **No persistent knowledge.** Each tick reasons from scratch. The bot has
   no structured memory of "what I tried", "what worked", "what the recipe
   for X is", "what kills this mob". `scenarios.jsonl` is a forgetful
   sliding window keyed by a coarse situation hash, not a queryable store.
2. **No coaching from failure.** A death is just a respawn event. Pi is
   never asked "why did I die, what should I do differently next time".
   Lessons aren't extracted and aren't recalled before the next attempt.
3. **No outside knowledge.** The bot doesn't read the wiki, doesn't know
   that creepers need 4-block kite distance, that fists vs hostile at
   night is suicide, that bread is the cheapest renewable food.

v0.2.0 fixes the substrate so future iterations get steeper progress per
hour-of-runtime. The bot becomes Voyager-shaped: structured skill library +
persistent learned-skill memory + post-mortem-driven self-improvement +
retrieval-augmented decision making + free chat persona.

## Pillars

### 1. Structured knowledge base (SQLite)

A single per-server `state/<host>/knowledge.db` (gitignored) with these
tables:

| Table | Purpose |
|---|---|
| `lessons` | Generalised lessons learned (one row = "don't attack hostiles at night without armor"). Pi extracts these from post-mortems and explicit incidents. Indexed by tag + situation key. Each lesson has `confidence` and `applied_count`. |
| `recipes` | Crafting recipes (seeded from `docs/minecraft-recipes.json`, augmented by wiki). |
| `mob_intel` | Per-mob behaviour: aggro range, drops, weakness, light requirement, "should we fight or flee" verdict. |
| `block_intel` | Per-block: required tool tier, drops, light emission, walkable. |
| `deaths` | One row per death event: ts, position, last skill, last hostile, hp/food at death, inventory loss summary, raw context blob. |
| `postmortems` | One row per Pi-analysed death event: links to `deaths`, contains structured Pi output (`cause`, `lesson`, `next_action_suggestion`). |
| `poi` | Points of interest: discovered ore vein, water, foreign build, danger zone. Spatial-grid indexed. Supersedes scatter in `world-journal.jsonl` for queryable retrieval; the journal stays as append-only event log. |
| `wiki_pages` | Cached `minecraft.wiki` pages with TTL. `etag`, `body`, `fetched_at`. |
| `chat_log` | Full inbound + outbound chat, structured (speaker, text, intent, replied_with). Supersedes the per-speaker LRU for long-tail player memory while keeping it as the fast in-process buffer. |
| `code_changes` | Audit of every auto-patch the bot applied to its own code: file, diff hash, proposal slug, success/rollback. |

SQLite chosen because: zero-config, perfect for single-process workloads,
embeddable, queryable by humans (`sqlite3 state/.../knowledge.db ".tables"`).
Migration: bundled `schema.sql` is applied idempotently on every boot;
schema version row gates additive changes.

**Graceful degradation:** if `better-sqlite3` isn't installed yet (fresh
`git pull` without `npm install`), `runtime/knowledge/store.js` logs one
warning and the knowledge subsystem becomes a no-op. The rest of the bot
keeps working.

### 2. Death post-mortem coach

`runtime/coach/postmortem.js`:

1. Hooks `bot.on('death')` (event already exists; the runtime currently
   only logs it).
2. Captures the death-context bundle: last 30s of log/scenarios, last
   dispatched skill, last hostile name + position, inventory delta (lost),
   plan.md current line, recent world-journal events nearby.
3. Inserts a row into `deaths`.
4. Enqueues a Pi job (rate-limited, max N per hour) that asks:
   > Look at this death event. In ≤120 words: what was the root cause,
   > what generalised lesson should I remember next time I see this
   > situation, and what skill should I dispatch differently?
5. Parses Pi's JSON reply into `postmortems` and `lessons`.

Lessons are tagged by situation: hostile name, biome, day/night, HP bucket,
inventory presence (weapon? armor?). At dispatch time, the curriculum
reflex calls `lessons.recall({skill, situationHash, hostile})` and adjusts
its plan if a high-confidence lesson is on point.

### 3. Wiki crawler (Phase 2 of v0.2.0)

`runtime/knowledge/wiki.js` — on-demand fetch of `minecraft.wiki/w/<Page>`
pages with the global fetch API. Parses recipe + mob + block info heuristically
(table-row scrape + targeted regex for hardness/light/drops). Respects
robots.txt and adds `User-Agent: pepa-pi-bot (https://github.com/...)`.
Pages cached in `wiki_pages` with 7-day TTL.

Used by `recipes`, `mob_intel`, `block_intel` when the local DB misses.
Triggered also by an explicit "go-learn" Pi suggestion ("look up phantom
behaviour before you sleep tonight").

### 4. Coach loop

`runtime/coach/index.js` — a periodic out-of-band coach pass that runs
every 30 minutes (configurable) and:

- Reads the last N deaths, last N skill failures, last N hours of the
  world-journal, the active plan.md.
- Asks Pi (one job, batched) to produce: lessons + plan adjustments +
  proposal hints.
- Writes lessons to the DB.
- If Pi recommends a plan change → patches plan.md (with a backup) and
  writes a diary entry explaining the change.
- If Pi recommends code surgery → writes a `state/<host>/proposals/<slug>.md`
  file in the existing self-improvement format, so the auto-patch pipeline
  picks it up.

This is the *proactive* counterpart to the reactive stuck-incident
proposal. The bot is no longer waiting to be 5×-broken; it's reflecting
every 30 minutes regardless.

### 5. Persona — narration & lifelike chat

`runtime/persona/chatter.js` — when the bot:

- starts a major skill (gather, build, travel ≥ 50 blocks)
- has a milestone success
- spots a notable threat ("4 креперов рядом")
- gets killed and respawns
- spots a player nearby

...it occasionally drops a single short Russian line in MC chat ("пошёл за
деревом", "ох, креперы, прячусь", "доброе утро"). Heavily rate-limited
(min 60s between lines, max 8/hour). Driven by simple templates + Pi
fallback for novelty.

`runtime/persona/look.js` (Phase 2 of v0.2.0) — `bot.look(yaw, pitch)`
during idle to simulate a living player: glance at sky, look at speakers,
look at target block before mining. Implemented as a low-priority
"animation" reflex that fires only when no real action is dispatched.

### 6. Retrieval-augmented dispatch

Before dispatching a skill, the curriculum reflex now queries:

```js
const advice = knowledge.recall({
  skillId,
  situation: snapshot.situationHash,
  hostile: snapshot.threats[0]?.name,
});
```

If advice contains a lesson with `confidence ≥ 0.6` and an explicit
`avoid: skillId` directive, the skill is backed off (and the lesson's
`applied_count` is incremented). If the lesson contains a `prefer:
alternative_skill_id`, that alternative is dispatched instead.

This is the loop that closes "I died yesterday in this exact spot
attacking a creeper with fists" → "tonight I avoid attack and dig
shelter instead".

## Phases — what ships in which rc

### v0.2.0-rc.1 (this branch)

- [x] WIP baseline commit (pathfinder/reflex/metrics fixes).
- [ ] This design doc.
- [ ] `better-sqlite3` dep + `runtime/knowledge/{store,schema,seed,index,lessons}.js`.
- [ ] Seed knowledge DB from `docs/minecraft-recipes.json` + the
      mob/biome/food/recipe tables in `docs/minecraft-knowledge.md`.
- [ ] `runtime/coach/postmortem.js` — death event capture, deaths table,
      Pi-extracted lessons (initial version uses async one-shot, no batch).
- [ ] `runtime/persona/chatter.js` — narration on death/respawn,
      gather-start, threat-spotted (templates only, no Pi yet).
- [ ] Wire-in: `bot.js` calls `coach.attach(bot)` and
      `persona.attach(bot)` once on connect.
- [ ] Tests for store, lessons, postmortem context capture, persona
      cooldowns.

### v0.2.0-rc.2

- [ ] Wiki crawler with cache + parser.
- [ ] Retrieval-augmented dispatch (curriculum + dispatch hooks).
- [ ] `runtime/persona/look.js` — head movement.
- [ ] `runtime/coach/index.js` — periodic batched Pi pass.

### v0.2.0 (final)

- [ ] Tune rate limits + thresholds with real data from the prior weeks.
- [ ] Cross-server lesson abstraction (raw → committed `skills/<name>.md`
      when applied repeatedly across situations).
- [ ] Roadmap update.

## Out of scope for v0.2.0

- Cross-server identity (still per-host).
- Telegram bridge (Phase 4 in roadmap).
- Voice/avatar mimicry beyond chat narration.
- PvP/grief tooling.

## File layout (new)

```
plans/
  v0.2.0-self-learning.md         (this file)

runtime/knowledge/
  store.js                        better-sqlite3 gateway, schema bootstrap
  schema.sql                      DDL (versioned)
  seed.js                         load docs/minecraft-recipes.json + mob_intel rows
  index.js                        public API: recall(), record(), lookupRecipe(), …
  lessons.js                      lesson tag-matching, scoring, retrieval
  wiki.js                         (rc.2) on-demand wiki fetcher

runtime/coach/
  postmortem.js                   on death → capture → DB → enqueue Pi
  index.js                        (rc.2) periodic batched coach pass

runtime/persona/
  chatter.js                      proactive narration
  look.js                         (rc.2) head movement
```

## Compatibility / safety

- All new modules: side-effect-free import (no top-level network, no DB
  open). Attach via explicit `.attach(bot)` calls in `bot.js`.
- DB writes are best-effort (try/catch). A broken DB never crashes the
  reflex loop.
- Pi calls from the coach respect the existing 6-per-hour banter limit
  via a shared rate-limiter (`runtime/pi-budget.js` — new tiny module).
- Chat narration respects existing `cmd:chat` rate limits + dialog-only
  policy: it's `bot.chat()` calls, no command verbs.
- All new tables in `state/<host>/knowledge.db` are gitignored via the
  existing `state/` rule.