pepa-pi-bot/docs/v0.2.0-self-learning.md

pepa v0.2.0 — self-learning agent

Status: active (started 2026-05-27). Carries forward the WIP that landed in the baseline commit of this branch (pathfinder watchdog, reflex metric-recovery, persistent skill metrics, new skills like flee, acquire-food, place-chest, sleep). Local-only repo — this file ships in the working tree but is not pushed.

Why v0.2.0

The v0.1.x bot survives in the abstract — reflex chain, curriculum, modes, auto-improve — but on the live server it spent ~15 hours overnight in a tight death spiral (≈300 deaths, never moved off spawn, no tools, no progress). Three failure modes dominate:

1. No persistent knowledge. Each tick reasons from scratch. The bot has no structured memory of "what I tried", "what worked", "what the recipe for X is", "what kills this mob". scenarios.jsonl is a forgetful sliding window keyed by a coarse situation hash, not a queryable store.
2. No coaching from failure. A death is just a respawn event. Pi is never asked "why did I die, what should I do differently next time". Lessons aren't extracted and aren't recalled before the next attempt.
3. No outside knowledge. The bot doesn't read the wiki, doesn't know that creepers need 4-block kite distance, that fists vs hostile at night is suicide, that bread is the cheapest renewable food.

v0.2.0 fixes the substrate so future iterations get steeper progress per hour-of-runtime. The bot becomes Voyager-shaped: structured skill library + persistent learned-skill memory + post-mortem-driven self-improvement + retrieval-augmented decision making + free chat persona.

Pillars

1. Structured knowledge base (SQLite)

A single per-server state/<host>/knowledge.db (gitignored) with these tables:

Table	Purpose
lessons	Generalised lessons learned (one row = "don't attack hostiles at night without armor"). Pi extracts these from post-mortems and explicit incidents. Indexed by tag + situation key. Each lesson has confidence and applied_count.
recipes	Crafting recipes (seeded from docs/minecraft-recipes.json, augmented by wiki).
mob_intel	Per-mob behaviour: aggro range, drops, weakness, light requirement, "should we fight or flee" verdict.
block_intel	Per-block: required tool tier, drops, light emission, walkable.
deaths	One row per death event: ts, position, last skill, last hostile, hp/food at death, inventory loss summary, raw context blob.
postmortems	One row per Pi-analysed death event: links to deaths, contains structured Pi output (cause, lesson, next_action_suggestion).
poi	Points of interest: discovered ore vein, water, foreign build, danger zone. Spatial-grid indexed. Supersedes scatter in world-journal.jsonl for queryable retrieval; the journal stays as append-only event log.
wiki_pages	Cached minecraft.wiki pages with TTL. etag, body, fetched_at.
chat_log	Full inbound + outbound chat, structured (speaker, text, intent, replied_with). Supersedes the per-speaker LRU for long-tail player memory while keeping it as the fast in-process buffer.
code_changes	Audit of every auto-patch the bot applied to its own code: file, diff hash, proposal slug, success/rollback.

SQLite chosen because: zero-config, perfect for single-process workloads, embeddable, queryable by humans (sqlite3 state/.../knowledge.db ".tables"). Migration: bundled schema.sql is applied idempotently on every boot; schema version row gates additive changes.

Graceful degradation: if better-sqlite3 isn't installed yet (fresh git pull without npm install), runtime/knowledge/store.js logs one warning and the knowledge subsystem becomes a no-op. The rest of the bot keeps working.

2. Death post-mortem coach

runtime/coach/postmortem.js:

1. Hooks bot.on('death') (event already exists; the runtime currently only logs it).
2. Captures the death-context bundle: last 30s of log/scenarios, last dispatched skill, last hostile name + position, inventory delta (lost), plan.md current line, recent world-journal events nearby.
3. Inserts a row into deaths.
4. Enqueues a Pi job (rate-limited, max N per hour) that asks:

   Look at this death event. In ≤120 words: what was the root cause, what generalised lesson should I remember next time I see this situation, and what skill should I dispatch differently?

5. Parses Pi's JSON reply into postmortems and lessons.

Lessons are tagged by situation: hostile name, biome, day/night, HP bucket, inventory presence (weapon? armor?). At dispatch time, the curriculum reflex calls lessons.recall({skill, situationHash, hostile}) and adjusts its plan if a high-confidence lesson is on point.

3. Wiki crawler (Phase 2 of v0.2.0)

runtime/knowledge/wiki.js — on-demand fetch of minecraft.wiki/w/<Page> pages with the global fetch API. Parses recipe + mob + block info heuristically (table-row scrape + targeted regex for hardness/light/drops). Respects robots.txt and adds User-Agent: pepa-pi-bot (https://github.com/...). Pages cached in wiki_pages with 7-day TTL.

Used by recipes, mob_intel, block_intel when the local DB misses. Triggered also by an explicit "go-learn" Pi suggestion ("look up phantom behaviour before you sleep tonight").

4. Coach loop

runtime/coach/index.js — a periodic out-of-band coach pass that runs every 30 minutes (configurable) and:

• Reads the last N deaths, last N skill failures, last N hours of the world-journal, the active plan.md.
• Asks Pi (one job, batched) to produce: lessons + plan adjustments + proposal hints.
• Writes lessons to the DB.
• If Pi recommends a plan change → patches plan.md (with a backup) and writes a diary entry explaining the change.
• If Pi recommends code surgery → writes a state/<host>/proposals/<slug>.md file in the existing self-improvement format, so the auto-patch pipeline picks it up.

This is the proactive counterpart to the reactive stuck-incident proposal. The bot is no longer waiting to be 5×-broken; it's reflecting every 30 minutes regardless.

5. Persona — narration & lifelike chat

runtime/persona/chatter.js — when the bot:

• starts a major skill (gather, build, travel ≥ 50 blocks)
• has a milestone success
• spots a notable threat ("4 креперов рядом")
• gets killed and respawns
• spots a player nearby

...it occasionally drops a single short Russian line in MC chat ("пошёл за деревом", "ох, креперы, прячусь", "доброе утро"). Heavily rate-limited (min 60s between lines, max 8/hour). Driven by simple templates + Pi fallback for novelty.

runtime/persona/look.js (Phase 2 of v0.2.0) — bot.look(yaw, pitch) during idle to simulate a living player: glance at sky, look at speakers, look at target block before mining. Implemented as a low-priority "animation" reflex that fires only when no real action is dispatched.

6. Retrieval-augmented dispatch

Before dispatching a skill, the curriculum reflex now queries:

const advice = knowledge.recall({
  skillId,
  situation: snapshot.situationHash,
  hostile: snapshot.threats[0]?.name,
});

If advice contains a lesson with confidence ≥ 0.6 and an explicit avoid: skillId directive, the skill is backed off (and the lesson's applied_count is incremented). If the lesson contains a prefer: alternative_skill_id, that alternative is dispatched instead.

This is the loop that closes "I died yesterday in this exact spot attacking a creeper with fists" → "tonight I avoid attack and dig shelter instead".

Phases — what shipped

Live status, observed metrics, and rc.4 followups: dev/v0.2.0/STATUS.md.

v0.2.0-rc.1 — substrate (merged 2026-05-27)

• WIP baseline commit (pathfinder/reflex/metrics fixes).
• This design doc.
• better-sqlite3 dep + runtime/knowledge/{store,schema,seed,index,lessons}.js.
• Seed knowledge DB from docs/minecraft-recipes.json + inline starter tables: 38 recipes, 15 mobs, 30 blocks, 12 starter lessons.
• runtime/coach/postmortem.js — death event capture, deaths table, Pi-extracted lessons (5 min drain, ≤3 Pi calls/h, 12 min cooldown).
• runtime/persona/chatter.js — narration on death/respawn, gather start, threat spotted, dusk/dawn, milestone done (templates).
• Wire-in: bot.js calls coach.attach, reflect.attach, persona.attach once on connect.
• Retrieval-augmented dispatch via runtime/coach/advice.js — consult() before curriculum + defend reflexes. SAFE_OVERRIDES whitelist guards what skills lessons may dispatch into.
• Auto-patch loop switched to PR-on-merge instead of direct cherry-pick. Branch protection on main: 1 required approver.
• Tests for store, lessons, postmortem context, advice consultation, persona cooldowns.

v0.2.0-rc.2 — P0 hardening (merged 2026-05-27)

• PEPA_HEADLESS=1 guard in extensions/mineflayer-bridge.ts so pi -p subprocesses don't open a second MC connection.
• Test state isolation — runtime/config.js redirects stateDir to /tmp/pepa-test-state-<pid>/ under node test runner.
• defendReflex outcome bug — reportAdviceOutcome now fires AFTER the flee skill returns, not before.
• Mode-name → skill-id translation (night_shelter → survive.sleep, etc.) in coach/advice.js.
• Self-reflection loop (runtime/coach/reflect.js) — every 30 min Pi gets asked "are you in a loop?". Verdict + summary + new lessons. Reflection written to state/<host>/reflections/<ts>.md.

v0.2.0-rc.3 — escape mechanics (merged 2026-05-27)

• survive.pillar-up — vertical pit escape, NO pickaxe required.
• Wedged-emergency reflex — auto-fires pillar-up after 60s of no horizontal progress + no hostile + placeable block.
• consult() runs on curriculum's wander/explore.far fallback path (closes the gap where Pi-coach lessons fired but were ignored by the dispatcher's fallback).
• recordPOI(kind:"danger") on every death — spatial memory.
• SAFE_OVERRIDES extended with survive.pillar-up, village.choose-base.

rc.4 candidates — see dev/v0.2.0/STATUS.md §"Known issues"

1. Fix hallucinated prefer_skill from Pi (e.g. choose.safe.surface).
2. Tool-progression auto-craft: gather.stone → missing_tool → craft.wooden-pickaxe.
3. village.choose-base ranking should penalise nearby danger POI.
4. Pillar-up perimeter sense — abort if ceiling above.
5. Persist persona cooldowns across restarts.
6. Populate chat_log table on every chat in/out.

Wiki crawler — pushed to v0.2.1

runtime/knowledge/wiki.js still planned but isn't on the critical path while the seeded intel + Pi-extracted lessons are sufficient. Will land when the bot has stabilised enough that "learning new things from the wiki" beats "applying things it already learned".

Out of scope for v0.2.0

• Cross-server identity (still per-host).
• Telegram bridge (Phase 4 in roadmap).
• Voice/avatar mimicry beyond chat narration.
• PvP/grief tooling.

File layout (new)

plans/
  v0.2.0-self-learning.md         (this file)

runtime/knowledge/
  store.js                        better-sqlite3 gateway, schema bootstrap
  schema.sql                      DDL (versioned)
  seed.js                         load docs/minecraft-recipes.json + mob_intel rows
  index.js                        public API: recall(), record(), lookupRecipe(), …
  lessons.js                      lesson tag-matching, scoring, retrieval
  wiki.js                         (rc.2) on-demand wiki fetcher

runtime/coach/
  postmortem.js                   on death → capture → DB → enqueue Pi
  index.js                        (rc.2) periodic batched coach pass

runtime/persona/
  chatter.js                      proactive narration
  look.js                         (rc.2) head movement

Compatibility / safety

• All new modules: side-effect-free import (no top-level network, no DB open). Attach via explicit .attach(bot) calls in bot.js.
• DB writes are best-effort (try/catch). A broken DB never crashes the reflex loop.
• Pi calls from the coach respect the existing 6-per-hour banter limit via a shared rate-limiter (runtime/pi-budget.js — new tiny module).
• Chat narration respects existing cmd:chat rate limits + dialog-only policy: it's bot.chat() calls, no command verbs.
• All new tables in state/<host>/knowledge.db are gitignored via the existing state/ rule.