dev/v0.3.1/PRD.md

@@ -0,0 +1,192 @@
+# pepa v0.3.1 — PRD: LLM prompt cost optimization
+
+**Status**: Design draft. No code in this version yet — this PRD is the
+spec future commits implement against. Owner: operator.
+**Trigger**: TimeWeb admin panel after first day of v0.3.0 live:
+~34K tokens used in a half-day session (mostly bot + some smoke).
+At the 6-calls/hour cap that projects to **~480 ₽/month** (101 ₽/M in,
+608 ₽/M out for gpt-5.4-mini). Manageable but worth shrinking — most
+of the per-call cost is repeated infrastructure tokens, not the
+situational signal the model actually uses.
+
+## Goals
+
+1. Cut per-advise() input tokens from ~800 → ≤300 (target 250).
+2. Preserve correctness: the LLM must still see enough context to
+   produce a valid `skill_id` from the registry and a useful rationale.
+3. Keep all changes transparent to the rest of the runtime — the
+   public `advise()` / `complete()` surface area doesn't change.
+
+Non-goals:
+- Switching providers. TimeWeb stays.
+- Caching the LLM's *responses* (cache key would be situational, too
+  many misses to be worth the bookkeeping).
+- Touching the analytical loops (postmortem / reflect). They're called
+  less often and need fuller context; cost there is acceptable.
+
+## Cost breakdown — what we're optimizing
+
+Measured on live advise() calls (TimeWeb gpt-5.4-mini, single advisor
+trigger):
+
+| Block                                | tokens (avg) | % of call |
+|--------------------------------------|--------------|-----------|
+| `skillRegistryPrompt({limit:1800})`  | ~450         | 56%       |
+| System instructions (rules + JSON)   | ~200         | 25%       |
+| User snapshot + threats + need + recent | ~150     | 19%       |
+| **Total input**                      | **~800**     | **100%**  |
+| Output (JSON answer)                 | ~40-50       | —         |
+
+The registry block dominates. It currently lists all 30+ registered
+skills with their human titles. The model rarely needs the full list —
+most decisions are within 5-8 plausible skills per trigger.
+
+## Proposed changes
+
+### 1. Compact registry format (P1, biggest win)
+
+Drop human titles and the per-skill descriptions; switch to
+namespace-grouped, comma-separated id lists.
+
+**Before** (~450 tokens):
+```
+Valid skill ids (USE ONLY THESE for avoid_skill / prefer_skill):
+  craft:
+    - craft.bed — Craft bed
+    - craft.chest — Craft chest
+    - craft.furnace — Craft furnace
+    ...
+  survive:
+    - survive.acquire-food — Acquire food
+    - survive.eat — Eat
+    ...
+```
+
+**After** (~100 tokens):
+```
+Valid skill ids (USE EXACTLY one of these or null):
+  craft: bed, chest, furnace, planks, sticks, torch, wooden-axe,
+         wooden-pickaxe, wooden-sword, stone-axe, stone-pickaxe, stone-sword
+  survive: acquire-food, eat, flee, pillar-up, sleep
+  gather: logs, stone, wool
+  recovery: tunnel-out
+  explore: far, wander
+  village: build-shelter, choose-base, deposit-surplus, place-chest
+  farm: wheat
+  diag: physics, scan, match
+```
+
+Saving: **~350 tokens/call**.
+
+Implementation: add `skillRegistryPrompt({ mode: "compact" })` mode in
+`runtime/skill-registry.js`. Default mode stays for slow analytical
+loops (postmortem / reflect) which can afford the verbose form.
+
+### 2. Need-scoped registry (P2, additional ~50 token saving)
+
+When `activeNeed` is set, filter the registry to skills plausibly
+relevant to that level + always-available safety skills.
+
+Relevance table (manually curated, lives in `runtime/manifesto/needs.js`):
+
+| Need              | Relevant skills (in addition to ALWAYS set)               |
+|-------------------|------------------------------------------------------------|
+| alive             | survive.flee, survive.eat, recovery.tunnel-out             |
+| food              | survive.acquire-food, survive.eat, farm.wheat              |
+| tools_wood        | gather.logs, craft.planks, craft.sticks, craft.wooden-*    |
+| shelter_basic     | gather.wool, craft.bed, village.build-shelter, village.choose-base |
+| tools_stone       | gather.stone, craft.sticks, craft.stone-*                  |
+| armor_basic       | gather.wool (placeholder)                                  |
+| food_security     | farm.wheat, survive.acquire-food                           |
+| tools_iron        | gather.stone                                               |
+| armor_iron        | (none — no skill yet)                                      |
+| village_seed      | craft.chest, village.deposit-surplus, village.build-shelter |
+| village_full      | (full registry)                                            |
+| ALWAYS            | survive.flee, survive.pillar-up, recovery.tunnel-out,      |
+|                   | explore.far, explore.wander                                |
+
+Compact + scoped = **~50 tokens** for the registry block (down from 450).
+
+Add a `prompt-builder.test.js` checking that:
+- `survive.flee` is always present (emergency safety)
+- The recommended skill from the previous call would still be in the
+  scoped registry (regression protection)
+
+### 3. Snapshot pruning (P3, ~50 tokens)
+
+The user-prompt snapshot includes fields the LLM rarely consults:
+`weather`, `experience`, `dimension`, `biome`, `players[]`. Drop them
+from the advise() user-prompt builder. Keep `position`, `hp`, `food`,
+`isDay`, `closestHostile`, `activeNeed`, `recent dispatches`,
+`hazards.footBlock` (lava detection), top inventory keys.
+
+### 4. Prompt caching — investigation (P4)
+
+OpenAI and Anthropic both support implicit prompt caching: when ≥1024
+prefix tokens are identical across consecutive requests, the prefix
+is billed once. TimeWeb's docs are silent on this.
+
+Task: probe whether TimeWeb passes through OpenAI's `prompt_tokens_details.cached_tokens`
+field. If yes, *increase* the system prefix length (keep verbose registry)
+because cached input is ~10x cheaper than fresh. If no, full optimization
+1+2+3 still wins.
+
+Add a one-off check in `scripts/check-timeweb.js`: print
+`payload?.usage?.prompt_tokens_details?.cached_tokens` if present.
+
+### 5. Telemetry — per-trigger token attribution (P5)
+
+Today `advisor_recommendations` records `tokens_in` per row but the
+operator has no easy view of *which trigger types* are most expensive.
+
+Extend `scripts/list-improvements.js --stats` to also print per-trigger:
+```
+trigger_reason         total  applied  ok  fail  avg_in  avg_out  cost_₽  share%
+wedged_*                  20       18   3   15    280    45        2.1     45%
+emergency_*                3        3   2    1    240    50        0.3      6%
+repeat_*                   8        7   0    7    260    42        0.8     17%
+preempt_retry_*           14       12   2   10    290    44        1.5     32%
+```
+
+`cost_₽` = avg_in × calls × IN_PRICE + avg_out × calls × OUT_PRICE,
+with prices read from env (`TIMEWEB_PRICE_IN_RUB_PER_M`,
+`TIMEWEB_PRICE_OUT_RUB_PER_M`).
+
+## Acceptance
+
+After v0.3.1 lands:
+- Re-run `node scripts/check-timeweb.js` probe 3 (`advise()`):
+  expect `tokens_in` ≤ 300 (was ~800).
+- Re-run probe 4 (auto-trigger flow): rationale still references the
+  registered skill correctly.
+- Run live for 1 hour, check `node scripts/list-improvements.js --stats`:
+  per-trigger `avg_in` ≤ 300.
+- Existing 360 tests still green; new prompt-builder tests cover the
+  scoped registry behaviour.
+
+## Out of scope (later versions)
+
+- Tool/function-calling instead of free-form JSON (TimeWeb support unclear).
+- Custom model selection per trigger (gpt-5.4-nano for routine repeats,
+  gpt-5.4-mini for emergencies). Defer until cost/quality data points
+  exist.
+- Embedding-based prior-recommendation similarity check ("we already
+  told the bot to tunnel-out at this exact wedge 10 minutes ago, skip").
+
+## Implementation order
+
+When this version is greenlit, work in this order on a single branch
+`v0.3.1` (one PR per session, per recently-updated workflow memory):
+
+1. P1 compact registry mode + prompt-builder test
+2. P2 need-scoped registry (extend `runtime/manifesto/needs.js` with
+   `relevantSkills`)
+3. P3 snapshot pruning in `fast-advisor.js#buildUserPrompt`
+4. P4 caching probe (one-off)
+5. P5 cost telemetry in CLI viewer
+6. STATUS.md + smoke retest + PR
+
+All changes are additive; no behaviour regression expected. If
+real-world after v0.3.1 shows the LLM giving worse advice with the
+compact registry, fall back to default mode by flipping a single
+constant in `fast-advisor.js`.