Commit Graph
1 Commits
Author SHA1 Message Date
mayatnikovandClaude Opus 4.7 451c0343cb docs(v0.3.1): PRD — LLM prompt cost optimization
Design-only commit; no runtime changes. Spec for the next patch iteration.

Goal: cut per-advise() input tokens from ~800 to ≤300, preserving the
LLM's ability to produce valid registered skill ids and useful rationale.

Five proposed changes ranked by impact:

  P1  Compact registry format (saves ~350t/call) — group by namespace,
      comma-list ids, drop human titles. Default mode for advisor;
      verbose mode kept for postmortem/reflect.
  P2  Need-scoped registry (~50t additional) — show LLM only skills
      relevant to the active Maslow need + always-available safety
      skills (survive.flee, pillar-up, recovery.tunnel-out, explore.*).
  P3  Snapshot pruning (~50t) — drop weather/experience/dimension/biome/
      players from the user prompt; the LLM doesn't consult them.
  P4  Prompt caching probe — check if TimeWeb passes through
      prompt_tokens_details.cached_tokens. If yes, restructure prefix
      to maximize cache hits (cached input is ~10x cheaper at OpenAI).
  P5  Per-trigger cost telemetry in scripts/list-improvements.js --stats:
      avg_in / avg_out / cost_₽ / share% per trigger_reason, using
      TIMEWEB_PRICE_IN_RUB_PER_M and TIMEWEB_PRICE_OUT_RUB_PER_M env.

Trigger: TimeWeb admin panel after first day of v0.3.0 live showed
34K tokens / day at low activity. At cap budget that projects to
~480₽/month (101₽/M in, 608₽/M out for gpt-5.4-mini). Manageable
but the savings are mostly free — repeated infra tokens, not signal.

All changes are additive; runtime behaviour stays the same. If the
LLM produces worse advice with the compact registry, flip back via a
single constant in fast-advisor.js.

Acceptance: re-run scripts/check-timeweb.js probe 3 — expect
tokens_in ≤ 300 (was ~800). Live for 1h, check --stats: avg_in ≤ 300
per trigger group. Existing 360 tests still green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:52:05 +03:00