feat(perception): mindcraft skills/world + 15 Pi tools (proper perceive→decide→act loop) #2

2026-05-25T14:24:51+03:00

halofourteen commented

2026-05-25 14:24:51 +03:00

(Migrated from github.com)

Why

The bot was acting blind: only its own coords, no view of surroundings. mc_goto's over-strict safety guards refused every real path. mc_dig was too low-level to drive a farming loop. The result: 4+ hours of "trying" with zero physical achievement (бот всё это время сидел на дереве Y≈88, не сделал ни шага по миру).

What

Reframes the bridge around a perceive→decide→act loop using proven primitives from Mindcraft (MIT — vendored with attribution).

Files added

extensions/lib/{world.js,skills.js,mcdata.js,settings.js,LICENSE-MINDCRAFT} — vendored Mindcraft library
- world.js: 21 perception functions
- skills.js: 30+ action primitives
- mcdata.js: adapter, with attachPluginsAndInit(bot) export so our externally-created bot gets the plugins loaded
extensions/mindcraft-skills.ts — new Pi extension with 15 high-level tools:
- Perception: mc_observe, mc_inventory, mc_nearby_blocks, mc_nearby_entities
- Action: mc_collect_block, mc_place_block, mc_go_to, mc_go_to_block, mc_craft, mc_equip, mc_consume, mc_defend_self, mc_avoid_enemies, mc_stay, mc_pickup_nearby

Files changed

extensions/mineflayer-bridge.ts — expose globalThis.__pepaPiBot so the new extension can access the bot; call attachPluginsAndInit() after createBot to wire pathfinder/pvp/collectblock/armorManager.
AGENTS.md — new "Perception → decision → action" section with full tool catalog and deprecation note for mc_goto / mc_build_pyramid_5x5 / mc_dig.
package.json — new deps: minecraft-data, vec3, mineflayer-pvp, prismarine-item.

Smoke test

Launched Pi locally on medium thinking. Bot called mc_observe and got a real JSON snapshot:

```json
{
"health": 20, "food": 20, "saturation": 5,
"nearbyBlocks": ["oak_log", "oak_leaves", "coal_ore", "stone", "water", "sand", ...],
"nearbyEntityTypes": ["player", "creeper", "zombie", "pillager", "skeleton"],
"entityCount": 5
}
```

For the first time today the bot actually sees its surroundings. Cost: $0.062 per snapshot on the ChatGPT Pro subscription.

Hard constraints honoured

✅ All hard safety rules in AGENTS.md preserved
✅ Old broken tools deprecated but not deleted yet (cleanup in follow-up PR after live verification)
✅ MIT attribution + LICENSE-MINDCRAFT vendored alongside source
✅ No skills/, no state/ touched — that's the bot's notebook
✅ Branch-based, not pushed to main

Next after merge

Restart Pi (Ctrl+D + `pi`)
Send autonomous prompt: "observe, then live like a farmer, no operator hand-holding"
Watch BlueMap + diary
If mc_collect_block + mc_place_block work end-to-end → bot can actually build things → village goal possible

🤖 Generated with Claude Code

## Why The bot was acting **blind**: only its own coords, no view of surroundings. `mc_goto`'s over-strict safety guards refused every real path. `mc_dig` was too low-level to drive a farming loop. The result: 4+ hours of "trying" with **zero physical achievement** (бот всё это время сидел на дереве Y≈88, не сделал ни шага по миру). ## What Reframes the bridge around a **perceive→decide→act loop** using proven primitives from [Mindcraft](https://github.com/kolbytn/mindcraft) (MIT — vendored with attribution). ### Files added - `extensions/lib/{world.js,skills.js,mcdata.js,settings.js,LICENSE-MINDCRAFT}` — vendored Mindcraft library - world.js: 21 perception functions - skills.js: 30+ action primitives - mcdata.js: adapter, with `attachPluginsAndInit(bot)` export so our externally-created bot gets the plugins loaded - `extensions/mindcraft-skills.ts` — new Pi extension with **15 high-level tools**: - Perception: `mc_observe`, `mc_inventory`, `mc_nearby_blocks`, `mc_nearby_entities` - Action: `mc_collect_block`, `mc_place_block`, `mc_go_to`, `mc_go_to_block`, `mc_craft`, `mc_equip`, `mc_consume`, `mc_defend_self`, `mc_avoid_enemies`, `mc_stay`, `mc_pickup_nearby` ### Files changed - `extensions/mineflayer-bridge.ts` — expose `globalThis.__pepaPiBot` so the new extension can access the bot; call `attachPluginsAndInit()` after `createBot` to wire pathfinder/pvp/collectblock/armorManager. - `AGENTS.md` — new \"Perception → decision → action\" section with full tool catalog and deprecation note for `mc_goto` / `mc_build_pyramid_5x5` / `mc_dig`. - `package.json` — new deps: `minecraft-data`, `vec3`, `mineflayer-pvp`, `prismarine-item`. ## Smoke test Launched Pi locally on `medium` thinking. Bot called `mc_observe` and got a **real** JSON snapshot: \`\`\`json { \"health\": 20, \"food\": 20, \"saturation\": 5, \"nearbyBlocks\": [\"oak_log\", \"oak_leaves\", \"coal_ore\", \"stone\", \"water\", \"sand\", ...], \"nearbyEntityTypes\": [\"player\", \"creeper\", \"zombie\", \"pillager\", \"skeleton\"], \"entityCount\": 5 } \`\`\` For the first time today the bot actually **sees** its surroundings. Cost: \$0.062 per snapshot on the ChatGPT Pro subscription. ## Hard constraints honoured - ✅ All hard safety rules in AGENTS.md preserved - ✅ Old broken tools deprecated but not deleted yet (cleanup in follow-up PR after live verification) - ✅ MIT attribution + LICENSE-MINDCRAFT vendored alongside source - ✅ No `skills/`, no `state/` touched — that's the bot's notebook - ✅ Branch-based, not pushed to main ## Next after merge 1. Restart Pi (Ctrl+D + \`pi\`) 2. Send autonomous prompt: \"observe, then live like a farmer, no operator hand-holding\" 3. Watch BlueMap + diary 4. If `mc_collect_block` + `mc_place_block` work end-to-end → bot can actually build things → village goal possible 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: xmatic/pepa-pi-bot#2