feat(runtime): fully autonomous self-healing — no operator approval #10
+54
-37
@@ -189,53 +189,70 @@ The protocol is intentionally tiny — anyone can write a second client
|
||||
(a Telegram bridge, a web UI, a one-shot CLI) by reading
|
||||
`runtime/ipc-protocol.js`.
|
||||
|
||||
## Self-improvement loop
|
||||
## Self-improvement loop (fully autonomous)
|
||||
|
||||
End-to-end and wired. The flow:
|
||||
End-to-end, no operator-in-the-loop. The bot writes proposals when it
|
||||
spots a *real* bug, applies them with Pi headless, and rolls them back
|
||||
if they break things. The flow:
|
||||
|
||||
```
|
||||
1. reflex chain dispatches an action → action returns { ok: false, detail }
|
||||
2. bot.js failure tracker accumulates the failure under its label
|
||||
3. same label fails 3× in a row → writeProposal() → markdown lands in
|
||||
state/<host>/proposals/<ts>-<slug>.md
|
||||
4. next IPC STATUS event includes pendingProposals: N
|
||||
5. TUI shows [proposals N, press y] badge
|
||||
6. operator presses y, reads the proposal, presses y again to approve
|
||||
7. proposal moves to state/<host>/proposals/approved/
|
||||
8. operator runs: npm run propose:apply <filename>
|
||||
9. script verifies clean working tree, creates feat/proposal-<slug>
|
||||
branch, spawns `pi -p` with the proposal + repo-conventions prompt
|
||||
10. Pi commits a patch on that branch (no push, no merge)
|
||||
11. operator reviews diff, runs `npm run bot` to smoke-test
|
||||
12. operator pushes the branch and opens a PR by hand
|
||||
13. supervisor on the running bot picks up runtime/*.js changes and
|
||||
hot-restarts the child the moment they hit disk
|
||||
1. reflex dispatches action → action returns { ok: false, detail }
|
||||
2. bot.js failure tracker classifies the detail:
|
||||
bug → TypeError / Cannot read / is not defined …
|
||||
timeout → "timed out after Ns"
|
||||
feature-gap → "no reachable log", "no food", "no bed" …
|
||||
other → anything else
|
||||
3. 5 consecutive failures with the SAME label, where the run is dominated
|
||||
by 'bug' or all 'timeout' → writeProposal()
|
||||
(feature gaps are SKIPPED — reflex routing solves those, not the LLM)
|
||||
4. runtime/auto-improve.js watcher (poll 2s) sees the new file,
|
||||
debounces 10s, then spawns scripts/auto-patch.js detached
|
||||
5. auto-patch.js:
|
||||
- refuses on dirty tree
|
||||
- moves proposal pending → approved/ (audit trail)
|
||||
- creates branch auto/<slug> off main
|
||||
- runs `pi -p "<patch prompt>"` with 10-min timeout
|
||||
- if Pi committed AND only touched runtime/ → cherry-pick onto main
|
||||
- else → discard branch, exit non-zero
|
||||
6. supervisor's runtime/*.js watcher fires the moment the cherry-pick
|
||||
lands → child restarts on the new code
|
||||
7. if the new code crashes >5 times in 60s AND the last commit on main
|
||||
is younger than 15 min AND it touched runtime/ → supervisor
|
||||
`git reset --hard HEAD~1` and restarts. Up to MAX_ROLLBACKS times
|
||||
per supervisor lifetime, then bails out for manual investigation.
|
||||
```
|
||||
|
||||
Operator is in the loop at three guardrails: approving the proposal,
|
||||
reviewing Pi's diff, deciding to merge.
|
||||
### Rate limits
|
||||
|
||||
### Triggers (today)
|
||||
- **Proposal cooldown**: 30 min between proposal files of any kind.
|
||||
- **Auto-improve cooldown**: 15 min between finished `auto-patch.js` runs.
|
||||
- **Hourly cap**: max 4 auto-patches per hour, even if cooldown allows.
|
||||
- **Rollback cap**: 3 rollbacks per supervisor lifetime; after that the
|
||||
supervisor exits and waits for human review.
|
||||
|
||||
Only one detector is wired: "same labelled action fails 3 times in a
|
||||
row" — for example, three back-to-back `flee from zombie` failures.
|
||||
30 min cooldown so the same proposal doesn't multiply when the bot
|
||||
keeps trying.
|
||||
### What counts as a bug
|
||||
|
||||
More triggers worth adding (each as a small follow-up):
|
||||
- "Pi auto-escalation fired but the snapshot didn't change in the next
|
||||
N ticks" → bot is fundamentally stuck, propose a code change.
|
||||
- "death count >K in M minutes at similar coords" → safety regression.
|
||||
- "operator typed the same chat command twice and the bot couldn't act"
|
||||
→ missing operator verb.
|
||||
`runtime/bot.js` ships two whitelists (`NORMAL_FAILURE_SUBSTRINGS` and
|
||||
`BUG_FAILURE_SUBSTRINGS`). The proposal trigger fires only when:
|
||||
- the trailing run of same-label failures contains at least one bug
|
||||
(TypeError / ReferenceError / "Cannot read properties" / etc.),
|
||||
- OR every failure in the run is a timeout (and they happened on the
|
||||
same operation, so it's probably broken not just unreachable).
|
||||
|
||||
### Why a manual `propose:apply` step
|
||||
Feature gaps like "no reachable log within 32 blocks" are *not* a bug
|
||||
— the autonomous reflex sees that result, sets `noTreesUntil` and
|
||||
switches to wander. If the script can't solve it via reflex routing,
|
||||
that's a design issue the operator fixes by editing `runtime/reflex.js`
|
||||
directly — not by asking Pi to patch around it.
|
||||
|
||||
Approval inside the TUI is cheap — one keypress. Spawning Pi to write a
|
||||
patch is not (subscription tokens, multiple minutes). Splitting "I want
|
||||
this addressed" (TUI) from "now actually run the patcher" (CLI) means
|
||||
you can approve five proposals over a session and dispatch them in a
|
||||
batch when convenient.
|
||||
### Manual escape hatches
|
||||
|
||||
These still work but should rarely be needed:
|
||||
- TUI hotkey `y` opens the latest pending proposal for inspection.
|
||||
- `npm run propose:apply <filename>` runs the *attended* version of the
|
||||
patcher — leaves the result on a `feat/proposal-<slug>` branch
|
||||
without cherry-picking, so the operator can review the diff manually.
|
||||
- `npm run stop` kills everything and clears lock/socket.
|
||||
|
||||
## File layout
|
||||
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
// Auto-improve watcher: when a new proposal file appears under
|
||||
// state/<host>/proposals/, debounce briefly (in case the writer is still
|
||||
// finishing), then spawn scripts/auto-patch.js as a detached background
|
||||
// process. The patcher creates a branch, runs Pi headless, and cherry-picks
|
||||
// any resulting commit onto main. The supervisor's runtime/*.js watcher
|
||||
// then triggers a child restart picking up the new code.
|
||||
//
|
||||
// One auto-improve in flight at a time; a 15-minute cooldown between
|
||||
// finished runs caps the Pi token burn rate. Failures keep the proposal
|
||||
// in approved/ (already auto-moved by the patcher) so a future run could
|
||||
// retry — but with the cooldown, this isn't a tight loop.
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { stateDir } from "./config.js";
|
||||
import { info, warn } from "./log.js";
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
const REPO_ROOT = path.resolve(__dirname, "..");
|
||||
const PATCH_SCRIPT = path.join(REPO_ROOT, "scripts", "auto-patch.js");
|
||||
const PROPOSALS_DIR = path.join(stateDir, "proposals");
|
||||
|
||||
const DEBOUNCE_MS = 10_000;
|
||||
const COOLDOWN_MS = 15 * 60 * 1000;
|
||||
const MAX_TOTAL_PER_HOUR = 4;
|
||||
|
||||
let inFlight = false;
|
||||
let lastFinishedAt = 0;
|
||||
const recentRuns = []; // timestamps
|
||||
|
||||
function withinHourlyCap() {
|
||||
const now = Date.now();
|
||||
while (recentRuns.length && now - recentRuns[0] > 3600_000) recentRuns.shift();
|
||||
return recentRuns.length >= MAX_TOTAL_PER_HOUR;
|
||||
}
|
||||
|
||||
function listPendingProposals() {
|
||||
try {
|
||||
return fs
|
||||
.readdirSync(PROPOSALS_DIR)
|
||||
.filter((f) => f.endsWith(".md"))
|
||||
.sort();
|
||||
} catch (e) {
|
||||
if (e.code === "ENOENT") return [];
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
function spawnPatcher(filename) {
|
||||
info("auto-improve", `spawning auto-patch for ${filename}`);
|
||||
inFlight = true;
|
||||
const child = spawn(process.execPath, [PATCH_SCRIPT, filename], {
|
||||
cwd: REPO_ROOT,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
env: { ...process.env },
|
||||
detached: false,
|
||||
});
|
||||
let stdoutBuf = "";
|
||||
let stderrBuf = "";
|
||||
child.stdout.on("data", (c) => {
|
||||
stdoutBuf += c.toString();
|
||||
});
|
||||
child.stderr.on("data", (c) => {
|
||||
stderrBuf += c.toString();
|
||||
});
|
||||
child.on("exit", (code) => {
|
||||
inFlight = false;
|
||||
lastFinishedAt = Date.now();
|
||||
recentRuns.push(lastFinishedAt);
|
||||
const tail = (stdoutBuf + "\n" + stderrBuf).trim().split("\n").slice(-3).join(" | ");
|
||||
if (code === 0) info("auto-improve", `patch applied for ${filename}: ${tail}`);
|
||||
else warn("auto-improve", `patch did not apply (code=${code}) for ${filename}: ${tail}`);
|
||||
// We do NOT trigger supervisor restart manually — the supervisor's
|
||||
// fs.watch on runtime/*.js fires the moment the cherry-pick lands.
|
||||
});
|
||||
}
|
||||
|
||||
function tick() {
|
||||
if (inFlight) return;
|
||||
if (withinHourlyCap()) return;
|
||||
if (Date.now() - lastFinishedAt < COOLDOWN_MS) return;
|
||||
|
||||
const pending = listPendingProposals();
|
||||
if (pending.length === 0) return;
|
||||
|
||||
// Pick the oldest pending proposal (sorted lexically — timestamps in name).
|
||||
const filename = pending[0];
|
||||
|
||||
// Debounce: ensure the file has been still for DEBOUNCE_MS so we don't
|
||||
// race a partial write. Track first-seen timestamp per filename.
|
||||
if (!debounceMap.has(filename)) {
|
||||
debounceMap.set(filename, Date.now());
|
||||
return;
|
||||
}
|
||||
const firstSeen = debounceMap.get(filename);
|
||||
if (Date.now() - firstSeen < DEBOUNCE_MS) return;
|
||||
|
||||
debounceMap.delete(filename);
|
||||
spawnPatcher(filename);
|
||||
}
|
||||
|
||||
const debounceMap = new Map();
|
||||
let pollTimer = null;
|
||||
|
||||
export function startAutoImprover() {
|
||||
if (pollTimer) return;
|
||||
info("auto-improve", `watching ${PROPOSALS_DIR} (cooldown=${COOLDOWN_MS / 1000}s, max ${MAX_TOTAL_PER_HOUR}/hr)`);
|
||||
pollTimer = setInterval(tick, 2000);
|
||||
}
|
||||
|
||||
export function stopAutoImprover() {
|
||||
if (pollTimer) clearInterval(pollTimer);
|
||||
pollTimer = null;
|
||||
}
|
||||
+87
-20
@@ -30,6 +30,7 @@ import {
|
||||
readProposal,
|
||||
approveProposal,
|
||||
} from "./state-store.js";
|
||||
import { startAutoImprover } from "./auto-improve.js";
|
||||
|
||||
fs.mkdirSync(stateDir, { recursive: true });
|
||||
const JOINED_FLAG = path.join(stateDir, "joined-before.flag");
|
||||
@@ -136,7 +137,9 @@ function dispatchAction(fn, label, opts = {}) {
|
||||
}
|
||||
reflexCtx.busy = true;
|
||||
reflexCtx.currentActionLabel = label;
|
||||
writeCurrentTask({ label, status: "in_progress", snapshot: lastSnapshot });
|
||||
// current-task is a resume anchor — keep it small. Embedding the full
|
||||
// perception snapshot blows the file up to ~3 KB per write × every action.
|
||||
writeCurrentTask({ label, status: "in_progress", position: lastSnapshot.position });
|
||||
info("dispatch", `→ ${label}`);
|
||||
Promise.resolve()
|
||||
.then(() => fn())
|
||||
@@ -170,14 +173,56 @@ function dispatchAction(fn, label, opts = {}) {
|
||||
}
|
||||
|
||||
// ---- failure tracking + proposal detection --------------------------------
|
||||
//
|
||||
// A proposal is a request to the LLM to patch the codebase. They cost tokens
|
||||
// and may produce risky patches that need rolling back. We file them ONLY for
|
||||
// failures that genuinely look like bugs the reflex layer can't handle on its
|
||||
// own. Everything else is a feature gap the script should solve via reflex
|
||||
// chain reordering, cooldowns, or new primitives.
|
||||
|
||||
const PROPOSAL_THRESHOLD = 3; // same labelled action fails 3+ times in a row
|
||||
const PROPOSAL_THRESHOLD = 5; // raised from 3 to dampen spam
|
||||
let lastProposalAt = 0;
|
||||
const PROPOSAL_COOLDOWN_MS = 30 * 60 * 1000; // don't spam proposal files
|
||||
const PROPOSAL_COOLDOWN_MS = 30 * 60 * 1000;
|
||||
|
||||
// Detail substrings that mean "this is a known feature gap, the bot handles
|
||||
// it via reflex routing already". Don't file a proposal — the bot will switch
|
||||
// strategies on its own. If something here is wrong, fix the routing.
|
||||
const NORMAL_FAILURE_SUBSTRINGS = [
|
||||
"no reachable log",
|
||||
"no log within",
|
||||
"no bed in range",
|
||||
"no food in inventory",
|
||||
"no target in reach",
|
||||
"rate-limited",
|
||||
"can't see you nearby",
|
||||
"returned false",
|
||||
"no result",
|
||||
];
|
||||
|
||||
// Detail substrings that look like a real bug — patch-worthy.
|
||||
const BUG_FAILURE_SUBSTRINGS = [
|
||||
"TypeError",
|
||||
"ReferenceError",
|
||||
"Cannot read properties",
|
||||
"is not a function",
|
||||
"is not iterable",
|
||||
"is not defined",
|
||||
"unknown block",
|
||||
"unknown item",
|
||||
];
|
||||
|
||||
function classifyFailure(detail) {
|
||||
const s = String(detail ?? "");
|
||||
if (BUG_FAILURE_SUBSTRINGS.some((sub) => s.includes(sub))) return "bug";
|
||||
if (NORMAL_FAILURE_SUBSTRINGS.some((sub) => s.includes(sub))) return "feature-gap";
|
||||
if (s.includes("timed out")) return "timeout";
|
||||
return "other";
|
||||
}
|
||||
|
||||
function recordFailure(label, detail) {
|
||||
reflexCtx.recentFailures.push({ ts: Date.now(), label, detail });
|
||||
if (reflexCtx.recentFailures.length > 10) reflexCtx.recentFailures.shift();
|
||||
const kind = classifyFailure(detail);
|
||||
reflexCtx.recentFailures.push({ ts: Date.now(), label, detail, kind });
|
||||
if (reflexCtx.recentFailures.length > 20) reflexCtx.recentFailures.shift();
|
||||
maybeFileProposal(label);
|
||||
}
|
||||
|
||||
@@ -186,7 +231,7 @@ function clearRecentFailures(label) {
|
||||
}
|
||||
|
||||
function maybeFileProposal(label) {
|
||||
// Count consecutive trailing failures with the same label.
|
||||
// Same-label trailing run.
|
||||
const trailing = [];
|
||||
for (let i = reflexCtx.recentFailures.length - 1; i >= 0; i--) {
|
||||
const f = reflexCtx.recentFailures[i];
|
||||
@@ -195,37 +240,58 @@ function maybeFileProposal(label) {
|
||||
}
|
||||
if (trailing.length < PROPOSAL_THRESHOLD) return;
|
||||
if (Date.now() - lastProposalAt < PROPOSAL_COOLDOWN_MS) return;
|
||||
|
||||
// Only file when the run is dominated by bug-class failures (any single
|
||||
// bug counts) OR persistent timeouts on the same operation. Feature gaps
|
||||
// are skipped — the reflex layer should re-route, not the LLM.
|
||||
const anyBug = trailing.some((f) => f.kind === "bug");
|
||||
const allTimeout = trailing.every((f) => f.kind === "timeout");
|
||||
if (!anyBug && !allTimeout) return;
|
||||
|
||||
lastProposalAt = Date.now();
|
||||
|
||||
const summary = `${label} failed ${trailing.length}× in a row`;
|
||||
const summary = `${label} failed ${trailing.length}× in a row (${anyBug ? "bug" : "persistent timeout"})`;
|
||||
const slimSnapshot = lastSnapshot && {
|
||||
position: lastSnapshot.position,
|
||||
health: lastSnapshot.health,
|
||||
food: lastSnapshot.food,
|
||||
inventory: lastSnapshot.inventory,
|
||||
isDay: lastSnapshot.isDay,
|
||||
closestHostile: lastSnapshot.closestHostile,
|
||||
dimension: lastSnapshot.dimension,
|
||||
};
|
||||
const body = [
|
||||
`# Repeated failure: ${label}`,
|
||||
"",
|
||||
`Class: **${anyBug ? "bug" : "persistent timeout"}**.`,
|
||||
"",
|
||||
"## What happened",
|
||||
"",
|
||||
`The reflex layer dispatched \`${label}\` ${trailing.length} times in succession without a single success.`,
|
||||
"",
|
||||
"## Most recent failures",
|
||||
"",
|
||||
...trailing.slice(0, 5).map(
|
||||
(f, i) => `${i + 1}. \`${new Date(f.ts).toISOString()}\` — ${JSON.stringify(f.detail).slice(0, 200)}`,
|
||||
),
|
||||
...trailing
|
||||
.slice(0, 5)
|
||||
.map(
|
||||
(f, i) =>
|
||||
`${i + 1}. \`${new Date(f.ts).toISOString()}\` [${f.kind}] ${JSON.stringify(f.detail).slice(0, 200)}`,
|
||||
),
|
||||
"",
|
||||
"## Snapshot at moment of last failure",
|
||||
"## Slim snapshot",
|
||||
"",
|
||||
"```json",
|
||||
JSON.stringify(lastSnapshot, null, 2),
|
||||
JSON.stringify(slimSnapshot, null, 2),
|
||||
"```",
|
||||
"",
|
||||
"## Suggested next step",
|
||||
"## Constraints for the patch",
|
||||
"",
|
||||
"Operator: review whether the reflex should:",
|
||||
"- back off (cooldown extension)",
|
||||
"- switch to a different action variant",
|
||||
"- escalate to Pi for situational reasoning",
|
||||
"- or whether the underlying primitive in `runtime/actions.js` needs work.",
|
||||
"",
|
||||
`Approve this proposal (move to \`proposals/approved/\`) and run \`npm run propose:apply <filename>\` to delegate a patch attempt to Pi headless.`,
|
||||
"- Touch only files under `runtime/`. Don't touch `extensions/`, `tui/`, or any docs.",
|
||||
"- Don't introduce new npm dependencies.",
|
||||
"- Don't change `.env` or anything in `state/`.",
|
||||
"- Don't push, don't open a PR. Commit on the current branch only.",
|
||||
"- Prefer the smallest viable fix. A 3-line guard is better than a 30-line refactor.",
|
||||
"- If the failure is genuinely irrecoverable (server-side, not code), document it in a code comment and exit 1.",
|
||||
].join("\n");
|
||||
|
||||
const { filename } = writeProposal({ kind: `repeated-fail-${label}`, summary, body });
|
||||
@@ -591,3 +657,4 @@ ipc = createIpcServer({
|
||||
});
|
||||
connect();
|
||||
startTickLoop();
|
||||
startAutoImprover();
|
||||
|
||||
+52
-1
@@ -11,7 +11,7 @@
|
||||
// Run via `npm run bot`. Falls back to plain `node runtime/bot.js` via
|
||||
// `npm run bot:bare` if you want to skip the supervisor.
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
@@ -21,11 +21,17 @@ import { stateDir } from "./config.js";
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
const RUNTIME_DIR = __dirname;
|
||||
const REPO_ROOT = path.resolve(RUNTIME_DIR, "..");
|
||||
const BOT_ENTRY = path.join(RUNTIME_DIR, "bot.js");
|
||||
|
||||
export const RELOAD_EXIT_CODE = 42;
|
||||
const WATCH_DEBOUNCE_MS = 800;
|
||||
const MAX_RESTARTS_PER_MINUTE = 5;
|
||||
// When a recent auto-patch breaks the bot, we roll it back. "Recent" =
|
||||
// landed on main within the last ROLLBACK_FRESHNESS_MS. The signal is
|
||||
// MAX_RESTARTS_PER_MINUTE exceeded — i.e. the patch reliably crashes.
|
||||
const ROLLBACK_FRESHNESS_MS = 15 * 60_000;
|
||||
const MAX_ROLLBACKS = 3; // hard ceiling per supervisor lifetime
|
||||
|
||||
// Pidfile prevents two supervisors from racing on the same MC nickname. The
|
||||
// Minecraft server refuses the second login ("Игрок с данным никнеймом уже
|
||||
@@ -37,6 +43,37 @@ const PID_FILE = path.join(stateDir, "supervisor.pid");
|
||||
let child = null;
|
||||
let restartingDueToWatch = false;
|
||||
const restartTimestamps = [];
|
||||
let rollbackCount = 0;
|
||||
|
||||
function lastCommitAgeMs() {
|
||||
const res = spawnSync("git", ["log", "-1", "--format=%ct", "HEAD"], { cwd: REPO_ROOT, encoding: "utf8" });
|
||||
if (res.status !== 0) return Number.POSITIVE_INFINITY;
|
||||
const epochSec = Number.parseInt(res.stdout.trim(), 10);
|
||||
if (!Number.isFinite(epochSec)) return Number.POSITIVE_INFINITY;
|
||||
return Date.now() - epochSec * 1000;
|
||||
}
|
||||
|
||||
function lastCommitTouchedRuntime() {
|
||||
const res = spawnSync("git", ["diff", "--name-only", "HEAD~1..HEAD"], { cwd: REPO_ROOT, encoding: "utf8" });
|
||||
if (res.status !== 0) return false;
|
||||
return res.stdout.split("\n").some((f) => f.startsWith("runtime/"));
|
||||
}
|
||||
|
||||
function rollbackLastCommit() {
|
||||
const sha = spawnSync("git", ["rev-parse", "HEAD"], { cwd: REPO_ROOT, encoding: "utf8" }).stdout.trim();
|
||||
console.log(`[supervisor] rolling back HEAD (${sha.slice(0, 8)})`);
|
||||
const reset = spawnSync("git", ["reset", "--hard", "HEAD~1"], {
|
||||
cwd: REPO_ROOT,
|
||||
encoding: "utf8",
|
||||
stdio: "inherit",
|
||||
});
|
||||
if (reset.status !== 0) {
|
||||
console.error(`[supervisor] git reset failed — operator intervention needed`);
|
||||
return false;
|
||||
}
|
||||
rollbackCount++;
|
||||
return true;
|
||||
}
|
||||
|
||||
function nowMs() {
|
||||
return Date.now();
|
||||
@@ -97,6 +134,20 @@ function spawnChild() {
|
||||
restartTimestamps.push(now);
|
||||
while (restartTimestamps.length && now - restartTimestamps[0] > 60_000) restartTimestamps.shift();
|
||||
if (restartTimestamps.length > MAX_RESTARTS_PER_MINUTE) {
|
||||
// Crash loop. If the last commit is young AND touched runtime/, it
|
||||
// probably broke us — roll it back and try once more.
|
||||
const ageMs = lastCommitAgeMs();
|
||||
if (
|
||||
ageMs < ROLLBACK_FRESHNESS_MS &&
|
||||
lastCommitTouchedRuntime() &&
|
||||
rollbackCount < MAX_ROLLBACKS &&
|
||||
rollbackLastCommit()
|
||||
) {
|
||||
console.log(`[supervisor] auto-rollback ${rollbackCount}/${MAX_ROLLBACKS} applied; restart counters reset`);
|
||||
restartTimestamps.length = 0;
|
||||
setTimeout(spawnChild, 500);
|
||||
return;
|
||||
}
|
||||
console.error(`[supervisor] too many restarts (${restartTimestamps.length} in 60s) — giving up`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
#!/usr/bin/env node
|
||||
// auto-patch.js <proposal-filename>
|
||||
//
|
||||
// Unattended sibling of propose-apply.js. Picks an *unapproved* proposal,
|
||||
// moves it to approved/, branches off main, runs `pi -p` headless, and if Pi
|
||||
// commits something — cherry-picks the commit back into main. The point is
|
||||
// to close the self-improvement loop with no operator interaction.
|
||||
//
|
||||
// Exit codes:
|
||||
// 0 patch applied cleanly (commit on main)
|
||||
// 1 pi spawned but produced no commit (no change to repo)
|
||||
// 2 preflight failed (dirty tree, missing proposal, etc.)
|
||||
// 3 pi exited non-zero
|
||||
// 4 cherry-pick conflict — left in unresolved state on a branch
|
||||
//
|
||||
// Designed to be launched by runtime/auto-improve.js as a detached child.
|
||||
// We deliberately avoid touching anything outside the repo and don't push.
|
||||
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
const REPO_ROOT = path.resolve(__dirname, "..");
|
||||
|
||||
function log(level, msg) {
|
||||
const line = `${new Date().toISOString()} [auto-patch] [${level}] ${msg}`;
|
||||
if (level === "error" || level === "warn") console.error(line);
|
||||
else console.log(line);
|
||||
}
|
||||
|
||||
function git(args, opts = {}) {
|
||||
return spawnSync("git", args, { cwd: REPO_ROOT, encoding: "utf8", ...opts });
|
||||
}
|
||||
|
||||
function exit(code, reason) {
|
||||
log(code === 0 ? "info" : "warn", `exit ${code}: ${reason}`);
|
||||
process.exit(code);
|
||||
}
|
||||
|
||||
const filenameArg = process.argv[2];
|
||||
if (!filenameArg) exit(2, "usage: auto-patch.js <proposal-filename>");
|
||||
|
||||
function findProposal(filename) {
|
||||
const stateRoot = path.join(REPO_ROOT, "state");
|
||||
if (!fs.existsSync(stateRoot)) return null;
|
||||
for (const host of fs.readdirSync(stateRoot)) {
|
||||
const pending = path.join(stateRoot, host, "proposals", filename);
|
||||
const approved = path.join(stateRoot, host, "proposals", "approved", filename);
|
||||
if (fs.existsSync(pending)) return { path: pending, host, status: "pending" };
|
||||
if (fs.existsSync(approved)) return { path: approved, host, status: "approved" };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
const proposal = findProposal(filenameArg);
|
||||
if (!proposal) exit(2, `proposal not found: ${filenameArg}`);
|
||||
|
||||
// Refuse on dirty tree — we'd lose the operator's WIP.
|
||||
const dirty = git(["status", "--porcelain"]).stdout.trim();
|
||||
if (dirty) exit(2, `working tree dirty: ${dirty.split("\n")[0]}`);
|
||||
|
||||
// Move pending → approved so we don't try to apply the same proposal twice.
|
||||
if (proposal.status === "pending") {
|
||||
const approvedDir = path.join(path.dirname(proposal.path), "approved");
|
||||
fs.mkdirSync(approvedDir, { recursive: true });
|
||||
const dst = path.join(approvedDir, filenameArg);
|
||||
const content = fs.readFileSync(proposal.path, "utf8").replace(/^approved: false/m, "approved: true (auto)");
|
||||
fs.writeFileSync(dst, content);
|
||||
fs.unlinkSync(proposal.path);
|
||||
proposal.path = dst;
|
||||
log("info", `moved pending → approved: ${filenameArg}`);
|
||||
}
|
||||
|
||||
const proposalText = fs.readFileSync(proposal.path, "utf8");
|
||||
|
||||
// Capture current main HEAD so we can roll back to it if cherry-pick fails.
|
||||
const baseSha = git(["rev-parse", "HEAD"]).stdout.trim();
|
||||
|
||||
const slug = filenameArg
|
||||
.replace(/\.md$/, "")
|
||||
.replace(/[^a-zA-Z0-9-]+/g, "-")
|
||||
.slice(0, 60);
|
||||
const branch = `auto/${slug}`;
|
||||
|
||||
// Delete the branch if it exists from a previous failed attempt.
|
||||
git(["branch", "-D", branch]); // ignore error if absent
|
||||
|
||||
const checkout = git(["checkout", "-b", branch]);
|
||||
if (checkout.status !== 0) exit(2, `cannot create branch ${branch}: ${checkout.stderr}`);
|
||||
|
||||
const prompt = [
|
||||
"You are patching the pepa-pi-bot repo to address an automatically-detected failure.",
|
||||
"This is an UNATTENDED run — no operator will review your output before it lands on main.",
|
||||
"Be conservative. Prefer guard clauses and small surgical edits.",
|
||||
"",
|
||||
"## The proposal",
|
||||
"",
|
||||
proposalText,
|
||||
"",
|
||||
"## Hard rules (non-negotiable)",
|
||||
"",
|
||||
"1. Touch only files under `runtime/`. Do NOT modify `tui/`, `extensions/`, `scripts/`, `docs/`, `package.json`, or anything in `state/`.",
|
||||
"2. Do not introduce npm dependencies.",
|
||||
"3. Do not push, do not open a PR. Commit on the current branch only.",
|
||||
"4. Use a conventional commit message: `fix(runtime/<file>): <one-line summary>`.",
|
||||
"5. If you can't safely fix the issue, write a short comment in the relevant runtime file explaining why and stop — do NOT make a speculative change.",
|
||||
"6. Make exactly ONE commit. If you find multiple issues, focus on the one the proposal describes.",
|
||||
"",
|
||||
"After you commit, your job is done.",
|
||||
].join("\n");
|
||||
|
||||
log("info", `spawning pi -p (timeout 10 min)`);
|
||||
const pi = spawn("pi", ["-p", prompt], {
|
||||
cwd: REPO_ROOT,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
let piStdout = "";
|
||||
let piStderr = "";
|
||||
pi.stdout.on("data", (chunk) => {
|
||||
piStdout += chunk.toString();
|
||||
});
|
||||
pi.stderr.on("data", (chunk) => {
|
||||
piStderr += chunk.toString();
|
||||
});
|
||||
|
||||
const PI_TIMEOUT_MS = 10 * 60 * 1000;
|
||||
const timer = setTimeout(() => {
|
||||
log("warn", "pi timeout — killing subprocess");
|
||||
pi.kill("SIGTERM");
|
||||
}, PI_TIMEOUT_MS);
|
||||
|
||||
pi.on("exit", (code) => {
|
||||
clearTimeout(timer);
|
||||
log("info", `pi exited code=${code}; stdout=${piStdout.length}B stderr=${piStderr.length}B`);
|
||||
|
||||
if (code !== 0) {
|
||||
// Pi crashed or timed out — return to main, drop the branch.
|
||||
git(["checkout", "main"]);
|
||||
git(["branch", "-D", branch]);
|
||||
exit(3, `pi exited ${code}`);
|
||||
}
|
||||
|
||||
const newHead = git(["rev-parse", "HEAD"]).stdout.trim();
|
||||
if (newHead === baseSha) {
|
||||
// Pi did not commit anything. Clean up.
|
||||
git(["checkout", "main"]);
|
||||
git(["branch", "-D", branch]);
|
||||
exit(1, "pi made no commit");
|
||||
}
|
||||
|
||||
// Verify the commit touched only runtime/.
|
||||
const filesChanged = git(["diff", "--name-only", `${baseSha}..HEAD`]).stdout.trim().split("\n").filter(Boolean);
|
||||
const outside = filesChanged.filter((f) => !f.startsWith("runtime/"));
|
||||
if (outside.length > 0) {
|
||||
log("error", `commit touched files outside runtime/: ${outside.join(", ")} — discarding`);
|
||||
git(["checkout", "main"]);
|
||||
git(["branch", "-D", branch]);
|
||||
exit(2, "patch touched off-limits files");
|
||||
}
|
||||
|
||||
// Cherry-pick onto main.
|
||||
git(["checkout", "main"]);
|
||||
const cherry = git(["cherry-pick", newHead]);
|
||||
if (cherry.status !== 0) {
|
||||
log("error", `cherry-pick failed: ${cherry.stderr}`);
|
||||
// Leave the branch around for operator inspection; abort the failed
|
||||
// cherry-pick so main is clean.
|
||||
git(["cherry-pick", "--abort"]);
|
||||
exit(4, `cherry-pick conflict — see branch ${branch}`);
|
||||
}
|
||||
|
||||
// Success — delete the feature branch (the commit is on main now).
|
||||
git(["branch", "-D", branch]);
|
||||
log("info", `patch applied to main as ${git(["rev-parse", "HEAD"]).stdout.trim().slice(0, 8)}`);
|
||||
exit(0, `applied ${filenameArg}`);
|
||||
});
|
||||
Reference in New Issue
Block a user