fix(activity): stabilize codex and opencode states

.gitignore

@@ -3,6 +3,8 @@ dist/
 test-results/
 playwright-report/
 tmp/
+.worktrees/
+.codex/
 .DS_Store
 .env
 .env.local

AGENTS.md

@@ -199,12 +199,112 @@ SAFE ALTERNATIVE: Provide a legal, safe approach to the same goal.
 
 ## Verification
 - Default: `npm run build` for type-checking.
+- After any changes, run `npm run build` and `npm test`.
+
+## Completion protocol (non-negotiable)
+- Gate order: 1) Build gate, 2) Evidence gate, 3) Regression gate.
+- Build gate: `npm run build` and `npm test` must pass with zero failures, and report the actual summary output.
+- Evidence gate: Every state-change fix must include a reproduction artifact.
+- Flicker fixes require a poller log (JSONL) showing zero `active->idle->active` transitions within the hold window during a live agent run.
+- State logic fixes require a before/after snapshot diff showing the incorrect state and the corrected state.
+- Event parsing fixes require a test case using real JSONL from a captured session, not synthetic data.
+- If an artifact cannot be produced, report the blocker and do not claim the fix works.
+- Regression gate: Confirm that no previously-passing test now fails and no unrelated behavior changed.
+- If unrelated files are modified, flag them explicitly and do not commit them.
+
+## Feature verification workflow (step-by-step)
+This section applies to every feature and every behavior-changing bug fix.
+
+### 1) Define success criteria (before coding)
+1. Write down:
+   - What should change (user-visible behavior and/or API behavior).
+   - How to reproduce the old behavior.
+   - What artifact will prove the fix (log, snapshot diff, test, video).
+2. If the change is UI-visible, define what must be shown in the demo video:
+   - Start state
+   - Active state
+   - End state
+
+### 2) Tests-first (before implementation)
+1. Add or update tests that fail on the current behavior:
+   - Unit tests for pure state/logic.
+   - Integration tests for parsing (prefer real captured JSONL fixtures).
+   - E2E tests for UI invariants (fast, deterministic).
+2. Run:
+   - `npm run build`
+   - `npm test`
+3. Confirm the new/updated tests fail for the expected reason before implementing the fix.
+
+### 3) Implement the feature/fix
+1. Make the minimal code changes required to satisfy the tests and success criteria.
+2. Keep scope tight. If unrelated changes are discovered, isolate them or stop and report.
+
+### 4) Build gate (hard requirement)
+1. Run:
+   - `npm run build`
+   - `npm test`
+2. In the PR description or review comment, paste:
+   - The final `vite build` summary line (e.g. `✓ built in ...ms`).
+   - The final Node test summary lines (`# pass ... # fail 0`).
+
+### 5) Evidence gate (hard requirement)
+Produce artifacts that prove the change, not just that it compiles.
+
+#### 5a) Activity/flicker fixes (required when touching inFlight/lastActivity/session selection)
+1. Run the poller:
+   - `node scripts/flicker-detect.js --interval-ms 250 --duration-ms 120000 --window-ms 10000 --out tmp/flicker-summary.json`
+2. Required artifacts:
+   - `tmp/flicker-summary.json`
+   - `tmp/flicker-summary.transitions.jsonl` (written automatically, or via `--out-jsonl`)
+3. Required report fields:
+   - Polling parameters (interval, duration, window).
+   - `totalFlickerCount` from the JSON summary.
+   - File paths of the JSON and JSONL artifacts.
+
+#### 5b) Snapshot/state fixes (required when touching scan/tail logic)
+1. Capture a "before" and "after" snapshot (or a minimal diff) that shows the incorrect state and corrected state.
+2. Attach the diff as a PR comment or a file under `tmp/` (gitignored) and reference its path.
+
+### 6) Demo video gate (hard requirement for every feature PR)
+Every feature PR must include a 30-second demo video showing the feature working end-to-end.
+
+#### 6a) Deterministic UI demo (preferred default)
+1. Create or update an env-gated Playwright demo test under `e2e/ui/`:
+   - Example pattern: `e2e/ui/<feature>Demo.pw.ts`
+   - It must `test.skip` unless an explicit env var is set.
+2. Record video:
+   - `PW_VIDEO=1 RUN_CODEX_UI_DEMO=1 npx playwright test e2e/ui/codexActivityDemo.pw.ts`
+3. Locate the recorded `video.webm` under `test-results/` (gitignored).
+4. Trim/copy a 30-second share artifact into `tmp/` (gitignored). If `ffmpeg` is available:
+   - `ffmpeg -y -i <path/to/video.webm> -t 30 -c:v libx264 -pix_fmt yuv420p tmp/<feature>-demo-30s.mp4`
+5. Manually review the video and confirm it shows the intended start/active/end states with no regressions.
+
+#### 6b) Live Codex TUI demo (required for Codex activity work)
+1. Use the live demo harness:
+   - `PW_VIDEO=1 RUN_LIVE_CODEX=1 CONSENSUS_PROCESS_MATCH=consensus-tui-demo- CONSENSUS_OPENCODE_EVENTS=0 CONSENSUS_OPENCODE_AUTOSTART=0 npx playwright test e2e/ui/codexTuiLiveDemo.pw.ts`
+2. Produce a 30-second share artifact in `tmp/` (mp4 or webm) and review it.
+
+#### 6c) PR requirement
+1. Upload the video to the PR (GitHub comment attachment or equivalent).
+2. Also record the local path in the PR for traceability (e.g. `tmp/<feature>-demo-30s.mp4`).
+
+## Completion rules
+- `npm test` success must include the summary line, not a claim.
+- "Flicker count: 0" must include the log file path and polling parameters used.
+- If validation is blocked (hooks not firing, file missing, process not found), report it as a blocker.
+- Do not say "if you want, I can validate." Validate first and report results.
+- Do not ask for a next step after a fix. Run the evidence gate and report pass or fail.
+
+## Decision protocol
+- If project instructions or prior conversation already answered the question, do not ask again. Act.
+- If two options exist and one is clearly safer under the "if unsure, stay active" principle, take it and explain why.
+- If progress is genuinely blocked, state the specific blocker in one sentence and stop.
 
 ## Configuration
 - `CONSENSUS_HOST`, `CONSENSUS_PORT`, `CONSENSUS_POLL_MS`, `CONSENSUS_CODEX_HOME`, `CONSENSUS_PROCESS_MATCH`, `CONSENSUS_REDACT_PII`.
 
 ## Do not
-- Add non-trivial tests unless explicitly requested.
+- Ship behavior changes without tests. Features and behavior-changing bug fixes must add or update tests.
 - Introduce large UI frameworks or build tooling.
 <!-- BEGIN BYTEROVER RULES -->
 
@@ -225,4 +325,4 @@ You are a coding agent integrated with ByteRover via MCP (Model Context Protocol
 
 ---
 Generated by ByteRover CLI for Codex
-<!-- END BYTEROVER RULES -->
+<!-- END BYTEROVER RULES -->

CONTINUITY.md

@@ -34,6 +34,7 @@ Key decisions:
 - React UI is primary; server should serve React build when present.
 - Agent lane follows vanilla JS behavior: show all open sessions (active + idle) and sort by state rank then CPU.
 - Layout uses pinned positions per agent identity; only new agents are placed; existing tiles never move; placement uses max-height bounds to prevent overlap.
+- Scale placement with spatial hash buckets (cell -> occupants) and persistent per-group spiral frontier; collisions checked by bounds against bucket occupants.
 - Codex prompt-like titles containing temp paths/turn markers are ignored for lane labels; fallback to repo or codex#pid.
 - Agent lane now shows only active/error sessions and uses stable sort (state rank then identity) to prevent reordering.
 - Active lane items now glow via `is-active` class (lane item box-shadow) to make active codex sessions visibly highlighted.
@@ -133,6 +134,11 @@ State:
     - Tests re-run: `npm run test:unit` (149 pass), `npm run test:integration` (54 pass), `npm run test:ui` (19 pass).
     - Build run after pinned layout changes: `npm run build` (pass).
     - Copied edited files to clipboard via pbcopy; temp bundle at `/tmp/consensus-edited-files.txt`.
+    - Implemented spatial hash buckets + persistent group spiral frontier; fixed cell packing and bounds-based collision checks (public/src/lib/layout.ts).
+    - Renderer now keeps a single RAF loop, exposes hit list, and uses correct view transform for hit tests (public/src/hooks/useCanvasRenderer.ts).
+    - CanvasScene no longer rebuilds hit list; uses renderer hit list and stable render loop (public/src/components/CanvasScene.tsx).
+    - Obstruction detection checks only front occluders with AABB guard (public/src/hooks/useCanvasRenderer.ts).
+    - Tests run after layout/renderer updates: `npm run test:unit` (152 pass), `npm run test:integration` (58 pass), `npm run test:ui` (19 pass).
   - Now:
     - Answer user question about hot reload configuration (Vite HMR + server watch + live reload SSE).
     - Answer whether any additional setup is missing when using Vite.
@@ -161,6 +167,7 @@ State:
     - Claude CLI hooks not firing for real sessions; user reports Claude does not show active when working.
     - Need to validate live Claude session while recording via agent-browser and inspect hook delivery.
     - Copy edited layout/test files to clipboard via pbcopy. (DONE)
+    - Implement spatial hash buckets + persistent group frontier with correct cell packing and bounds checks. (DONE)
     - User requested a best-practice plan (no code edits) to address review comment: `npm run dev` no longer starts Vite, so TSX client fails unless dev:client or build is run.
     - Q&A answered for dev workflow: always start Vite; use CONSENSUS_UI_PORT for port; on port conflict try next port.
     - Review plan requested for new P2/P3 items: Codex in-flight timeout default in codexLogs; TOML [tui] notifications insertion in cli/setup.

README.md

@@ -70,7 +70,7 @@ consensus dev server running on http://127.0.0.1:8787
 ## Configuration
 - `CONSENSUS_HOST`: bind address (default `127.0.0.1`).
 - `CONSENSUS_PORT`: server port (default `8787`).
-- `CONSENSUS_POLL_MS`: process presence polling interval in ms (default `500`).
+- `CONSENSUS_POLL_MS`: process presence polling interval in ms (default `250`, min `50`).
 - `CONSENSUS_SCAN_TIMEOUT_MS`: scan timeout in ms (default `5000`).
 - `CONSENSUS_SCAN_STALL_MS`: scan stall warning threshold in ms (default `60%` of timeout, min `250`).
 - `CONSENSUS_SCAN_STALL_CHECK_MS`: scan stall check interval in ms (default `min(1000, stallMs)`, min `250`).
@@ -85,18 +85,19 @@ consensus dev server running on http://127.0.0.1:8787
 - `CONSENSUS_OPENCODE_EVENT_ACTIVE_MS`: OpenCode active window after last event in ms (default `0`).
 - `CONSENSUS_OPENCODE_ACTIVE_HOLD_MS`: OpenCode hold window in ms (default `3000`).
 - `CONSENSUS_OPENCODE_INFLIGHT_IDLE_MS`: OpenCode in-flight idle timeout in ms (defaults to `CONSENSUS_OPENCODE_INFLIGHT_TIMEOUT_MS`).
-- `CONSENSUS_OPENCODE_INFLIGHT_TIMEOUT_MS`: OpenCode hard in-flight timeout in ms (default `15000`).
+- `CONSENSUS_OPENCODE_INFLIGHT_TIMEOUT_MS`: OpenCode hard in-flight timeout in ms (default `2500`).
+- `CONSENSUS_OPENCODE_INFLIGHT_STALE_MS`: OpenCode stale in-flight cutoff in ms (default `0`).
 - `CONSENSUS_PROCESS_MATCH`: regex to match codex processes.
 - `CONSENSUS_REDACT_PII`: set to `0` to disable redaction (default enabled).
 - `CONSENSUS_UI_PORT`: dev UI port for Vite when running `npm run dev` (default `5173`).
 - `CONSENSUS_DEBUG_OPENCODE`: set to `1` to log OpenCode server discovery.
 - `CONSENSUS_CODEX_EVENT_ACTIVE_MS`: Codex active window after last event in ms (default `30000`).
-- `CONSENSUS_CODEX_ACTIVE_HOLD_MS`: Codex hold window in ms (default `3000`).
+- `CONSENSUS_CODEX_ACTIVE_HOLD_MS`: Codex hold window in ms (default `2000`, set to `0` to disable).
 - `CONSENSUS_CODEX_INFLIGHT_IDLE_MS`: Codex in-flight idle timeout in ms (default `30000`, set to `0` to disable).
 - `CONSENSUS_CODEX_CPU_SUSTAIN_MS`: sustained CPU window before Codex becomes active without logs (default `500`).
 - `CONSENSUS_CODEX_CPU_SPIKE`: Codex CPU spike threshold for immediate activation (default derived).
-- `CONSENSUS_CODEX_INFLIGHT_TIMEOUT_MS`: Codex in-flight timeout in ms (default `3000`).
-- `CONSENSUS_CODEX_SIGNAL_MAX_AGE_MS`: Codex max event age for in-flight signals (default `CONSENSUS_CODEX_INFLIGHT_TIMEOUT_MS`).
+- `CONSENSUS_CODEX_INFLIGHT_TIMEOUT_MS`: Codex in-flight timeout in ms (default `2500`).
+- `CONSENSUS_CODEX_SIGNAL_MAX_AGE_MS`: Codex max event age for in-flight signals (default `max(CONSENSUS_CODEX_INFLIGHT_TIMEOUT_MS, 2500)`).
 - `CONSENSUS_PROCESS_CACHE_MS`: process cache TTL in ms for full scans (default `1000`).
 - `CONSENSUS_PROCESS_CACHE_FAST_MS`: process cache TTL in ms for fast scans (default `500`).
 - `CONSENSUS_SESSION_CACHE_MS`: Codex session list cache TTL in ms for full scans (default `1000`).

dev/dev.mjs

@@ -32,6 +32,15 @@ if (!fs.existsSync(vitePath)) {
 }
 
 const children = [];
+const OUTPUT_CAP = 32_768;
+
+const shutdown = (code = 0) => {
+  for (const child of children) {
+    if (child.killed) continue;
+    child.kill("SIGTERM");
+  }
+  process.exit(code);
+};
 
 const spawnChild = (label, args) => {
   const child = spawn(process.execPath, args, { stdio: "inherit" });
@@ -52,7 +61,6 @@ const tsc = spawnChild("tsc", [
   "--pretty",
   "false",
 ]);
-const server = spawnChild("server", [tsxPath, "watch", "src/server.ts"]);
 
 const parsePort = (value) => {
   const num = Number(value);
@@ -66,7 +74,81 @@ const basePort =
   parsePort(process.env.VITE_PORT) ??
   5173;
 
-const startVite = async () => {
+const baseServerPort = parsePort(process.env.CONSENSUS_PORT) ?? 8787;
+
+const startServer = async () => {
+  const maxAttempts = 6;
+
+  for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
+    const port = baseServerPort + attempt;
+    process.stderr.write(`[dev] starting server on port ${port}\n`);
+
+    const child = spawn(
+      process.execPath,
+      [tsxPath, "watch", "src/server.ts"],
+      {
+        env: { ...process.env, CONSENSUS_PORT: String(port) },
+        stdio: ["ignore", "pipe", "pipe"],
+      }
+    );
+
+    let output = "";
+    let running = false;
+    let settled = false;
+    const onData = (chunk) => {
+      const text = chunk.toString();
+      if (!settled) {
+        output = (output + text).slice(-OUTPUT_CAP);
+      }
+      process.stdout.write(text);
+    };
+    child.stdout?.on("data", onData);
+    child.stderr?.on("data", onData);
+
+    child.on("exit", (code) => {
+      if (running) {
+        process.stderr.write(`[dev] server exited with ${code ?? 0}\n`);
+        shutdown(code ?? 1);
+      }
+    });
+
+    const result = await new Promise((resolve) => {
+      const timer = setTimeout(() => {
+        if (settled) return;
+        settled = true;
+        running = true;
+        children.push(child);
+        resolve({ status: "running" });
+      }, 1500);
+
+      child.on("exit", (code) => {
+        if (settled) return;
+        settled = true;
+        clearTimeout(timer);
+        const conflict = /EADDRINUSE|address already in use|port .* in use/i.test(
+          output
+        );
+        resolve({ status: conflict ? "conflict" : "exit", code });
+      });
+    });
+
+    if (result.status === "running") return port;
+    if (result.status === "conflict") {
+      process.stderr.write(`[dev] port ${port} in use, trying next\n`);
+      continue;
+    }
+
+    process.stderr.write("[dev] server exited unexpectedly\n");
+    shutdown(1);
+    return;
+  }
+
+  process.stderr.write("[dev] failed to find an open port for server\n");
+  shutdown(1);
+  return null;
+};
+
+const startVite = async (serverPort) => {
   const maxAttempts = 6;
 
   for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
@@ -78,22 +160,28 @@ const startVite = async () => {
       [vitePath, "--port", String(port), "--host", "127.0.0.1", "--strictPort"],
       {
         cwd: path.join(root, "public"),
-        env: { ...process.env, CONSENSUS_UI_PORT: String(port) },
+        env: {
+          ...process.env,
+          CONSENSUS_UI_PORT: String(port),
+          CONSENSUS_PORT: String(serverPort),
+        },
         stdio: ["ignore", "pipe", "pipe"],
       }
     );
 
     let output = "";
+    let settled = false;
     const onData = (chunk) => {
       const text = chunk.toString();
-      output += text;
+      if (!settled) {
+        output = (output + text).slice(-OUTPUT_CAP);
+      }
       process.stdout.write(text);
     };
     child.stdout?.on("data", onData);
     child.stderr?.on("data", onData);
 
     const result = await new Promise((resolve) => {
-      let settled = false;
       const timer = setTimeout(() => {
         if (settled) return;
         settled = true;
@@ -127,18 +215,15 @@ const startVite = async () => {
   shutdown(1);
 };
 
-void startVite();
-
-const shutdown = (code = 0) => {
-  for (const child of children) {
-    if (child.killed) continue;
-    child.kill("SIGTERM");
-  }
-  process.exit(code);
+const main = async () => {
+  const serverPort = await startServer();
+  if (!serverPort) return;
+  await startVite(serverPort);
 };
 
+void main();
+
 tsc.on("exit", (code) => shutdown(code ?? 0));
-server.on("exit", (code) => shutdown(code ?? 0));
 
 process.on("SIGINT", () => shutdown(0));
 process.on("SIGTERM", () => shutdown(0));

dev/tmux-start.sh

@@ -2,4 +2,13 @@
 set -euo pipefail
 
 SCOUT_DIR="$(pwd -P)"
-exec env SCOUT_DIR="$SCOUT_DIR" tmux -f ./dev/tmux.conf new -A -s scout
+
+if tmux has-session -t scout 2>/dev/null; then
+  tmux set-environment -g SCOUT_DIR "$SCOUT_DIR"
+  tmux set-environment -g PATH "$PATH"
+  tmux set-environment -t scout SCOUT_DIR "$SCOUT_DIR"
+  tmux set-environment -t scout PATH "$PATH"
+  exec tmux -f ./dev/tmux.conf attach -t scout
+fi
+
+exec env SCOUT_DIR="$SCOUT_DIR" PATH="$PATH" tmux -f ./dev/tmux.conf new -s scout

dev/tmux.conf

@@ -1,8 +1,11 @@
+# Keep PATH in sync between shell and tmux.
+set -ga update-environment "PATH"
+
 # Show useful window names automatically: "<command>:<directory>"
 # This makes Ctrl+b w show both the agent (command) and the worktree (dir).
 setw -g automatic-rename on
 setw -g automatic-rename-format '#{pane_current_command}:#{b:pane_current_path}'
 
-# New windows should always start in the default worktree ("scout directory" in the demo).
-# We set SCOUT_DIR in the session start command (see next section).
-bind c new-window -c "#{env:SCOUT_DIR}"
+# New windows should start in the default worktree when SCOUT_DIR is set.
+# Fallback to the current pane path if SCOUT_DIR is missing.
+bind c new-window -c "#{?env:SCOUT_DIR,#{env:SCOUT_DIR},#{pane_current_path}}"