authorgravatar for julius0216@outlook.comJulius Marminge <julius0216@outlook.com> 2026-02-24 11:35:33-08:00
committergravatar for jmarminge@gmail.comJulius Marminge <jmarminge@gmail.com> 2026-02-26 13:20:36-08:00
log877ab769345a13e90509c564b62ef3f50f480607
tree97e650b2b586f36fe6c9dda15d4efecf2d86e4c1
parent032c0e584943be48e197198ea1daaec8df9236bf

terminal plan v1

1 files changed, 154 insertions(+), 0 deletions(-)

.plans/terminal-runtime-state-plan.md created+154
......@@ -0,0 +1,154 @@
1# Terminal Runtime And State Plan
2
3## Summary
4
5Terminals appear to "hang" because PTY processes are exiting immediately at startup in the current runtime.
6
7Repro in this branch:
8
9- `node` + `node-pty`: interactive shell stays alive and accepts writes.
10- `bun` + `node-pty`: shell exits immediately (`exitCode: 0`, `signal: 1`), so the UI opens a terminal pane but no live process remains.
11
12This means the current breakage is primarily a runtime compatibility issue, plus missing reconciliation between desired terminal state and observed process state.
13
14## Root Cause (Current)
15
161. Server dev runtime uses Bun (`apps/server/package.json` `dev` script).
172. Terminal PTY implementation depends on `node-pty`.
183. Under Bun, spawned PTY shells exit immediately.
194. Frontend still mounts terminals, so UX looks like startup hang/blank terminal.
20
21## Design Goals
22
231. Do not store full terminal output in SQLite.
242. Preserve terminal topology and intent across reconnects/restarts:
25 - open/split/new/close
26 - active terminal/group
27 - cwd/env profile
283. Keep runtime truth accurate even when processes change outside the app.
294. Support deterministic recovery and idempotent command handling.
30
31## Proposed Architecture
32
33### 1) Runtime Boundary
34
35- Run terminal PTY host on Node runtime only.
36- If main server remains Bun, isolate PTY behind a Node sidecar process (JSON-RPC/IPC).
37- Treat PTY host as volatile runtime, not source of durable truth.
38
39### 2) Event-Sourced In SQLite (Control Plane Only)
40
41Persist only control/state events, not output payloads.
42
43Recommended event families:
44
45- `terminal.layout.updated`
46 - terminal ids, split groups, active terminal id/group id
47- `terminal.open.requested`
48 - threadId, terminalId, cwd, envHash/envProfileId, cols, rows
49- `terminal.close.requested`
50 - terminalId, deleteHistory
51- `terminal.resize.requested`
52 - cols, rows
53- `terminal.clear.requested`
54- `terminal.input.sent`
55 - input metadata only (`byteLength`, optional hash, optional commandBoundary flag), no raw payload
56- `terminal.runtime.started`
57 - pid, startedAt, runtime instance id
58- `terminal.runtime.exited`
59 - pid, exitCode, exitSignal, observedAt, reason (`normal|killed|external|crash|unknown`)
60- `terminal.runtime.error`
61 - normalized error code/message
62- `terminal.output.segment.recorded`
63 - segment file id + byte offsets only
64
65Use idempotency keys/command ids for each client command so reconnect retries are safe.
66
67### 3) Output Storage In Log Files (Data Plane)
68
69- Store raw output only in append-only files (per thread+terminal).
70- Rotate into segments (size-based and/or time-based).
71- Keep lightweight SQLite index rows:
72 - `segment_id`, `thread_id`, `terminal_id`, `start_offset`, `end_offset`, `created_at`, checksum.
73- Optional retention policy (max bytes / days / segment count per terminal).
74
75This keeps SQLite small while preserving replay capability.
76
77### 4) Reconciliation Loop (Runtime Truth)
78
79Do not treat client dispatch as sole truth.
80
81Maintain dual state:
82
83- Desired state: from event stream/projection.
84- Observed runtime state: from PTY host watchers (process exit, health checks, poll).
85
86On mismatch, emit runtime observation events:
87
88- process missing but desired running -> `terminal.runtime.exited` with reason `external|unknown`
89- process exists but projection says closed -> terminate process and emit correction event
90
91This makes out-of-band kills/restarts explicit and recoverable.
92
93### 5) Startup/Recovery Flow
94
951. Rebuild terminal projection from SQLite control events.
962. Load terminal layout/topology immediately for UI.
973. Reconcile each desired running terminal with runtime:
98 - if process alive: attach stream + tail recent output from logs
99 - if process dead: mark exited; auto-restart only if policy says so
1004. Resume incremental output from last persisted offset.
101
102## Data Model Additions
103
104Add dedicated tables (or equivalent repos):
105
106- `terminal_control_events` (or reuse orchestration_events with terminal aggregate kind)
107- `terminal_output_segments` (index only, no raw output)
108- `terminal_runtime_snapshot` (latest observed runtime state cache)
109
110Avoid embedding output blobs in projection snapshots.
111
112## Implementation Plan
113
114### Phase 0: Stop the bleeding
115
1161. Run PTY on Node runtime in all dev/prod paths (or ship Node sidecar now).
1172. Add startup diagnostics: runtime, PTY spawn result, immediate-exit telemetry.
1183. Surface clear UI error when PTY host unavailable.
119
120### Phase 1: Durable control events
121
1221. Introduce terminal aggregate + commands/events.
1232. Persist layout and terminal lifecycle intent events.
1243. Build projection for terminal UI state from events.
125
126### Phase 2: File-backed output + index
127
1281. Implement segmented append-only output logger.
1292. Persist only segment metadata/offsets in SQLite.
1303. Update terminal history restore to tail from segments (bounded bytes/lines).
131
132### Phase 3: Runtime reconciliation
133
1341. Add process observer + periodic verifier.
1352. Emit runtime observation events on drift.
1363. Add recovery tests for out-of-band process termination.
137
138### Phase 4: Hardening
139
1401. Backpressure and truncation policies for high-volume output.
1412. Crash-safe fsync/flush strategy for segment writes.
1423. Retention and compaction jobs.
143
144## Testing Strategy
145
146Backend tests should cover:
147
1481. Rebuild projection from control events (open/split/close/resize/active terminal).
1492. Runtime reconciliation when process is killed externally.
1503. Recovery after restart with existing segment files.
1514. Idempotent command handling on duplicate dispatches.
1525. Segment index consistency and retention behavior.
153
154Use real runtime-layer tests for terminal adapter behavior where possible; only mock external process/service boundaries.