1# Shared project commands
2
3`tools/project-tools/` owns the VM's `jj`, `pnpm`, and `project` wrappers.
4The config repo's `users/clover/agents/vm` entries link here; installed wrappers
5use `~/.local/share/project-tools/current`. Keep native executables adjacent to
6the public commands as `jj-real` and `pnpm-real`.
7
8## Defaults
9
10- `project read rg ARGS...` searches on Linux beside ZFS and streams normal
11 ripgrep output and exit status. `project read rg --files` lists files.
12 `.gitignore` works in Jujutsu checkouts without a visible `.git` directory;
13 `--require-git` restores ripgrep's usual Git-only policy. Search preprocessors
14 are disabled. `project read sha256sum FILE...` hashes named shared files
15 without transferring their contents over NFS. Run these from the project
16 share; read requests have read-only mounts, no network, and no repository lock.
17- Noninteractive `jj` operations in `/Volumes/Code/project` run on Snow Globe.
18 This includes workspace creation, snapshots, rebases, restores, and reads.
19 The guest's resolved configuration, identity, ignores, and paths travel with
20 the request. Both sides must run exactly the same Jujutsu build.
21- Editors, external diff tools, signing, network Git commands, and local-disk
22 repositories retain native execution. Explicit file paths for a shared
23 repository cannot escape the project share, including through symlinks.
24- `pnpm install`, `i`, `add`, `update`, and `remove` resolve, fetch, and import
25 packages on Linux, including changed manifests and lockfiles. The runtime must
26 match the project's exact pin: 10.32.1, 11.10.0, or 12.14.0. Directory selection,
27 workspace filters, frozen/offline installs, and lockfile-only updates retain
28 their normal meanings. `--ignore-scripts` skips the Mac hook phase.
29- Activation links Vite+'s installed managed pnpm launchers to this same wrapper,
30 preserving each original as the adjacent `pnpm-real`. This covers the
31 project-local JavaScript `vp` launcher, which otherwise bypasses pnpm on PATH.
32 Vite+ keeps package-manager detection and argument handling; its notifier
33 setting does not prevent offloading. Explicit `VP_PACKAGE_MANAGER` overrides
34 retain native execution. Reactivate after Vite+ downloads another pnpm version.
35- `pnpm run`, `exec`, and script shortcuts execute natively. Their implicit
36 dependency installs use Linux too; dependency verification opt-outs and custom
37 verification policies are preserved. Inherited npm client metadata and Node
38 heap limits retain offloading; `NODE_ENV=production` reaches the pinned pnpm
39 unchanged on both hosts. Unsupported options, runtime versions, other custom
40 install environment settings, and unreviewed pnpmfile hooks retain native
41 execution. Environment fallbacks name the variables without exposing values.
42 A failed remote operation is reported once.
43- `project worktree PROJECT NAME` uses those same `jj` and `pnpm` commands.
44 `--no-install` skips dependencies. The checkout path is printed only after
45 installation and Mac hooks succeed, including with `--warm-from`.
46 New workspaces wait for NFS visibility.
47
48Linux installs with lifecycle scripts disabled, selects Darwin
49ARM64 packages, validates the guest's Node version, and requires ZFS reflinks.
50Knife's reviewed `.pnpmfile.cjs` runs in the Linux sandbox: its peer-range change
51and Windows-only architecture selection behave identically on Linux and Darwin.
52The backend checks its exact SHA-256 before offloading. Changed hooks, other
53pnpmfiles, and config dependencies retain native installation.
54Pending dependency and workspace hooks finish natively on the Mac. pnpm 10/11
55use `pnpm --recursive --include-workspace-root rebuild --pending`. pnpm 12 uses
56a frozen install with the original filters and dependency groups, preserving
57prepared patches. Its repeat-install shortcut and build-cache reads/writes are
58disabled for this phase: restoring cached generated files over NFS can take
59longer than running their hooks. Ordinary native installs retain their cache policy.
60The wrapper announces the Mac handoff.
61It also announces storage-host installation and package-store lock contention.
62The two phases share policy and registry metadata in the project's `.pnpm-cache`
63directory, outside its workspaces. This avoids repeating network verification on
64the Mac. The handoff waits until NFS exposes the server's new modules manifest.
65Repository configuration is not temporarily rewritten; manifests and
66lockfiles change only as requested by the dependency command.
67
68The root-owned `profiles.json` binds manifests, lockfiles, workspace settings,
69npmrc, pnpmfiles, patches, and local tarballs. Matching alphaXiv pnpm 10.32.1 and
70Knife pnpm 11.10.0 profiles on Node 26 can additionally restore reviewed Darwin
71side effects on Linux after SHA-512 verification using `project-pnpm/`'s patches.
72A profile miss still installs on Linux; it leaves native builds pending for the
73Mac instead of replaying an older build. Validation does not snapshot the working
74copy. Ordinary Linux side-effect cache reuse is disabled.
75
76## Locking and storage
77
78The Mac wrapper uses `flock`; Linux uses a POSIX record lock on the same
79`.jj/repo/command.lock` inode. macOS NFS translates `flock` into that server lock.
80The executing Jujutsu process inherits the descriptor. The forwarding guest
81never holds a second lock. Ctrl-C cancels a waiter; never unlink the lock inode.
82A command lock does not reserve revisions across a sequence of commands.
83
84The Code export is one ZFS dataset. `project worktree --warm-from DONOR` uses
85per-file FICLONE, preserving independent inodes and source timestamps, excluding
86root repository metadata and Mac sidecars. It restores the requested revision
87and retains ignored build outputs. The guest holds the donor's Cargo locks;
88editors and other writers must also be idle. No byte-copy fallback is allowed.
89
90The dedicated SSH key forces the unprivileged uid-3000 Bubblewrap launcher.
91Jujutsu requests have no network. Dependency requests get network access with
92DNS and the host CA bundle; lifecycle scripts remain disabled on Linux. Neither path
93exposes host credentials, forwarding, or snapshot privileges.
94The sandbox binds projects and package stores at their guest paths as well as
95host paths. Identical store paths allow repeated installs without repaving
96`node_modules`. Artifacts remain under Code. pnpm 11 uses a guest-local SQLite
97index under `~/Library/Caches/project/PROJECT/v11`; pnpm 12 uses the shared
98`/Volumes/Code/cache/pnpm12` file index. Both phases use workspace-local virtual
99stores so platform-specific build outputs belong to that workspace.
100
101## Updating the existing deployment
102
103Upload `project-store-install.py`, `project-store.py`, `project-store-test.py`,
104`project-tools/`, and `project-pnpm/` together to a private staging directory on
105Snow Globe. Run the installer there as root:
106
107```sh
108python3 project-store-install.py stage RELEASE
109```
110
111It verifies pinned upstream downloads, applies the cache patches, preserves the
112reviewed profiles, and builds a versioned runtime under
113`/opt/studio-project-store/releases/`. Existing `project-store-*` Nix GC roots
114pin Python, Node, Bubblewrap, and coreutils. `current` is unchanged by staging.
115Test the staged `ssh-command` as `clo` with disposable projects and a temporary
116restricted key. Do not run Jujutsu as root against shared repositories.
117
118Upload `project-tools/` to a guest staging directory and run its installer as
119`admin`. After the staged host and guest tests pass:
120
121```sh
122# Snow Globe
123python3 project-store-install.py activate RELEASE
124# VM, using tools/project-tools/install.py
125python3 install.py stage RELEASE
126python3 install.py activate RELEASE
127```
128
129Guest activation saves the prior tools under `project-tools/previous`, preserves
130the native executables, and changes symlinks atomically. Subsequent updates switch
131the complete wrapper directory through `current`. Keep the previous host release
132and guest tools for rollback. Roll back both `current` links; do not remove
133workspaces, locks, stores, or caches.
134
135## Verification
136
137```sh
138python3 tools/project-store-test.py
139python3 tools/project-tools/test.py
140python3 tools/project-tools/jj-test.py
141```
142
143Run `project-tools/platform-test.py` inside the development VM against the
144normal endpoint. Its disposable project checks Darwin-only optional dependencies
145immediately after the Linux import, before Mac build scripts can repair them.
146`project-tools/finish-test.py` checks native hooks, denied scripts, patch inode
147preservation, repeated installs, filtered dependency groups, and nested mutations.
148
149The backend tests create disposable repositories. `PROJECT_STORE_REAL_REFLINK=1`
150with `TMPDIR` on disposable Linux/ZFS storage exercises real clone independence,
151timestamps, donor edits, path rejection, and failure cleanup. The config repo's
152`vm/test.py` covers overlay preparation, worktrees, and shared memory links.
153`project-pnpm/restore-test.cjs` checks cache integrity and path guards on Linux/ZFS.
154
155On 2026-10-22, the endpoint created an 801-file workspace through the VM in 2.9
156seconds and rebased a two-commit fixture in 1.6 seconds. Cross-client lock
157contention and cancellation passed. Disposable pnpm 10, 11, and 12 projects
158installed changed dependencies on Linux and ran Darwin dependency and workspace
159hooks on the Mac. Frozen-lockfile failures, lockfile-only updates, filtered
160add/update/remove commands, ignored scripts, and native esbuild execution passed.
161A pnpm 12 filtered update took 12.4 seconds, including 1.4 seconds on Linux;
162a subsequent script ran in 1.2 seconds without reinstalling. Nested pnpm scripts
163and dependency-verification opt-outs passed. An existing global virtual store
164migrated successfully. After activation, ordinary commands against the real
165shared store passed: a fresh install took 70.5 seconds, a filtered add took 32.9
166seconds, and a script took 2.5 seconds. Native rebuilding left all 1,048 sampled
167lodash files' inodes and mtimes unchanged. These are fixture measurements with other agents
168active, not latency bounds or full-project build measurements.
169
170Vite+ 1.3.0 was also verified through the normal endpoint with pnpm 12.14.0
171and Knife's reviewed `.pnpmfile.cjs`: package imports ran on Linux/ZFS, while
172the workspace postinstall and native esbuild ran on Darwin ARM64. Frozen/offline
173installs, add/remove, lockfile-only, ignored scripts, script execution, and frozen
174lockfile failure propagation passed. Editing the hook selected Mac installation.
175The host and VM tools were activated as `20261022-vp-preview-2`.
176
177pnpm 12 needs `supportedArchitectures: [darwin-arm64]` for host-side package
178selection; the older `config.os`/`config.cpu` overrides do not select its optional
179dependencies. This is passed as a command setting without editing the workspace.
180The platform regression fails against the old endpoint and passes with the fix.
181The full Knife workspace imported 75 missing Darwin packages, and its Vite+,
182Vite, Vitest, Rolldown, and linter bindings loaded successfully before rebuilding.
183With Mac build-cache writes disabled, a normal frozen install in `knife-merge`
184completed successfully in 278 seconds, including the mobile dependency's native
185asset downloads and the root prepare hook. The host and guest were activated as
186`20261023-vp-platform-3`. Linux side-effect cache reads and writes are explicitly
187disabled with pnpm 12's supported boolean settings.
188After installation, `pnpm exec vp --version`, the TypeScript compiler, and the
189Vitest CLI started successfully. The node-pty addon loaded, and sharp encoded a
190small PNG using its Darwin native module.
191
192A follow-up trace in `knife-disk` found the Mac phase copying patched packages
193and restoring cached mobile assets over NFS. The frozen-install completion
194described above disables those cache transfers. Five disposable VM cases cover
195native and workspace hooks, denied scripts, patch preservation, filtered
196production/development installs, nested adds, repeated installs, and inherited
197agent environments. A deployed
198frozen install in `knife-disk` completed in 43.5 seconds with its native assets
199already present. Vite+, TypeScript, and the node-pty native binding loaded.
200The real project-local `vp i --frozen-lockfile` then completed in 28.8 seconds
201through Vite+'s managed pnpm launcher and the shared wrapper. Exporting
202`VP_PNPM_MANAGER=no` alone did not route that JavaScript launcher through PATH;
203the obsolete VM environment setting was removed.
204
205The host and guest were activated as `20261023-vp-env-4`. Staged pnpm 10, 11,
206and 12 installs with `NODE_ENV=production` preserved their native dependency
207selection; production mode is forwarded as environment, without inventing a
208`--prod` flag. The VM fixture also checked that Mac hooks receive that value
209while npm client metadata and Node memory limits remain compatible with offloading.
210A real `vp i --frozen-lockfile` with those inherited metadata and memory settings
211completed in 23.2 seconds on `knife-disk`, including Mac hooks with native assets
212already present. Vite+'s tools, TypeScript, and the node-pty Darwin binding loaded.
213
214Release `20261023-tools-6` adds the read-only storage tools and waits for NFS
215visibility before opening warm workspaces. A VM request listed 2,001 fixture
216files in 1.93 seconds and hashed a 330 MiB fixture in 0.95 seconds. Staged
217checks verified read-only mounts, network isolation, ignore rules, normal
218search exit statuses, hash equality, and preprocessor rejection. A real warm
219checkout installed a changed dependency and its Darwin hook in 7.81 seconds;
220the donor and an explicit `--no-install` copy retained the original dependency.