clover's git

the-knife

1	# Schema construction costs
2
3	This diagnostic measures the current implementation before proposing changes. It leaves library sources unchanged.
4	It measures elapsed construction time in ordinary fresh Node processes, then uses separate V8 CPU-profiled processes
5	to locate the work. Profile shares are estimates of sampled execution, not directly timed phases or predicted savings.
6
7	Run from the repository root:
8
9	```sh
10	node packages/effect/runtimeperf/suites/schema-construction/run.mts tmp/schema-construction 24 1000
11	```
12
13	Arguments are a new output directory, rounds, root count, and an optional comma-separated list of case names.
14	The output directory's parent must exist. Existing output directories are rejected. The source tree must be clean.
15	Every raw observation is saved in `results.json`; raw `.cpuprofile` files can be opened in a CPU profile viewer.
16	Copies of the measured harness files are retained in `harness/` to preserve uncommitted analysis code.
17	The report records the commit, source-tree object, harness and lockfile hashes, Node/V8 versions, machine and settings.
18	Interrupted or failed runs retain `complete: false` and must not be reported as complete measurements.
19
20	## Scope
21
22	Each worker imports Schema, prepares fixture keys and validation inputs, and preallocates a root array before timing.
23	It then builds 1,000 new root schemas and retains them until the process exits. The timer includes field-map creation,
24	child schemas, checks, transformations, options and all eager internal work triggered by the declaration.
25	There is no explicit warmup. Schema's own import-time initialization has already run and can warm shared constructors
26	and caches. Node startup, TypeScript loading, module imports, decoder preparation and decoding are excluded.
27
28	Fixtures cover literals; structs with 2, 32 and 256 fields; nested structs; arrays with checks, optional keys and literals;
29	new string-to-number transformations; and unions of eight newly constructed tagged members. Root counts are not counts
30	of every intermediate schema. Shared primitive schemas such as `Schema.String` are imported constants. The `struct-N`
31	fixtures build field maps with `Object.fromEntries` during timing. The additional `struct-ready-N` fixtures reuse a field
32	map prepared before timing, to isolate `Schema.Struct(fields)` with ready inputs. Primitive schemas, field-name strings
33	and sample values are prepared beforehand in both cases. Dynamically generating fields is a distinct caller cost; it
34	does not model a hand-written object literal's cost.
35
36	Each fixture is validated in a separate process before measurement, including valid decode, type-side `.make`, and invalid
37	input rejection. Measured workers check root count, object kind and distinctness after timing. No prior validation warms
38	the measured worker. This protocol makes no claim about complete HttpApi/Confect startup, memory usage or warmed throughput.
39
40	## Measurement and attribution
41
42	Each round runs ordinary Node, profiling at a requested 100 microsecond interval, and profiling at 500 microseconds.
43	Workers run serially. The six possible orders rotate by round; fixture order also rotates. The default 24 rounds
44	balance all six orders. Do not run other benchmarks, builds or tests alongside the measurement.
45
46	Elapsed times are medians across independent processes, with median absolute deviation. Profiler perturbation is measured
47	through paired ratios against the ordinary process in the same round, using the existing deterministic bootstrap with
48	10,000 resamples. This is a check on measurement bias, not a comparison of optimized code.
49
50	Each CPU sample is assigned to one category:
51
52	| Category | Included work |
53	| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
54	| `type-projection` | `toType` and its recursive traversal, clones and memoization, during synchronous maker preparation |
55	| `ast` | Initial AST construction and changes made by checks, annotations, optional keys and transformations |
56	| `schema-object` | Central schema constructor after subtracting identifiable child phases: function object, prototype, options, property and closure setup |
57	| `make-sync` | Identifiable synchronous maker factory frames outside the type projection |
58	| `make-effect` | Identifiable effectful maker factory frames outside the Option adapter |
59	| `make-option` | Option adapter factory, including its separately allocated effectful maker |
60	| `schema-api` | Other Schema API work, including composition wrappers and check factories outside AST constructors |
61	| `fixture` | Caller-attributed samples: field maps, member lists, the loop, and library work inlined into those frames |
62	| `gc` | Garbage collector samples inside the construction window |
63	| `runtime` | Other runtime samples without an attributable Schema stack |
64
65	Child work is never added again to its parent. V8 can omit an inlined `SchemaParser.make` frame. In these fixtures,
66	the only eager AST/Function subtree beneath central `make` is `toType`; this path is classified as projection even when
67	the parser wrapper is absent. The fixtures do not use options with getters that could invoke arbitrary AST work.
68	Completely inlined work that cannot be separated remains with the caller. The `fixture` category is therefore an
69	unresolved caller category, not a measurement of pure benchmark overhead or field-map creation. A ready-field fixture
70	whose body only calls `Schema.Struct(fields)` still receives caller-attributed samples. Small or zero factory shares do not prove
71	that their allocations are free; allocation effects can appear in central construction or GC.
72
73	Profiles are delimited by the first and last samples containing the worker's `constructBatch` frame. Samples outside
74	that span are excluded, including inspector setup. Interior GC/runtime samples are retained. Inspector timestamps and
75	`hrtime` are not directly compared because their origins can differ. Missing boundary samples, especially GC before the
76	first or after the last observed construction frame, are a limitation. The report keeps sample counts, sampled span
77	coverage and counts of processes with fewer than ten samples. A profile with no construction samples remains recorded
78	and is excluded from share estimates. One sample has zero measured span coverage and is marked sparse.
79
80	Shares are unweighted sample proportions within each process, averaged across processes. Their 95% intervals bootstrap
81	whole processes, not individual correlated samples. Categories sum to 100% for the point estimates; interval endpoints
82	need not sum to 100%. These intervals cover sampling variability, not profiler bias or attribution errors. Compare the
83	two sampling frequencies and their overhead before interpreting the shares. Do not convert them into exact milliseconds
84	or predicted speedups. Short fixtures may support a reliable elapsed time without a useful internal breakdown.
85
86	Top leaf functions and source-position ticks are retained as secondary diagnostics. Source-position ticks are not an
87	additional additive cost breakdown and may accumulate at a function's entry after optimization.
88
89	## Validation
90
91	```sh
92	node --test packages/effect/runtimeperf/test/*.test.mts
93	pnpm exec dprint check --includes-override 'packages/effect/runtimeperf/suites/schema-construction/*.mts' 'packages/effect/runtimeperf/test/schema-construction.test.mts'
94	```
95
96	The attribution tests cover non-overlapping phases, inlined maker frames, Option's nested effect maker, profile boundaries,
97	interior garbage collection, missing samples and deterministic confidence intervals.