clover's git

the-knife

1	---
2	title: Scaling
3	description: Reads scale like a CDN and writes scale like git. The shape, the per-repository limits, the measured numbers, and a worked example of pushing something large and swapping the hasher.
4	---
5
6	The examples use `Authentication` from
7	[Getting Started](/git/getting-started): the application's request middleware.
8
9	Every repository is its own Durable Object, so the service's capacity
10	is the sum of its repositories. Within one repository the two
11	directions scale differently, and the difference is git's, not the
12	host's.
13
14	**Reads scale out.** A full clone streams a precomputed bundle from
15	the blob store through the Worker. The repository's Durable Object
16	plans the clone and never touches a pack byte, so concurrent clones
17	scale with Workers and object storage. Incremental fetches are small
18	by construction and run against a commit graph in SQLite.
19
20	**Writes serialize on the ref.** A push is a compare-and-swap on a
21	ref, and git's own semantics say two pushes to one branch cannot both
22	win. One Durable Object per repository is that serialization point.
23	Receiving, hashing, and staging the pack happen in the Worker and fan
24	out to the [hasher](/git/blocks/hasher).
25
26	| Dimension | Per repository | Bound by |
27	| --- | --- | --- |
28	| Clone bandwidth | edge line rate | bundle bytes live in the blob store |
29	| Clone rate | thousands per second | the advertisement is cached, the Durable Object is called once per push |
30	| Incremental fetch | milliseconds for recent tips | the commit graph in SQLite |
31	| Push rate | 5–20 per second | a ref is a serialization point in git's own semantics |
32	| Repository size | unbounded | packs live in the blob store |
33	| Push size | unbounded | the body streams to the blob store as it arrives |
34
35	## Where the limits are
36
37	**Push rate to one branch** is inherent. Across branches and across
38	repositories there is no shared limit.
39
40	**The registry** is one object handling creates, deletes, and name
41	lookups for the whole service. Lookups are cached at the Worker for a
42	minute, and [D1](/git/blocks/registry) replicates them.
43
44	**First-byte latency** for a repository's Durable Object is one
45	datacenter's distance. Cached advertisements hide most of it, and
46	clone bytes come from the blob store rather than the object.
47
48	**Very large histories.** Computing which objects a fetch needs walks
49	the commit graph. For repositories with tens of millions of objects
50	that walk wants reachability bitmaps, which the service does not have
51	yet.
52
53	## A large push, and the hasher
54
55	Push a repository with a 40 MiB history to the host from
56	[Getting Started](/git/getting-started) and
57	read the timing the repository reports:
58
59	```sh
60	curl -s -u "x:$GIT_SECRET" "$HOST/api/v1/repos/acme/big" | jq .lastPush
61	```
62
63	Ingest dominates. Every object was inflated and hashed on the one
64	thread that received the body. Swap the hasher and deploy:
65
66	```diff lang="typescript"
67	+import * as GitHasher from "alchemy/Git/Hasher";
68
69	const GitLive = Git.ApiLive.pipe(
70	Layer.provide(Git.ApiHandlersLive),
71	Layer.provide(Authentication.layer),
72	Layer.provide(Git.ReposDurableObject),
73	Layer.provide(Git.RegistryDurableObject),
74	- Layer.provide(Git.HasherInline),
75	+ Layer.provide(GitHasher.HasherWorkerLoader()),
76	Layer.provide(Git.BlobStoreR2(GitObjects)),
77	);
78	```
79
80	Four dynamically loaded Workers now hash four 4 MiB chunks at once,
81	each in its own isolate. Push again, read `lastPush` again. Nothing
82	else about the host changed.
83
84	One client, the same edge, a fresh repository on each host. The
85	delta-heavy pack is 40 MiB and 15.6k objects, the whole-blob pack is
86	44 MiB:
87
88	| | 40 MiB deltas | 44 MiB blobs | incremental |
89	| --- | --- | --- | --- |
90	| GitHub | 3.3 s | 4.6 s | 1.2 s |
91	| `HasherInline` | 9.9–13.5 s | 8.8 s | 0.2–0.4 s |
92	| `HasherWorkerLoader` | 5.9 s best, 6.5 s median | 4.8 s best, 6.4 s median | 0.2–0.4 s |
93	| `HasherLambda` | 6.8–7.9 s | 6.3–6.7 s | 0.3–0.8 s |
94
95	Incremental pushes are the same on every hasher, so the choice only
96	matters for pushes of tens of megabytes. `HasherLambda` runs every
97	chunk at once on a 3 GB function and needs an AWS account in the
98	stack. [Hasher](/git/blocks/hasher) has the push path and each
99	implementation. [Bytes in S3, hashing on Lambda](/git/recipes/cloudflare-aws)
100	is the two-cloud stack.
101
102	## Clones
103
104	The same 36 MiB, 15.6k-object repository, one client, the same edge:
105
106	| Path | Throughput |
107	| --- | --- |
108	| a real `git clone` from a bundle | 32.7 MiB/s typical, 80 MiB/s best (0.69 s end to end) |
109	| raw bundle, four concurrent streams | 93 MiB/s aggregate |
110	| dynamic fetch, time to first byte | 0.43–0.56 s |
111	| dynamic fetch, transfer | 7–10 MiB/s |
112
113	The bundle path is the client's line rate. Compaction is what keeps it
114	that way: on a real repository of 13,699 objects, cloning back took
115	19.5 s before compaction and 3.4 s after.
116	[Repository](/git/blocks/repositories) has the clone path.
117
118	The largest real push measured is the alchemy monorepo: 44,051 objects
119	in a 67 MiB thin pack, cloned back byte-identical under
120	`git fsck --strict`.