| 1 | # RFC: Git Building Blocks |
| 2 | |
| 3 | ## Current implementation |
| 4 | |
| 5 | Git contributes native Effect route and storage layers. Applications own their |
| 6 | API schema and server. The public `Git.ApiLive` is composed beside application |
| 7 | routes; `Git.InternalApiLive` is mounted separately from user authentication. |
| 8 | |
| 9 | ```ts |
| 10 | const PublicRoutes = Layer.mergeAll(AppApiLive, Git.ApiLive).pipe( |
| 11 | Layer.provide(Authentication.layer), |
| 12 | ); |
| 13 | const Routes = Layer.mergeAll(PublicRoutes, Git.InternalApiLive).pipe( |
| 14 | Layer.provide(Git.ApiHandlersLive), |
| 15 | // repository, registry, hasher, blob, and platform layers |
| 16 | ); |
| 17 | const fetch = yield* HttpRouter.toHttpEffect(Routes); |
| 18 | ``` |
| 19 | |
| 20 | Applications can also build every group against their own `AppApi`, including |
| 21 | native `HttpApiMiddleware` that provides a typed user. Raw protocol handlers use |
| 22 | `alchemy/Git/Http` to decode streaming requests and encode Git responses, and |
| 23 | `Git.Engine` for repository operations. `preparePush` stages and validates objects; |
| 24 | application effects authorize refs and inspect content; `commitPush` advances refs. |
| 25 | Ref writes and merges have scoped preparation methods too. No policy callback |
| 26 | service is installed. See [DESIGN.md §8](./DESIGN.md#8-auth-model-nothing-inside-the-engine) |
| 27 | and [Engine operations](https://alchemy.run/git/blocks/engine). |
| 28 | |
| 29 | ## Historical proposal |
| 30 | |
| 31 | The original proposal below is retained as design history. Its `Git.Server`, |
| 32 | `Git.ServerLive`, and Git-specific authentication services were removed; those |
| 33 | examples are superseded by the composition above. The storage design discussion |
| 34 | remains useful, but the historical API names are not current exports. |
| 35 | |
| 36 | **On an AWS-native assembly (the DynamoDB question):** a `RefStore` on |
| 37 | DynamoDB pay-per-request is *conditionally* feasible — conditional writes |
| 38 | give ref CAS directly, and `TransactWriteItems` (100-item atomic cap) |
| 39 | covers `commitPush` **only if** staged-object visibility moves from |
| 40 | per-row flips to a push-epoch marker (commit = one item flip; membership |
| 41 | checks join against committed pushes). That redesign is the honest |
| 42 | price. But a RefStore alone is not AWS parity: the protocol engine, |
| 43 | ObjectStore index, commit-graph walks, alarms (compaction/bundles/GC), |
| 44 | and the push serialization point all live in the DO today — an AWS |
| 45 | assembly means Lambda-hosted planes, DynamoDB-backed stores with |
| 46 | optimistic transactions replacing `transactionSync`, and EventBridge/SQS |
| 47 | replacing alarms. Feasible, PPR-friendly on cost, and exactly what the |
| 48 | storage-agnostic contracts of §3.3 point at — but it is a Phase-D-scale |
| 49 | port, not a store swap. Until then `BlobStoreS3` serves the real hybrid: |
| 50 | Cloudflare compute, S3 bytes. |
| 51 | |
| 52 | Re-thinks distribution of `alchemy/Git`. Today users import a finished |
| 53 | appliance: |
| 54 | |
| 55 | ```ts |
| 56 | const git = yield* GitService(); // one worker, one bucket, our routes, our auth |
| 57 | ``` |
| 58 | |
| 59 | The proposal: distribute **git building blocks** — Effect `Context.Service` |
| 60 | contracts with swappable `Layer` implementations — that users assemble into |
| 61 | *their own* Worker. The package ships **no Worker and no `GitService()`**: |
| 62 | the top-level block is `Git.Server`, a `Context.Service` exposing `fetch`. |
| 63 | Users construct the `Cloudflare.Worker` themselves, build one layer graph, |
| 64 | and wire `fetch` in. |
| 65 | |
| 66 | ```ts |
| 67 | import * as Git from "alchemy/Git"; |
| 68 | import * as BetterAuth from "@alchemy.run/better-auth"; |
| 69 | |
| 70 | // One layer graph, one Effect.provide. Every block is swappable. |
| 71 | const GitLive = Git.ServerLive.pipe( |
| 72 | Layer.provide(Git.ReposDurableObject), // refs + objects in Durable Objects |
| 73 | Layer.provide(Git.RegistryDurableObject), |
| 74 | Layer.provide(Git.PullStoreDurableObject), // pull requests |
| 75 | Layer.provide(Git.BlobStoreR2(Bucket)), // packs & clone bundles in R2 |
| 76 | Layer.provide( |
| 77 | Git.AuthBetterAuth({ |
| 78 | // your rules, in git's vocabulary — this line is branch protection |
| 79 | authorize: ({ user, repo, action }) => |
| 80 | action._tag === "Fetch" |
| 81 | ? repo.public || user != null |
| 82 | : action._tag === "Push" |
| 83 | ? action.updates.every((u) => |
| 84 | u.ref === "refs/heads/main" |
| 85 | ? user?.id === repo.owner |
| 86 | : user != null, |
| 87 | ) |
| 88 | : user?.id === repo.owner, |
| 89 | }), |
| 90 | ), |
| 91 | Layer.provide(BetterAuth.CloudflareD1(Database)), |
| 92 | ); |
| 93 | |
| 94 | export default class GitHost extends Cloudflare.Worker<GitHost>()( |
| 95 | "git", |
| 96 | { main: import.meta.url }, |
| 97 | Effect.gen(function* () { |
| 98 | const git = yield* Git.Server; |
| 99 | return { fetch: git.fetch }; |
| 100 | }).pipe(Effect.provide(GitLive)), |
| 101 | ) {} |
| 102 | ``` |
| 103 | |
| 104 | --- |
| 105 | |
| 106 | ## 1. Why |
| 107 | |
| 108 | 1. **Swappability.** R2 is an implementation detail that currently leaks |
| 109 | through six modules (`ObjectStore` exposes the raw |
| 110 | `ReadWriteBucketClient`; `PackSource`, `IncomingBody`, `Compact`, |
| 111 | `Bundle`, `Purge` all import the R2 client type; the Worker binds the |
| 112 | bucket directly for the bundle splice). Nothing about pack storage is |
| 113 | R2-specific — it needs ranged reads, streaming multipart writes, and |
| 114 | delete. S3 satisfies that through alchemy's *existing* AWS S3 |
| 115 | capabilities. |
| 116 | 2. **Partial adoption.** Some users want only the wire plane behind their |
| 117 | own auth; some want the REST plane inside an existing HttpApi; some want |
| 118 | the GitHub facade and nothing else. Today it's all-or-nothing. |
| 119 | 3. **Auth is the sharpest customization demand.** DESIGN §8 already calls |
| 120 | the `Credentials` service "the v2 identity seam". Making `Auth` a |
| 121 | first-class swappable service is how OAuth/JWT/SSO happens without |
| 122 | forking the package. |
| 123 | 4. It's the alchemy thesis applied to ourselves: infrastructure as |
| 124 | composable, typed Effect programs — not appliances. |
| 125 | |
| 126 | ## 2. One layer graph, two runtime contexts |
| 127 | |
| 128 | A git repo needs one serialization point: refs CAS, push staging, pull |
| 129 | numbering. On Cloudflare that is a Durable Object, and code that needs |
| 130 | `transactionSync` must run *inside* it. But — and this is the load-bearing |
| 131 | simplification — that is a **runtime** fact, not an **authorship** fact. |
| 132 | The DO classes live in the same script as the Worker, so the user authors |
| 133 | ONE layer graph; alchemy's DurableObject machinery captures the composed |
| 134 | layers and constructs the DO-internal sub-graph per instance on first |
| 135 | event (exactly how `GitRepoLive.pipe(Layer.provide(R2Binding))` works |
| 136 | today). |
| 137 | |
| 138 | ``` |
| 139 | ONE authored layer graph (the user's GitLive) |
| 140 | │ |
| 141 | ┌─────────────────────┴──────────────────────┐ |
| 142 | Worker runtime Repo DO runtime |
| 143 | Wire/Rest/GitHubCompat routes protocol engine |
| 144 | Registry resolution · Auth ObjectStore · RefStore |
| 145 | BlobStore (bundle splice reads) PullStore · TokenStore |
| 146 | jobs · BlobStore (packs/spill) |
| 147 | ``` |
| 148 | |
| 149 | Consequences: |
| 150 | |
| 151 | - Providing `Git.BlobStoreS3(bucket)` **once** satisfies every consumer on |
| 152 | both sides. There is no per-DO `layer:` option, no DO subclassing, and |
| 153 | no manual class re-export: providing `Git.ReposDurableObject` / |
| 154 | `Git.RegistryDurableObject` pulls the DO classes into the bundle through |
| 155 | the layer graph, and alchemy's generated worker entry exports them for |
| 156 | workerd (verify during Phase C that the generated-entry path covers |
| 157 | package-shipped DO classes; if not, that is a Phase C work item in |
| 158 | alchemy core, not user ceremony). |
| 159 | - The two runtime contexts still construct *separate instances* of a |
| 160 | layer's service (a Worker-side BlobStore client and a DO-side one). |
| 161 | That is invisible to the author but matters for layer implementors: |
| 162 | construction must be cheap and context-free (the isolate-scope rules |
| 163 | from AGENTS.md apply on both sides). |
| 164 | - Swappable-per-repo storage is out of scope: one graph means one |
| 165 | configuration per deployment, which is the sane default anyway. |
| 166 | |
| 167 | ## 3. Service taxonomy |
| 168 | |
| 169 | **Naming convention: one flat `Git` namespace, no nesting.** Contracts are |
| 170 | bare names (`Git.BlobStore`, `Git.Auth`, `Git.Server`); |
| 171 | implementations are `{Contract}{Impl}` (`Git.BlobStoreR2`, |
| 172 | `Git.BlobStoreS3`, `Git.AuthTokens`, `Git.ReposDurableObject`, |
| 173 | `Git.RegistryDurableObject`, `Git.PullStoreDurableObject`, |
| 174 | `Git.ServerLive` — all plain layers in one graph). Mirrors the package-level |
| 175 | flat-export convention (`ReadBucketBinding`, `GitAuthLive`) rather than |
| 176 | the `Cloudflare.KV.*` sub-namespacing. |
| 177 | |
| 178 | ### 3.1 `BlobStore` — the R2 ↔ S3 seam (the headline) |
| 179 | |
| 180 | Immutable, content-addressed bulk bytes. Everything the pack plane needs |
| 181 | and nothing more: |
| 182 | |
| 183 | ```ts |
| 184 | export class BlobStore extends Context.Service<BlobStore, { |
| 185 | /** Ranged read; `undefined` = whole object. `null` = missing. */ |
| 186 | readonly get: ( |
| 187 | key: string, |
| 188 | range?: { readonly offset: number; readonly length: number }, |
| 189 | ) => Effect.Effect<BlobBody | null, BlobStoreError, RuntimeContext>; |
| 190 | /** Whole-object write with known length (streams allowed). */ |
| 191 | readonly put: ( |
| 192 | key: string, |
| 193 | body: Uint8Array | Stream.Stream<Uint8Array, BlobStoreError>, |
| 194 | options: { readonly contentLength: number }, |
| 195 | ) => Effect.Effect<void, BlobStoreError, RuntimeContext>; |
| 196 | /** Streaming write of unknown length (push-body spill). */ |
| 197 | readonly multipart: ( |
| 198 | key: string, |
| 199 | ) => Effect.Effect<BlobMultipart, BlobStoreError, RuntimeContext>; |
| 200 | readonly delete: ( |
| 201 | keys: string | ReadonlyArray<string>, |
| 202 | ) => Effect.Effect<void, BlobStoreError, RuntimeContext>; |
| 203 | /** Prefix listing — purge/GC only, never on a serving path. */ |
| 204 | readonly list: ( |
| 205 | prefix: string, |
| 206 | ) => Stream.Stream<BlobMeta, BlobStoreError, RuntimeContext>; |
| 207 | }>()("alchemy/Git/BlobStore") {} |
| 208 | |
| 209 | export interface BlobBody { |
| 210 | readonly size: number; |
| 211 | readonly bytes: Effect.Effect<Uint8Array, BlobStoreError>; |
| 212 | readonly stream: Stream.Stream<Uint8Array, BlobStoreError>; |
| 213 | } |
| 214 | |
| 215 | export interface BlobMultipart { |
| 216 | readonly uploadPart: (n: number, part: Uint8Array) => |
| 217 | Effect.Effect<void, BlobStoreError>; |
| 218 | readonly complete: Effect.Effect<void, BlobStoreError>; |
| 219 | readonly abort: Effect.Effect<void, BlobStoreError>; |
| 220 | } |
| 221 | ``` |
| 222 | |
| 223 | Implementations shipped: |
| 224 | |
| 225 | | Layer | Rides on | Notes | |
| 226 | |---|---|---| |
| 227 | | `Git.BlobStoreR2(bucket)` | `Cloudflare.R2.ReadWriteBucket` | today's behavior, extracted | |
| 228 | | `Git.BlobStoreS3(bucket)` | `AWS.S3` capabilities (`GetObject`/`PutObject`/multipart) | already exist as alchemy bindings | |
| 229 | | `Git.BlobStoreMemory()` | Map | tests; kills the R2-shaped mocks | |
| 230 | |
| 231 | Uniform-part-size and part-count limits differ per store (R2: uniform |
| 232 | parts, 10k max; S3: 5 MiB min, 10k max) — the contract mandates the |
| 233 | *intersection* (uniform parts ≥ 5 MiB except last), which `IncomingBody` |
| 234 | already satisfies. |
| 235 | |
| 236 | Consumers refactored onto it: `ObjectStore` (stops leaking `bucket`), |
| 237 | `PackSource` (`r2RandomAccess` → `blobRandomAccess`), `IncomingBody`, |
| 238 | `Jobs/Compact`, `Jobs/Bundle`, `Jobs/Purge`, the Worker's bundle splice. |
| 239 | |
| 240 | ### 3.2 `Auth` — authentication + domain-specific authorization |
| 241 | |
| 242 | **Why not "AuthPolicy" or "AuthStore"?** The contract *decides* and stores |
| 243 | nothing: token secrets live behind `TokenStore`, sessions live in the auth |
| 244 | system's own database. |
| 245 | |
| 246 | **The contract is domain-specific — no imposed role ladder.** An earlier |
| 247 | draft had users return `"read" | "write" | "admin"` scopes; that imposes |
| 248 | *our default token implementation's* vocabulary on everyone. Instead, the |
| 249 | engine asks questions in the domain's own terms and the user answers |
| 250 | yes/no. The engine is the only party that knows what is actually being |
| 251 | attempted — actions are protocol facts (a receive-pack's parsed ref |
| 252 | commands, a merge's PR number), which is also why auth cannot live in a |
| 253 | fronting middleware: |
| 254 | |
| 255 | ```ts |
| 256 | /** What the caller is attempting — the engine's vocabulary, not ours. */ |
| 257 | export type GitAction = |
| 258 | | { readonly _tag: "Fetch" } // clone/fetch, REST reads |
| 259 | | { readonly _tag: "Push"; // receive-pack, ref writes |
| 260 | readonly updates: ReadonlyArray<RefUpdate> } // (per-branch rules!) |
| 261 | | { readonly _tag: "CreatePull"; readonly base: string; readonly head: string } |
| 262 | | { readonly _tag: "UpdatePull"; readonly number: number } |
| 263 | | { readonly _tag: "MergePull"; readonly number: number } |
| 264 | | { readonly _tag: "ReadRepo" } |
| 265 | | { readonly _tag: "UpdateRepo" } // visibility, default branch |
| 266 | | { readonly _tag: "DeleteRepo" } |
| 267 | | { readonly _tag: "CreateRepo"; readonly owner: string } |
| 268 | | { readonly _tag: "ManageTokens" }; |
| 269 | |
| 270 | export class Auth extends Context.Service<Auth, { |
| 271 | /** AUTHENTICATION: who is calling? (never fails closed on anonymous) */ |
| 272 | readonly authenticate: ( |
| 273 | headers: Readonly<Record<string, string | undefined>>, |
| 274 | ) => Effect.Effect<Actor, GitAuthError, RuntimeContext>; |
| 275 | /** AUTHORIZATION: may this actor perform this action on this repo? */ |
| 276 | readonly authorize: (input: { |
| 277 | readonly actor: Actor; |
| 278 | /** Repo state the engine supplies (visibility, owner) — `null` for |
| 279 | * repo-less actions like CreateRepo. */ |
| 280 | readonly repo: RepoContext | null; |
| 281 | readonly action: GitAction; |
| 282 | }) => Effect.Effect<boolean, GitAuthError, RuntimeContext>; |
| 283 | }>()("alchemy/Git/Auth") {} |
| 284 | ``` |
| 285 | |
| 286 | Users express *their* model in *their* vocabulary — org roles, per-branch |
| 287 | protection, PR-only workflows — because the action carries the facts: |
| 288 | |
| 289 | ```ts |
| 290 | const MyAuth = Git.AuthBetterAuth({ |
| 291 | authorize: ({ user, repo, action }) => { |
| 292 | switch (action._tag) { |
| 293 | case "Fetch": |
| 294 | case "ReadRepo": |
| 295 | return repo?.public || user != null; |
| 296 | case "Push": |
| 297 | // protected main: only the repo owner may update it directly |
| 298 | return action.updates.every((u) => |
| 299 | u.ref === "refs/heads/main" ? user?.id === repo.owner : user != null, |
| 300 | ); |
| 301 | case "MergePull": |
| 302 | return user?.id === repo.owner; |
| 303 | default: |
| 304 | return user?.id === repo.owner; |
| 305 | } |
| 306 | }, |
| 307 | }); |
| 308 | ``` |
| 309 | |
| 310 | **Worked backwards from `packages/better-auth`** (the exercise that shaped |
| 311 | this): issuance and auth UX stay *outside* the blocks — the user mounts |
| 312 | `auth.fetch` at `/api/auth/*` next to `git.fetch` in their worker. Per- |
| 313 | request decisions happen *inside* via this contract: the adapter |
| 314 | authenticates browser sessions with `auth.getSession(headers)` and git-CLI |
| 315 | Basic-auth PATs with the Better Auth API-key plugin (git cannot do cookies |
| 316 | or redirects — the password field carries the key), then runs the user's |
| 317 | `authorize` with the resolved user. |
| 318 | |
| 319 | **Where the old scope ladder went**: inside `Git.AuthTokens`. The default |
| 320 | implementation stores `read|write|admin` on its tokens and implements |
| 321 | `authorize` by mapping each `GitAction` to a required scope internally — |
| 322 | its business, invisible to the contract. Public-repo anonymous reads are |
| 323 | likewise just the default `authorize` behavior (`Fetch`/`ReadRepo` on |
| 324 | `repo.public`), no longer a special engine rule; `readOnly` write |
| 325 | rejection stays in the DO because it is repo state independent of the |
| 326 | caller. |
| 327 | |
| 328 | **Enforcement across the runtime boundary** (as implemented): the |
| 329 | Worker runs `authenticate` once per request and forwards the resulting |
| 330 | `Actor` to the DO over the trusted internal channel — the DO trusts |
| 331 | identity, never re-derives it, and the wire path keeps the |
| 332 | `ADMIN_HEADER` discipline (worker strips inbound forgeries, mints the |
| 333 | header only after its own timing-safe verify; pinned by a forged-header |
| 334 | 401 test). `authorize` runs at the site that owns the action's facts: |
| 335 | the Worker for registry-level actions (`CreateRepo`, `ListRepos`, with |
| 336 | `repo: null`), the Repo DO for everything per-repo — including |
| 337 | `git-receive-pack`, which authorizes twice: once at entry with |
| 338 | `Push { updates: [] }` ("may this caller push at all?", cheap rejection |
| 339 | before the pack streams) and again after command parsing with the real |
| 340 | `updates`, which is where per-branch policies bite. The DO enriches |
| 341 | `token` actors from its tokens table before asking (identity is the |
| 342 | engine's job; the scope's *meaning* is `AuthTokens`'s). Both runtime |
| 343 | contexts resolve the same `Auth` layer from the one graph (§2), so |
| 344 | `authorize` must be a fast, pure decision — it sits on the DO hot path. |
| 345 | |
| 346 | **Why have `Auth` at all, instead of "write your own auth over the top"?** |
| 347 | A fronting middleware can only allow or deny whole requests. It cannot see |
| 348 | the actions — which ref a push updates, which PR a merge targets — without |
| 349 | reimplementing the protocol, and it cannot stamp identity into domain |
| 350 | objects without smuggling it through headers, which is an auth seam |
| 351 | anyway, just untyped. `Auth` IS auth-over-the-top in its minimal typed |
| 352 | form: two functions. Implementing it from scratch is a page of code; |
| 353 | `Git.AuthTokens` makes fresh deploys secure by default; |
| 354 | `Git.AuthBetterAuth` is optional sugar. Coarse gating (rate limits, IP |
| 355 | allowlists) still composes by wrapping `git.fetch`. |
| 356 | |
| 357 | ### 3.3 Storage services — defined by domain semantics, not SQL |
| 358 | |
| 359 | The repo's storage services are **storage-agnostic domain contracts**. |
| 360 | There is no shared `SqlClient`/`RepoSql` service and no SQL anywhere in a |
| 361 | contract — SQLite-in-the-DO is an implementation detail of the *default |
| 362 | layers*, invisible to consumers and to alternative implementations. |
| 363 | |
| 364 | - **`ObjectStore`** — content-addressed object index + hot bytes: |
| 365 | `getMeta(Batch)` / `readContent` / `insertStagedBatch` / |
| 366 | `missingObjects` / `promoteStaged` / compaction bookkeeping. (Bulk cold |
| 367 | bytes already live behind `BlobStore`.) |
| 368 | - **`RefStore`** — ref reads, HEAD, and the repo's **atomic commit |
| 369 | points** expressed as domain operations, not exposed transactions: |
| 370 | |
| 371 | ```ts |
| 372 | /** Atomically: verify CAS expectations, flip staged objects live, |
| 373 | * apply ref updates, run registered stamps (e.g. mark a pull |
| 374 | * merged). Implementations MUST make this a single atomic unit. */ |
| 375 | readonly commitPush: (input: CommitPushInput) => |
| 376 | Effect.Effect<CommitPushResult, RefConflict | ..., RuntimeContext>; |
| 377 | ``` |
| 378 | |
| 379 | Atomicity is a *documented obligation of the contract* — the default |
| 380 | layer discharges it with the DO's `transactionSync`; a hypothetical |
| 381 | FoundationDB/Postgres layer would use its own transactions. Cross- |
| 382 | service atomicity (refs + staged objects + pull stamping) is handled by |
| 383 | making the composite operation itself the contract surface — never by |
| 384 | leaking a transaction handle across services. |
| 385 | - **`PullStore`** — pull rows + per-repo numbering (`create` allocates |
| 386 | the next number atomically; how is the layer's business). |
| 387 | - **`TokenStore`** — mint/verify/revoke (storage behind the default |
| 388 | `Git.AuthTokens`). |
| 389 | |
| 390 | Default layers ship as flat `Git.{Service}DurableObject` implementations, |
| 391 | all provided in the user's single layer graph. The near-term win is honesty and |
| 392 | testability (contracts mockable without SQL fixtures; each default layer |
| 393 | unit-tested in isolation); the long-term win is that "swap the metadata |
| 394 | store" is now a layer, not a fork. |
| 395 | |
| 396 | ### 3.4 Worker-site services |
| 397 | |
| 398 | - **`Registry`** — `owner/name → repoId` resolution + listing (contract |
| 399 | over the Registry DO; a single-tenant "static registry" layer becomes |
| 400 | possible for users who want exactly one repo and no registry DO). |
| 401 | - **HTTP groups** — `Git.Protocol`, `Git.Repos`, `Git.Refs`, `Git.Objects`, |
| 402 | `Git.Pulls`, and `Git.GitHub`. Each is independently mountable in an |
| 403 | Effect `HttpApi`. `Git.Handlers` exposes the corresponding handler |
| 404 | objects; `Git.ApiHandlersLive` builds them over `RegistryStore`, |
| 405 | `RepoStore`, `BlobStore`, and `Hasher`. |
| 406 | |
| 407 | ## 4. `Git.Server` — the top-level block |
| 408 | |
| 409 | There is no `GitService()` and no shipped Worker. The package's largest |
| 410 | unit is a service: |
| 411 | |
| 412 | ```ts |
| 413 | // Open default: all six groups, with the shared handler implementation. |
| 414 | const GitLive = Git.ServerLive.pipe( |
| 415 | Layer.provide(Git.ReposDurableObject), |
| 416 | Layer.provide(Git.RegistryDurableObject), |
| 417 | Layer.provide(Git.HasherInline), |
| 418 | Layer.provide(Git.BlobStoreR2(GitObjects)), |
| 419 | ); |
| 420 | |
| 421 | // A custom management API: select a group, attach middleware, and implement it. |
| 422 | class ManagementApi extends HttpApi.make("git-management") |
| 423 | .add(Git.Repos) |
| 424 | .middleware(Authenticated) {} |
| 425 | |
| 426 | const ManagementLive = HttpApiBuilder.group(ManagementApi, "repos", (h) => |
| 427 | Effect.map(Git.Handlers, (git) => h.handleAll(git.repos)), |
| 428 | ); |
| 429 | |
| 430 | const ManagementServer = Git.Server.layer(ManagementApi, ManagementLive).pipe( |
| 431 | Layer.provide(Git.ApiHandlersLive), |
| 432 | Layer.provide(AuthenticatedLive), |
| 433 | // provide the storage and hasher layers as above |
| 434 | ); |
| 435 | ``` |
| 436 | |
| 437 | `Git.Server` exposes `fetch`, so users can mount it whole, nest it under |
| 438 | another router, or register the exported API groups directly. |
| 439 | `Server.layer` mounts `Git.InternalApi` separately from application |
| 440 | middleware. A host that builds its own router can pair `InternalApi` |
| 441 | with `Git.InternalLive`. Users own the `Cloudflare.Worker`, its bindings, |
| 442 | domains, assets, and Durable Object exports. |
| 443 | |
| 444 | Consequences accepted: this is a **breaking change** — `GitService()` and |
| 445 | the internal `GitWorker` class are deleted, and the example app becomes |
| 446 | the reference assembly (its own Worker + `Git.ServerLive` + default |
| 447 | layers + its one-origin SPA front door). |
| 448 | |
| 449 | ## 5. Migration phases |
| 450 | |
| 451 | Each phase lands green on the full suite before the next starts. |
| 452 | |
| 453 | - **Phase A — `BlobStore`.** Define the contract + `R2`/`Memory` layers; |
| 454 | refactor the six consumers and the Worker splice onto it; delete the |
| 455 | `bucket` leak from `ObjectStore`. Pure refactor, no behavior change. |
| 456 | Then add `Git.BlobStoreS3` + a live S3-flavored test (deploys the DO with |
| 457 | the S3 layer against a real bucket). |
| 458 | - **Phase B — `Auth` + DO-site services.** Extract `RefStore` / |
| 459 | `PullStore` / `TokenStore` / `ObjectStore`-as-service; `RepoObject` |
| 460 | becomes choreography. `Auth` replaces the scattered |
| 461 | `parseBasicOrBearer`/`verifyAdminKey` call sites. |
| 462 | - **Phase C — plane split + `Git.Server`.** Native HTTP groups exported |
| 463 | individually; `Git.Server` service + `Git.ServerLive`/`Git.Server.layer(api, groups)`; |
| 464 | DO classes re-exported by users (no subclass factory); **`GitService()` |
| 465 | and the shipped `GitWorker` deleted**; the example app rewritten as the reference |
| 466 | assembly; tests re-pointed at a fixture assembly; docs (`@layer` pages |
| 467 | per block, assembly guide). All documentation and examples compose with |
| 468 | `Layer.mergeAll`/`Layer.provide` into a single layer handed to one |
| 469 | `Effect.provide` — chained `Effect.provide` calls are a lint-level |
| 470 | error in this package. |
| 471 | |
| 472 | ## 6. AWS-native assembly (no Durable Objects) — parity analysis |
| 473 | |
| 474 | > **Status: DEFERRED.** Lambda cannot accept streaming/large request |
| 475 | > bodies (6 MB sync-invoke cap, base64 overhead → ~4.5 MB effective), so |
| 476 | > there is no serverless answer for `receive-pack` on AWS today. The |
| 477 | > practical AWS path is Durable Objects hosted on ECS (Atypus), which is |
| 478 | > separate ongoing work. This section is kept as the design record; the |
| 479 | > project focus is Cloudflare-native assemblies. |
| 480 | |
| 481 | What the Repo DO actually provides, and the AWS-native replacement for |
| 482 | each. The pure `Protocol/` codec layer, `BlobStore` (S3 layer shipped), and |
| 483 | the HTTP planes (Effect `HttpApi`/`HttpRouter` — portable to alchemy's |
| 484 | Lambda hosts) need **no** work. The rest: |
| 485 | |
| 486 | | DO capability | AWS-native replacement | difficulty | |
| 487 | |---|---|---| |
| 488 | | refs CAS | DynamoDB conditional writes | easy | |
| 489 | | `commitPush` atomicity (refs + staged flip + pull stamp) | `TransactWriteItems` (100-item cap) — **requires the push-epoch redesign** | medium | |
| 490 | | push semaphore | mostly unnecessary: staging is content-addressed and idempotent; losers fail at the commit CAS. Optional lease lock for resource hygiene | easy | |
| 491 | | object index + hot bytes (SQLite rows) | metadata items in DynamoDB (`BatchGetItem` ×100 for `missingObjects`/`getMetaBatch`); bytes in S3 from day one (DynamoDB's 400 KB item cap rules out the 1 MiB inline tier) | medium | |
| 492 | | commit graph + merge-base walks (10k sub-ms SELECTs) | batched-frontier BFS over `BatchGetItem`; acceptable for PR-sized divergence, painful for deep walks — **reachability bitmaps get promoted from optimization to prerequisite** | hard | |
| 493 | | wire-plane hosting (streaming HTTP both directions) | **the real blocker, and it is not DynamoDB**: Function URLs stream *responses* (up to 200 MB, bandwidth-capped past 6 MB) but **not requests** — the request body is buffered into the invocation event under the 6 MB sync-invoke cap (binary is base64-encoded into that JSON, so ~4.5 MB effective), which breaks any non-trivial `git push`. Options: (a) Fargate/App Runner container for the wire plane (no caps, long-lived), (b) a two-phase push (presign → S3 → commit) which stock git clients cannot speak, or (c) accept a push cap. Clones are fine — response streaming covers upload-pack, and presigned S3 bundle URLs make the serving plane *better* than the DO splice (zero compute) | hard | |
| 494 | | alarms (compact / bundle / gc / purge) | SQS delayed messages or EventBridge Scheduler one-shots enqueued at the commit point | easy | |
| 495 | | Registry DO | DynamoDB table, conditional create | easy | |
| 496 | | tokens / pulls tables | DynamoDB tables (PPR) | easy | |
| 497 | |
| 498 | **Single-table design is mandatory.** The DynamoDB stores above are one |
| 499 | table, not five: everything keys as `PK = REPO#<repoId>` with typed sort |
| 500 | keys (`REF#refs/heads/main`, `OBJ#<oid>`, `PUSH#<pushId>`, |
| 501 | `PULL#<n>`, `META`), plus `OWNER#<owner>` / `NAME#<name>` items for the |
| 502 | registry and `TOKEN#<hash>` for tokens. This is not a style preference: |
| 503 | `TransactWriteItems` spans items freely but the whole `commitPush` |
| 504 | transaction (ref CAS + push flip + pull stamp) staying in one table keeps |
| 505 | capacity, backup, and IAM surface singular, and the per-repo partition |
| 506 | key gives every store the same locality the DO gave for free. The |
| 507 | storage-agnostic contracts (§3.3) never see any of this — key schema is |
| 508 | an implementation detail of the `*Dynamo` layers, exactly as SQL never |
| 509 | appears in the contracts today. |
| 510 | |
| 511 | **The push-epoch redesign** (the load-bearing prerequisite): today a |
| 512 | commit flips thousands of staged object rows live inside one SQLite |
| 513 | transaction — impossible under a 100-item transaction cap. Instead, |
| 514 | objects reference their `pushId` permanently and visibility is a single |
| 515 | `pushes` item flipping to `committed`; membership checks join against |
| 516 | committed pushes. One item flip = fits any transaction budget. Notably |
| 517 | this would *also* speed up the DO implementation (no bulk row flip), so |
| 518 | it is worth doing in Phase B regardless of AWS plans. |
| 519 | |
| 520 | **Sequencing if AWS parity becomes real:** Phase B store extraction |
| 521 | (the prerequisite for everything — the contracts must exist before |
| 522 | Dynamo layers can implement them) → push-epoch redesign (benefits both |
| 523 | backends) → `RefStore`/`ObjectStore`/`PullStore`/`TokenStore`/`Registry` |
| 524 | Dynamo layers → Lambda-hosted REST + GitHub-compat planes (no size |
| 525 | issues, lands early) → reachability bitmaps → the wire-plane hosting |
| 526 | decision (Fargate container vs. capped Lambda) last, because it is the |
| 527 | only piece with no clean serverless answer. |
| 528 | |
| 529 | Cost shape under pay-per-request: a 44k-object push ≈ 44k WCUs ≈ $0.05; |
| 530 | metadata reads are cheap; the closure-walk read amplification is the |
| 531 | number to watch, and bitmaps are what contain it. |
| 532 | |
| 533 | ## 7. Open questions (want your calls) |
| 534 | |
| 535 | 1. **S3 priority**: is `Git.BlobStoreS3` a Phase-A deliverable (proves the |
| 536 | seam immediately) or is the seam + `Memory` enough until someone asks? |
| 537 | 2. ~~Namespace~~ **Resolved**: one flat `Git` namespace — |
| 538 | `Git.BlobStoreS3`, `Git.ReposDurableObject`, `Git.ServerLive`; no |
| 539 | nested sub-namespaces. |
| 540 | 3. **Registry optionality**: is the single-tenant/no-registry layer worth |
| 541 | designing for now, or YAGNI? |
| 542 | 4. **How far to shrink `RepoObject`**: Phase B as scoped keeps the wire |
| 543 | protocol choreography inline. Extracting a `Protocol` service too is |
| 544 | possible but touches the most battle-tested code for the least |
| 545 | swappability value — I'd leave it. |
| 546 | 5. ~~Does `Auth` absorb the identity feature?~~ **Resolved by the |
| 547 | better-auth exercise**: yes — `identify()` + the `grant` variant carry |
| 548 | identity, and PR authorship/merge attribution consume it in Phase B. |