1---
2title: Bytes in S3, hashing on Lambda
3description: Compute on Cloudflare, bytes in S3, and large pushes hashed one Lambda per chunk. One stack carries both provider sets, and Alchemy mints the cross-cloud identity.
4---
5
6The examples use `Authentication` from
7[Getting Started](/git/getting-started): the application's request middleware.
8
9import { Code, Tabs, TabItem } from "@astrojs/starlight/components";
10
11The Worker and the Durable Objects stay on Cloudflare. Packs, bundles,
12and spilled pushes live in S3, and every 4 MiB chunk of a push is
13hashed on its own Lambda:
14
15```typescript
16// src/git.ts
17import * as AWS from "alchemy/AWS";
18import * as Cloudflare from "alchemy/Cloudflare";
19import * as Git from "alchemy/Git";
20import * as GitHasher from "alchemy/Git/Hasher";
21import * as Effect from "effect/Effect";
22import * as Layer from "effect/Layer";
23import * as HttpRouter from "effect/http/HttpRouter";
24import * as Http from "alchemy/Http";
25
26const GitObjects = AWS.S3.Bucket("GitObjects");
27
28const GitLive = Git.ApiLive.pipe(
29 Layer.provide(Git.ApiHandlersLive),
30 Layer.provide(Authentication.layer),
31 Layer.provide(Git.ReposDurableObject),
32 Layer.provide(Git.RegistryDurableObject),
33 Layer.provide(Git.BlobStoreS3(GitObjects)),
34 Layer.provide(GitHasher.HasherLambda(GitHasher.HasherFunction)),
35 Layer.provide(AWS.Lambda.InvokeFunctionHttp),
36);
37
38export default class GitHost extends Cloudflare.Worker<GitHost>()(
39 "Git",
40 { main: import.meta.url, ...Git.GIT_WORKER_OPTIONS },
41 Effect.gen(function* () {
42 const fetch = yield* HttpRouter.toHttpEffect(GitLive.pipe(Layer.provide(Http.Platform)));
43 return { fetch };
44 }),
45) {}
46```
47
48```typescript
49// alchemy.run.ts
50import * as Alchemy from "alchemy";
51import * as AWS from "alchemy/AWS";
52import * as Cloudflare from "alchemy/Cloudflare";
53import * as Effect from "effect/Effect";
54import * as Layer from "effect/Layer";
55import GitHost from "./src/git.ts";
56
57export default Alchemy.Stack(
58 "GitService",
59 {
60 providers: Layer.mergeAll(Cloudflare.providers(), AWS.providers()),
61 state: Cloudflare.state(),
62 },
63 Effect.gen(function* () {
64 const git = yield* GitHost;
65 return { url: git.url.as<string>() };
66 }),
67);
68```
69
70<Tabs syncKey="pkgManager">
71 <TabItem label="bun" icon="bun">
72 <Code code="bun alchemy deploy" lang="sh" />
73 </TabItem>
74 <TabItem label="npm" icon="npm">
75 <Code code="npm run alchemy deploy" lang="sh" />
76 </TabItem>
77 <TabItem label="pnpm" icon="pnpm">
78 <Code code="pnpm alchemy deploy" lang="sh" />
79 </TabItem>
80 <TabItem label="yarn" icon="seti:yarn">
81 <Code code="yarn alchemy deploy" lang="sh" />
82 </TabItem>
83</Tabs>
84
85## What gets created
86
87On Cloudflare, the same Worker and Durable Object namespaces as
88[All on Cloudflare](/git/recipes/cloudflare), minus the bucket.
89
90On AWS:
91
92- `GitObjects`, an S3 bucket with an engine-generated name in the
93 deploying profile's region.
94- `HasherFunction`, a 3 GB Lambda whose only handler answers hash
95 events, with its execution role.
96- One IAM user, one access key, and one least-privilege role the Worker
97 assumes. `BlobStoreS3` and `InvokeFunctionHttp` share that identity:
98 the key and the role's ARN are bound onto the Worker, each request is
99 signed with credentials assumed at runtime, and the role is granted
100 the S3 actions on that one bucket and `lambda:InvokeFunction` on that
101 one function.
102
103## What crosses the internet
104
105A push's chunks go to Lambda and their scan results come back. Pack
106reads for a fetch, bundle streams for a clone, and pack writes from
107compaction all go to S3. Reads are coalesced into few large ranges,
108and the Worker streams bundle bytes straight through, but every one of
109those is a hop the R2 stack does not make.
110
111## When it is worth it
112
113Take this stack when the bytes have to be in AWS. Take the Lambda
114hasher on its own when pushes are regularly wider than four chunks.
115Lambda hashes every chunk at once. The loaded Workers hash four at a
116time:
117
118```diff lang="typescript"
119const GitLive = Git.ApiLive.pipe(
120 Layer.provide(Git.ApiHandlersLive),
121 Layer.provide(Authentication.layer),
122 Layer.provide(Git.ReposDurableObject),
123 Layer.provide(Git.RegistryDurableObject),
124- Layer.provide(GitHasher.HasherWorkerLoader()),
125+ Layer.provide(GitHasher.HasherLambda(GitHasher.HasherFunction)),
126 Layer.provide(Git.BlobStoreR2(GitObjects)),
127+ Layer.provide(AWS.Lambda.InvokeFunctionHttp),
128);
129```
130
131Measured on the same 40 MiB delta-heavy push, the two hashers land
132within a second of each other. [Hasher](/git/blocks/hasher) has the
133table.