clover's git

the-knife

1	---
2	title: How bindings grant IAM
3	description: What a yield* of a GCP binding actually does — the per-host service account, the role it grants, and how removing a binding revokes it.
4	---
5
6	On GCP, giving a container permission to call an API normally means
7	three separate things: create a service account, attach it to the Cloud
8	Run service, and add role bindings to the project IAM policy. Alchemy
9	collapses all three into the line that gets you the client.
10
11	```typescript
12	const events = yield* GCP.PubSub.WriteTopic(topic);
13	```
14
15	That expression:
16
17	1. Creates (or adopts) a runtime service account for this host.
18	2. Adds `roles/pubsub.publisher` to it on that one topic's IAM policy.
19	3. Injects whatever the call needs into the revision.
20	4. Returns a typed client (`events.publish(...)`) you call from a
21	request handler.
22
23	It is the GCP counterpart of AWS `policyStatements` on a Lambda and of
24	the scoped `AccountApiToken` a Cloudflare binding mints.
25
26	## The per-host service account
27
28	Every effectful GCP host — `GCP.Run.Service` (and its alias
29	`GCP.Function`), `GCP.Run.Job`, `GCP.Run.WorkerPool`, and
30	`GCP.CloudFunctions.Function` — gets its own account. The id is the
31	host's name plus a hash of its type and full resource name, so no two
32	hosts share an account — not across stages, and not the old and new
33	generation of a replacement:
34
35	```
36	alch-api-3f9c0a1b2d@my-project.iam.gserviceaccount.com
37	```
38
39	Accounts Alchemy mints carry the display name `alchemy-host`, which is
40	how `pnpm nuke:gcp` finds them. Alchemy also grants
41	`roles/iam.serviceAccountUser` to the Cloud Run and Cloud Functions
42	service agents so they can `actAs` the account, and retries the deploy
43	while that grant propagates.
44
45	Pass your own `serviceAccount` on the host and Alchemy uses it instead.
46	The difference is what happens on removal, below.
47
48	## Access-level clients
49
50	For the data services you use most, a binding is an access level on a
51	resource rather than an API operation — the same Read / Write /
52	ReadWrite split as Cloudflare's R2 and KV bindings. Ask for the least
53	you need; `ReadWrite` is the union of the other two.
54
55	| Resource | Read | Write | ReadWrite |
56	| --- | --- | --- | --- |
57	| `Storage.Bucket` | `ReadBucket`: `head`, `get`, `list` | `WriteBucket`: `put`, `delete` | `ReadWriteBucket` |
58	| `PubSub.Topic` | — | `WriteTopic`: `publish`, `publishBatch` | — |
59	| `PubSub.Subscription` | `ReadSubscription`: `pull`, `acknowledge`, `modifyAckDeadline` | — | — |
60	| `Firestore.Database` | `ReadDatabase`: `get`, `list`, `query` | `WriteDatabase`: `set`, `update`, `create`, `delete` | `ReadWriteDatabase` |
61	| `BigQuery.Table` | `ReadTable`: `list`, `query` | `WriteTable`: `insert` | `ReadWriteTable` |
62	| `SecretManager.Secret` | `ReadSecret`: `access`, `accessBytes` | `WriteSecret`: `addVersion`, `disableVersion`, `destroyVersion` | `ReadWriteSecret` |
63
64	A topic is only ever written to and a subscription only ever read, so
65	Pub/Sub has one client each. The clients speak plain JavaScript:
66	Firestore fields and BigQuery rows are encoded and decoded for you,
67	Pub/Sub and Secret Manager payloads come back as strings or bytes, and
68	a missing document or secret version reads as `undefined` rather than
69	an error.
70
71	```typescript
72	const db = yield* GCP.Firestore.ReadWriteDatabase(database);
73	const doc = yield* db.get("counters/visits");
74	const count = typeof doc?.fields.count === "number" ? doc.fields.count : 0;
75	yield* db.set("counters/visits", { count: count + 1 });
76	```
77
78	Everything without an access-level client — KMS `Encrypt` / `Decrypt`,
79	`Run.RunJob`, Cloud Tasks, Memorystore, and the long tail of generated
80	services — is a per-operation binding that returns one callable.
81
82	## Which role a binding grants
83
84	Each binding grants the narrowest predefined role that covers it:
85
86	| Binding | Role | Granted on |
87	| --- | --- | --- |
88	| `Storage.ReadBucket` | `roles/storage.objectViewer` | the bucket |
89	| `Storage.WriteBucket`, `ReadWriteBucket` | `roles/storage.objectUser` | the bucket |
90	| `PubSub.WriteTopic` | `roles/pubsub.publisher` | the topic |
91	| `PubSub.ReadSubscription` | `roles/pubsub.subscriber` | the subscription |
92	| `Firestore.ReadDatabase` | `roles/datastore.viewer` | the project, conditioned on the database |
93	| `Firestore.WriteDatabase`, `ReadWriteDatabase` | `roles/datastore.user` | the project, conditioned on the database |
94	| `BigQuery.ReadTable` | `roles/bigquery.dataViewer` + `roles/bigquery.jobUser` | the table + the project |
95	| `BigQuery.WriteTable` | `roles/bigquery.dataEditor` | the table |
96	| `BigQuery.ReadWriteTable` | `roles/bigquery.dataEditor` + `roles/bigquery.jobUser` | the table + the project |
97	| `BigQuery.Query` | `roles/bigquery.dataViewer` + `roles/bigquery.jobUser` | the dataset's access list + the project |
98	| `SecretManager.ReadSecret` | `roles/secretmanager.secretAccessor` | the secret |
99	| `SecretManager.WriteSecret` | `roles/secretmanager.secretVersionManager` | the secret |
100	| `SecretManager.ReadWriteSecret` | both of the above | the secret |
101	| `KMS.Encrypt` / `Decrypt` | `roles/cloudkms.cryptoKeyEncrypter` / `Decrypter` | the key |
102	| `CloudTasks.CreateTask` | `roles/cloudtasks.enqueuer` | the queue |
103	| `Run.RunJob` | `roles/run.jobsExecutorWithOverrides` | the job |
104	| `SQL.*` | `roles/cloudsql.*` | the project, conditioned on the instance |
105
106	## How grants are scoped
107
108	A grant lands in the narrowest place GCP lets it, in this order:
109
110	1. **The resource's own IAM policy** — the GCP counterpart of an AWS
111	statement's `Resource: [arn]`. Buckets, topics, subscriptions,
112	secrets, KMS keys, Cloud Tasks queues, Run jobs and services, and
113	BigQuery tables all have one.
114	2. **The dataset access list** for BigQuery datasets, which have no
115	`setIamPolicy`: the access list *is* the dataset's policy, so
116	Alchemy adds the account as an access entry.
117	3. **The project policy under an IAM Condition** for services that
118	have no per-resource policy but support conditions — Firestore,
119	Cloud SQL, and Managed Kafka. The binding carries a condition like
120	`resource.name == "projects/p/databases/links" ||
121	resource.name.startsWith("projects/p/databases/links/")`, so the
122	role applies to that one resource and its children and nothing else
123	in the project.
124	4. **The project policy, unconditioned**, only where GCP offers
125	neither resource IAM nor condition support: Memorystore, AlloyDB,
126	Filestore, and Vertex AI publisher models (Gemini). BigQuery's
127	`roles/bigquery.jobUser` is also project-only, because running a
128	query job is a project permission.
129
130	The Memorystore RESP bindings (`ReadRedis`, `WriteRedis`,
131	`ReadWriteRedis`) grant nothing at all: RESP authenticates with the
132	instance's AUTH string, which travels to the runtime with the
133	connection URL.
134
135	IAM is eventually consistent, and conditional project grants are the
136	slowest to land: expect a freshly deployed host to see `Forbidden`
137	from Firestore for a few minutes.
138
139	## Adding and removing
140
141	The set of bindings you yield *is* the desired IAM state. The host
142	records every grant it made; on the next deploy, a grant no binding
143	asks for any more is revoked from that resource — delete a `yield*` and
144	the permission goes away. For an Alchemy-minted account, the project
145	policy is also synced against what GCP reports, so project roles added
146	out of band are removed too. Accounts you supply yourself only ever
147	lose grants Alchemy recorded, because Alchemy cannot tell which of the
148	account's other roles matter.
149
150	Policies are updated read-modify-write under their etag, so parallel
151	hosts deploying against one project or topic converge instead of
152	overwriting each other. Conditional bindings are matched on their exact
153	condition: Alchemy only ever edits the conditional bindings it wrote,
154	and never one someone else added.
155
156	Destroying the host revokes every role Alchemy granted and deletes the
157	minted account.
158
159	## Provide the implementation
160
161	Yielding a binding gives you the interface. The layer that implements
162	it is provided once, at the end of the host's Effect:
163
164	```typescript
165	export default class Api extends GCP.Function<Api>()(
166	"Api",
167	{ main: import.meta.url, location: "us-central1" },
168	Effect.gen(function* () {
169	const events = yield* GCP.PubSub.WriteTopic(topic);
170	// ...
171	}).pipe(
172	Effect.provide([GCP.PubSub.WriteTopicHttp]),
173	),
174	) {}
175	```
176
177	The convention is the binding name plus `Http`: `WriteTopic` →
178	`WriteTopicHttp`, `ReadWriteDatabase` → `ReadWriteDatabaseHttp`,
179	`RunJob` → `RunJobHttp`. Forget one and the type error names the
180	missing layer.
181
182	## Errors are typed
183
184	Bindings return the operation's real error union, so the failure modes
185	are in the type rather than in a status-code comparison:
186
187	```typescript
188	yield* db.create(`links/${code}`, { url }).pipe(
189	Effect.catchTag("GCP.Firestore.DocumentAlreadyExists", () =>
190	Effect.succeed(undefined),
191	),
192	);
193	```
194
195	`Forbidden`, `BadRequest`, and client-specific tags such as
196	`DocumentAlreadyExists` or BigQuery's `InsertRowsFailed` are the ones
197	you will handle most; per-operation bindings also surface `NotFound`
198	and `Conflict`. Anything you do not catch stays in the channel until
199	you `Effect.orDie` it.
200
201	## Which host to use
202
203	- **`GCP.Function`** — alias of `GCP.Run.Service`. An HTTP server with
204	a `fetch` entry. Scales to zero. Public only when you pass
205	`invokerIamDisabled: true`.
206	- **`GCP.Run.Job`** — a `run` entry that executes and exits. Nothing is
207	billed between executions. Start it with a `GCP.Run.RunJob` binding
208	or Cloud Scheduler.
209	- **`GCP.Run.WorkerPool`** — a long-running pool with no request path,
210	for pulling from a queue continuously.
211	- **`GCP.CloudFunctions.Function`** — a gen2 function. With `main`,
212	Alchemy bundles the program for Node.js, uploads the archive, and
213	serves `fetch` through the Functions Framework, so the Effect runs
214	exactly as on a Service. Without `main` it deploys your own archive.
215
216	The Run hosts build a container image locally, which means Docker has
217	to be running to deploy them. Cloud Functions build in the cloud.
218
219	## Event sources
220
221	Bindings are calls your code makes. Event sources are the reverse:
222	Google calls your code. Each one is a contract in the source service
223	plus an implementation layer for the kind of host, the same split AWS
224	uses for `SQS.consumeQueueMessages` with Lambda or a server host:
225
226	| Consume with | Contract | Push hosts (Service, Function) | Pull hosts (Job, WorkerPool) |
227	| --- | --- | --- | --- |
228	| `GCP.PubSub.consumeTopicMessages` | `PubSub.TopicEventSource` | `GCP.Run.TopicEventSource` | `GCP.Run.TopicPullEventSource` |
229	| `GCP.Storage.consumeBucketEvents` | `Storage.BucketEventSource` | `Storage.BucketEventSourceLive` + a topic layer | same |
230	| `GCP.CloudScheduler.consumeSchedule` | `CloudScheduler.ScheduleEventSource` | `GCP.Run.ScheduleEventSource` | — |
231	| `GCP.Eventarc.consumeEvents` | `Eventarc.EventSource` | `GCP.Run.EventarcEventSource` | — |
232
233	```typescript
234	yield* GCP.PubSub.consumeTopicMessages(orders, (messages) =>
235	messages.pipe(Stream.runForEach(({ message }) => handle(message))),
236	);
237	// …provided with Effect.provide(GCP.Run.TopicEventSource)
238	```
239
240	`GCP.Eventarc.consumeEvents` covers everything else Google emits as a
241	CloudEvent: Cloud Storage and Firestore changes, Audit Log entries for
242	any Google API call, and partner channels. You pass Eventarc's
243	attribute filters and get each decoded CloudEvent.
244
245	A push source provisions the delivery (a push subscription, a Storage
246	notification, a scheduler job, an Eventarc trigger) at the host's URL on a path under
247	`/__alchemy/`, signed with an OIDC token for the host's runtime service
248	account, and grants that account `roles/run.invoker` on the host. At
249	runtime the host claims those paths before your `fetch` and verifies
250	every token's signature, audience, and email, so the service can stay
251	private. A 2xx acks; a failed handler answers 500 and Pub/Sub
252	redelivers. Every deploy blocks until GCP reports the delivery in place:
253	IAM grants are read back, the subscription and scheduler job exist and
254	are enabled, and an Eventarc trigger reports healthy. A pull source creates a subscription the host owns and runs
255	a pull loop that acks each batch after the handler succeeds.
256
257	## Next
258
259	- [Serve an API on Cloud Run](/gcp/guides/cloud-run-api) — bindings in
260	a request path.
261	- [Ingest events into BigQuery](/gcp/guides/event-pipeline) — a Service
262	triggering a Job.