Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Events — public ICS calendar feed URL (includes an auth token as a query param)
EVENTS_ICS_URL=

# Notion API
NOTION_SECRET=
NOTION_DB_ID=
NOTION_SIGNATORIES_DB_ID=
NOTION_FAQ_DB_ID=
NOTION_IDEAS_DB_ID=
Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ Many routes exist in pairs, e.g. `about.astro` / `about-new.astro`, `events.astr

### Data sources

- **Notion API** (`src/store/notionClient.ts`) — events and other dynamic content via `fetchNotionEvents()` / `fetchNotionEventById()`. Images are direct Notion-hosted URLs (no proxy/cache — that worker was removed), expire after ~1hr, client falls back to `/images/default.jpg` on load error. See `docs/EVENTS.md` / `docs/NOTION.md`.
- **Events ICS feed** (`src/store/eventsClient.ts`) — `fetchEvents()` fetches and hand-parses a public ICS calendar feed (`EVENTS_ICS_URL`, Mattermost Events Calendar plugin), server-side only since the URL carries an auth token. Consumed by `/api/events` and grouped into category sections (`src/utils/eventSections.ts`) by both `events.astro`/`Events.tsx` and `events-new.astro`/`EventsNew.tsx`. See `docs/EVENTS.md`.
- **Notion API** (`src/store/notionClient.ts`) — FAQ, ideas, agenda/speakers, E4P signatories, endorsements, and community calls (events no longer come from Notion). Images are direct Notion-hosted URLs (no proxy/cache — that worker was removed), expire after ~1hr, client falls back to `/images/default.jpg` on load error. See `docs/NOTION.md`.
- **ProjectHub** (external service) — `src/pages/api/projects.ts` calls `projecthub.techforpalestine.org/api/public/projects` directly and is fetched client-side via `/api/projects` by both `ProjectsDirectory.tsx` and `ProjectsNew.tsx`. `src/pages/api/project-proxy.ts` is unrelated — it's a generic authenticated proxy used only by the volunteer/incubator application forms. See `docs/PROJECTS.md`.
- **Cloudflare KV** — `DROPPED_CONVERSIONS` namespace (bound in `wrangler.toml`) is used for conversion tracking, surfaced at `src/pages/admin/conversions.astro` and `src/pages/api/admin/conversion-stats.ts`.
- **Content collections** (`src/content/config.ts`) — currently empty (`collections = {}`); older docs referencing `content/ideas` and `content/projects` markdown collections are stale — check `src/content/` before relying on this.
Expand Down
5 changes: 4 additions & 1 deletion DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,13 @@ The site deploys automatically to Cloudflare Pages on push to `main`.

Set the following in the Cloudflare Pages dashboard (Production and Preview are separate — set both). See `.env.example` for the canonical list; grouped here by subsystem (docs in parens). For how `.env`/`.dev.vars`/the dashboard interact locally, and a full audit of what's actually wired up, see [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md).

**Events** ([docs/EVENTS.md](docs/EVENTS.md))

- `EVENTS_ICS_URL` — public ICS calendar feed URL (includes an auth token as a query param); fetched server-side only

**Notion** ([docs/NOTION.md](docs/NOTION.md))

- `NOTION_SECRET` — shared integration token
- `NOTION_DB_ID` — Events database ID
- `NOTION_SIGNATORIES_DB_ID`, `NOTION_FAQ_DB_ID`, `NOTION_IDEAS_DB_ID`, `NOTION_AGENDA_DB_ID`, `NOTION_ENDORSEMENTS_DB_ID`, `NOTION_COMMUNITY_CALLS_DB_ID`
- `NOTION_SPEAKERS_DB_ID` — listed for completeness; not currently read by any route

Expand Down
35 changes: 35 additions & 0 deletions almanac/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
title: CodeAlmanac Wiki
topics: [concepts]
sources: []
---

# CodeAlmanac Wiki

This is the living wiki for this repository. It records the durable knowledge
the code cannot say: decisions, flows, invariants, incidents, gotchas, and
project context that future agents should not rediscover from scratch.

## Notability Bar

Write a page when it preserves non-obvious knowledge that will help a future
agent work safely in this codebase.

Good pages explain:

- a decision that took research or trial-and-error
- a cross-file flow
- an invariant or gotcha not visible from one file
- an external dependency as this repo uses it
- a product or operational constraint that shapes future work

Do not write pages that restate nearby code.

## Topic Taxonomy

Topics live in `topics.yaml`. Pages are Markdown files directly under
`almanac/`, including nested folders.

## Links

Use normal Markdown links between pages. Put file evidence in `sources:`.
55 changes: 55 additions & 0 deletions almanac/architecture/api-route-conventions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
title: "Shared API Route Conventions"
summary: "Every src/pages/api/*.ts route disables prerendering, resolves secrets through getEnv, checks Origin before parsing a write request's body, and reports errors generically through a shared Sentry helper."
topics: [architecture]
sources:
- id: api-doc
type: file
path: docs/API.md
- id: donation-complete
type: file
path: src/pages/api/donation-complete.ts
- id: endorsement-request
type: file
path: src/pages/api/endorsement-request.ts
- id: report-error
type: file
path: src/lib/report-error.ts
- id: origin-util
type: file
path: src/utils/origin.ts
- id: pipe-route
type: file
path: src/pages/api/pipe.ts
- id: sentry-webhook
type: file
path: src/pages/api/sentry-webhook.ts
- id: get-env
type: file
path: src/utils/getEnv.ts
---

Every route under `src/pages/api/` talks to a different upstream — Notion, ProjectHub, EmailOctopus, Plausible, the Hub API, Mattermost — but they all follow the same handful of rules, stated as the "Conventions to follow for new routes" in `docs/API.md` [@api-doc] and consistently applied in the route files themselves. A route that skips one of these — forgets the `Origin` check, reads `process.env` directly, or lets a caught error reach the client — is the kind of gap the repository's automated security review is specifically watching for, so this shared shape is also the contract new routes are expected to match; see [Shared Security Hardening Baseline](../decisions/security-hardening-baseline) for how that review gate came to exist and [Add an API Route](../guides/add-an-api-route) for the step-by-step version of what follows.

## `prerender = false` and `getEnv` instead of `process.env`

Every API route sets `export const prerender = false;` [@donation-complete][@endorsement-request], which tells Astro's server-output build not to try to render the route at build time — it must run per request, since it reads the incoming request and calls external services. Routes also resolve every secret and configuration value through `getEnv(name, locals)` rather than reading `process.env` directly [@donation-complete][@endorsement-request]; `getEnv` checks the Cloudflare runtime environment (`locals.runtime.env`), then Astro's build-time `import.meta.env`, then `process.env`, in that order [@get-env]. On the deployed Cloudflare Pages runtime only the first tier is populated, so a route that reads `process.env.SOME_SECRET` directly instead of going through `getEnv` would silently get `undefined` in production. The three-tier resolution model itself, and why it exists, is covered on [Environment Variable Resolution](../concepts/environment-variable-resolution).

## Origin allowlisting before the body is touched

Public write endpoints — anything that accepts a `POST` from a browser — validate the request's `Origin` header before doing anything else, including before parsing the JSON body. The shared logic lives in `src/utils/origin.ts`: `isAllowedOrigin(origin, policy)` returns `false` for a missing `Origin` header unless the policy explicitly opts in with `allowMissingOrigin`, checks the origin against an exact-match list (`policy.allowedOrigins`, defaulting to just `https://techforpalestine.org`), and falls back to a hostname-suffix check against `policy.allowedSuffixes` [@origin-util]. `corsHeaders(origin, methods)` builds the matching `Access-Control-Allow-*` headers for the response [@origin-util].

Different routes widen this policy by different amounts, and the width tracks how much damage a forged request could do:

- `endorsement-request.ts` calls `isAllowedOrigin(origin)` with no policy argument, so it falls back to the single-origin default — only `https://techforpalestine.org` is accepted [@endorsement-request].
- `donation-complete.ts` (and `membership-complete.ts`, the same pattern) builds an explicit `OriginPolicy` that adds `.website-aun.pages.dev` as an allowed suffix, so preview deploys on that specific Cloudflare Pages project can also call it, plus `localhost:4321` outside production builds [@donation-complete].
- `pipe.ts`, the server-side proxy in front of Plausible's ingest endpoint, allows any `.pages.dev` suffix and sets `allowMissingOrigin: true` [@pipe-route]. This is deliberately the widest policy in the codebase: `pipe.ts` only relays analytics events, so a forged call has no meaningful blast radius, and same-origin requests without an `Origin` header still need to work.
- `sentry-webhook.ts` inverts the pattern entirely — it rejects any request that carries an `Origin` header at all, on the reasoning that genuine server-to-server webhook calls from Sentry never send one [@sentry-webhook]. Instead of Origin checking, it authenticates via an HMAC signature in the `sentry-hook-signature` header, verified with a constant-time comparison against a value computed from `SENTRY_WEBHOOK_SECRET` [@sentry-webhook].

For write endpoints that do accept a body, `docs/API.md` also documents required-field presence checks, an email-format regex, URL fields validated with a `try { new URL(x) } catch`, and a 2000-character cap on free-text fields [@api-doc] — `endorsement-request.ts` runs all four checks, in that order, after the Origin check and before calling Notion [@endorsement-request]. The exact allowlist for every route is tabulated on the [API routes](../reference/api-routes) reference page.

## Errors: report to Sentry, return nothing specific to the client

Every route wraps its upstream call in a `try`/`catch` and, on failure, calls `reportError(error, { context: "route-name" })` from `src/lib/report-error.ts` [@donation-complete][@endorsement-request]. `reportError` itself just logs to the console and forwards the exception to Sentry inside `Sentry.withScope`, attaching whatever context object was passed [@report-error]. It does not flush anything — each route does that separately, immediately after calling `reportError`, with `ctx?.waitUntil(Promise.resolve(Sentry.flush(2000)))` [@donation-complete][@endorsement-request]. `ctx` comes from `locals.runtime.ctx`, the Cloudflare Workers execution context; `waitUntil` is necessary because a Worker is free to terminate the request as soon as the `Response` is returned, which can happen before Sentry's asynchronous network flush finishes — without `waitUntil`, error reports would be dropped intermittently. Whatever happened internally, the client only ever receives a fixed, generic message such as `"Failed to process request"` — never the caught error object, its message, or a stack trace [@donation-complete][@endorsement-request].

This combination — disable prerendering, resolve config through `getEnv`, gate writes on `Origin` before parsing, report through Sentry with an explicit flush, and never leak error detail to the caller — is what "matching repo conventions" means for a new route. It also underlies the integrations documented separately: the Notion-backed routes, the ProjectHub proxy, and the donation/membership conversion pipeline all sit on top of this same shape rather than reinventing it.
55 changes: 55 additions & 0 deletions almanac/architecture/integrations/donation-conversion-pipeline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
title: "The Donation and Membership Conversion Pipeline"
summary: "Donation and membership completions fan out to EmailOctopus and the Hub API, while a separate Plausible proxy tracks conversion goals and falls back to a Cloudflare KV store when Plausible silently drops an event, so the admin dashboard can merge both counts."
topics: [architecture, integrations, donations, analytics]
sources:
- id: donation-complete
type: file
path: src/pages/api/donation-complete.ts
- id: membership-complete
type: file
path: src/pages/api/membership-complete.ts
- id: pipe-route
type: file
path: src/pages/api/pipe.ts
- id: conversion-stats
type: file
path: src/pages/api/admin/conversion-stats.ts
- id: conversions-page
type: file
path: src/pages/admin/conversions.astro
- id: basic-auth
type: file
path: src/utils/basicAuth.ts
- id: wrangler-toml
type: file
path: wrangler.toml
- id: donations-doc
type: file
path: docs/DONATIONS.md
- id: origin-util
type: file
path: src/utils/origin.ts
---

A donation or membership payment on the site produces two independent, uncoordinated trails of data: side effects triggered by a client-side success callback, and an analytics conversion event fired separately through Plausible. Neither trail talks to the other while it runs — they are only brought back together afterward, on the admin conversion dashboard, by a route that reads both sources and merges them by date.

## Completion callbacks: EmailOctopus and the Hub

`src/pages/api/donation-complete.ts` and `src/pages/api/membership-complete.ts` are POST endpoints the QGIV donation widget's client-side success callback calls once a payment finishes [@donation-complete] [@donations-doc]. Both validate the request `Origin` against the production domain plus any `*.website-aun.pages.dev` preview deploy before touching the body, validate the email with a regex, and truncate first/last name fields to 200 characters [@donation-complete] [@membership-complete].

`donation-complete` does one thing past validation: if `EO_API_KEY` is configured, it subscribes the contact to an EmailOctopus list tagged `"donor"` [@donation-complete]. `membership-complete` does two things, run concurrently with `Promise.allSettled` so a failure in one doesn't block the other: it subscribes the same way but tagged `"member"`, and it calls the internal Hub API (`POST {HUB_API_URL}/api/auth/invite` with `{ email, type: "paid" }`, authorized by `HUB_API_KEY`) to invite the new member as a paid Hub user [@membership-complete]. `Promise.allSettled` here means neither the EmailOctopus tag nor the Hub invite can fail the other or fail the client-visible response — both routes return `{ success: true }` even if their downstream call errors, logging the failure through `reportError` instead of surfacing it to the browser [@donation-complete] [@membership-complete]. Neither route reads from or writes to the Cloudflare KV store described below; the KV fallback only exists on the separate analytics path.

## The Plausible proxy and its KV fallback

Conversion tracking is a second, unrelated request: the client fires a Plausible goal event (`Monthly-donate`, `One-time-donate`, or `Membership-complete`) through `POST /api/pipe`, which proxies it to `https://plausible.io/api/event` rather than the browser calling Plausible directly [@pipe-route]. Routing analytics through a same-origin endpoint means a browser extension or ad-blocker that specifically blocks third-party requests to `plausible.io` doesn't block the traffic, since from the browser's perspective the request never leaves the site's own origin. `pipe.ts` uses a looser origin policy than the completion callbacks — production plus *any* `*.pages.dev` suffix, since it only proxies analytics and carries no side effect worth restricting further [@pipe-route] [@origin-util].

Even routed through the same origin, Plausible can still silently drop an event server-side (bot filtering, for example) — the upstream response carries an `x-plausible-dropped: 1` header when that happens [@pipe-route]. `pipe.ts` checks that header, and if the event was dropped *and* its name is one of the three tracked conversion goals, it writes a fallback record into a Cloudflare KV namespace bound as `DROPPED_CONVERSIONS` in `wrangler.toml` [@pipe-route] [@wrangler-toml]. The key is `dropped:<date>:<time>:<random>`, and the stored value is deliberately minimal: event name, timestamp, a few non-PII props (`source`, `amount`, `membership_variant` when present), and boolean flags recording only *whether* an IP and user-agent were available on the original request — never the IP or user-agent values themselves [@pipe-route] [@donations-doc]. The KV write happens inside `ctx.waitUntil`, so it doesn't add latency to the response the browser is waiting on and can still complete after the response has already been sent.

## Merging both sources for the admin dashboard

`/api/admin/conversion-stats` is a `GET` route gated by HTTP Basic Auth: `isAuthorized`/`unauthorizedResponse` in `src/utils/basicAuth.ts` decode the `Authorization` header and compare the username and password against `ADMIN_USERNAME`/`ADMIN_PASSWORD` using a constant-time comparison, rather than a plain `===`, so the check can't leak timing information about how much of the credential matched [@basic-auth]. `src/pages/admin/conversions.astro` runs the identical `isAuthorized` check server-side before it even renders the page shell, so an unauthenticated request never receives the dashboard's HTML — the client-only React dashboard component is a second layer behind a page that's already gated [@conversions-page].

For a given date range, `conversion-stats.ts` runs three queries in parallel: live goal counts from the Plausible Stats API v2 (broken down further by `source` for the two donation goals, and by `membership_variant` for membership), a scan of every `dropped:*` KV key in that range with its own per-goal/per-source/per-day aggregation, and a second Plausible query for conversion detail breakdowns (amount, variant) [@conversion-stats]. It returns the Plausible numbers and the KV-derived numbers as separate arrays alongside a combined `details` array, rather than pre-summing them server-side — `src/components/ConversionDashboard.tsx` is what actually adds the two sources together per goal and per day when it renders the chart and summary cards. The dashboard exists specifically so ad-blocker-driven undercounting on the direct Plausible path doesn't make donation conversions look lower than they really are: the KV fallback recovers exactly the events that would otherwise have vanished.

This pipeline is one instance of the repository's [API route conventions](api-route-conventions) — `prerender = false`, `getEnv` for secrets, Origin checks before body parsing, generic client-facing errors, and `reportError` plus `Sentry.flush` on failure all appear identically across `donation-complete.ts`, `membership-complete.ts`, `pipe.ts`, and `conversion-stats.ts`. The Origin allowlist and Basic Auth choices here are part of the broader [security hardening baseline](security-hardening-baseline); the full set of environment variables this pipeline depends on (`EO_API_KEY`, `HUB_API_URL`, `HUB_API_KEY`, `PLAUSIBLE_API_KEY`, `ADMIN_USERNAME`, `ADMIN_PASSWORD`) is catalogued in the [environment variables reference](environment-variables).
Loading
Loading