Add white paper drafting and build pipeline

.devcontainer/devcontainer.json

@@ -0,0 +1,17 @@
+{
+  "name": "Zeid Data Research",
+  "image": "mcr.microsoft.com/devcontainers/base:ubuntu",
+  "features": {
+    "ghcr.io/devcontainers/features/github-cli:1": {},
+    "ghcr.io/devcontainers/features/python:1": { "version": "3.12" }
+  },
+  "postCreateCommand": "bash .devcontainer/setup.sh",
+  "remoteEnv": {
+    "PATH": "${containerEnv:HOME}/.local/bin:${containerEnv:PATH}"
+  },
+  "customizations": {
+    "vscode": {
+      "extensions": ["James-Yu.latex-workshop"]
+    }
+  }
+}

.devcontainer/setup.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+# Codespace setup. Makes tectonic and the brand fonts ready so the build and
+# bootstrap scripts run with no further steps. Runs once when the Codespace is
+# created. Safe to re-run.
+mkdir -p "$HOME/.local/bin"
+export PATH="$HOME/.local/bin:$PATH"
+bash tools/scripts/zd_whitepaper_bootstrap.sh --with-tectonic
+echo ""
+echo "Codespace ready."
+echo "Build a paper:  tools/scripts/zd_whitepaper_build.sh research/<section>/<slug>"
+echo "Draft a paper:  feed tools/whitepaper-agent.md to your command-line agent"

docs/whitepaper-pipeline.md

@@ -0,0 +1,60 @@
+# White paper pipeline
+
+Automated drafting and placement of white papers into `research/<section>/`,
+with a human pull request gate. No paper reaches `main` without review.
+
+## One time setup
+```
+tools/scripts/zd_whitepaper_bootstrap.sh --with-tectonic
+```
+Vendors the OFL brand fonts (DM Serif Display, Inter) into
+`templates/whitepaper/fonts/`, writes checksums, and installs `tectonic`.
+The fonts are local only and are not committed. Re-run on any new machine.
+
+## Drafting a paper
+Feed the operating prompt at `tools/whitepaper-agent.md` to your command-line
+research agent, with either a topic or a path to source material:
+```
+<topic string>            # generate mode: research and cite
+path/to/notes.md          # transform mode: build from existing material
+```
+The agent classifies the section, drafts with cited sources, builds the PDF,
+and opens a PR into `main`. The agent never merges.
+
+Once a paper folder exists at `research/<section>/<slug>/`, you can build it,
+branch, commit, push, and open the PR in one command:
+```
+tools/scripts/zd_whitepaper_ship.sh research/<section>/<slug>
+```
+It reads the title and section from `meta.yml`, assembles the PR body from
+`sources.json`, refuses to run on a dirty tree, and never merges.
+
+## Drafting in a Codespace (no local clone)
+
+If the repo is too large to keep on your machine, draft in the cloud. On the
+repo, use the green Code button, open the Codespaces tab, and create a Codespace.
+The devcontainer in `.devcontainer/` installs `tectonic` and vendors the brand
+fonts on first create, so the build and bootstrap scripts work with no further
+setup. From the Codespace terminal:
+```
+tools/scripts/zd_whitepaper_build.sh research/<section>/<slug>
+```
+The repo lives in the cloud, nothing is downloaded to your disk. To make startup
+instant instead of a one minute setup, enable Codespaces prebuilds for the repo
+in Settings, Codespaces.
+
+## Building a paper by hand
+```
+tools/scripts/zd_whitepaper_build.sh research/<section>/<slug>
+```
+Detects the toolchain (tectonic, then latexmk, then xelatex), fails loud if
+none is present. PDF lands next to the source.
+
+## Layout contract
+- Papers live at `research/<section>/<slug>/` (exactly three levels deep).
+- Each paper folder holds `paper.tex`, `paper.pdf`, `sources.json`, `meta.yml`.
+- `paper.tex` sets `\def\zdroot{../../../}` and includes the house preamble at
+  `templates/whitepaper/zd-whitepaper.tex`.
+- Every claim maps to a real URL in `sources.json`, or it sits in the mandatory
+  "Limitations and Unverified Claims" section. Link-check CI enforces resolvable
+  URLs.

research/.gitignore

@@ -0,0 +1,5 @@
+**/*.aux
+**/*.log
+**/*.out
+**/*.fls
+**/*.fdb_latexmk

templates/whitepaper/fonts/.gitignore

@@ -0,0 +1,3 @@
+*.ttf
+*.otf
+SHA256SUMS

templates/whitepaper/fonts/README.md

@@ -0,0 +1,8 @@
+# Vendored fonts (SIL Open Font License 1.1)
+
+- DMSerifDisplay-Regular.ttf, DM Serif Display, Colophon Foundry, OFL 1.1.
+- Inter-Regular.ttf, Inter, Rasmus Andersson, OFL 1.1. Instanced to a static
+  weight (wght 400, opsz 14) from the upstream variable font.
+
+Both fonts are redistributable under the OFL. Checksums in SHA256SUMS.
+Loaded by path so PDF output is identical on every machine.

templates/whitepaper/paper.tex.template

@@ -0,0 +1,22 @@
+\documentclass[11pt]{article}
+% Papers live at research/<section>/<slug>/, exactly three levels under the repo
+% root, so \zdroot is ../../../ . Adjust only if you place a paper elsewhere.
+\def\zdroot{../../../}
+\input{\zdroot templates/whitepaper/zd-whitepaper.tex}
+\begin{document}
+\zdtitle{PAPER TITLE}{One paragraph abstract.}{YYYY-MM-DD}
+
+\section{Background}
+Body text with a cited claim.\,[1]
+
+\section{Analysis}
+More body. Every factual claim points to a real source.\,[2]
+
+\section{Limitations and Unverified Claims}
+What the evidence does not yet support. Mandatory section, even if short.
+
+\section{References}
+\begin{enumerate}[label={[\arabic*]}]
+  \item Source title. \href{https://example.com}{example.com}. Accessed YYYY-MM-DD.
+\end{enumerate}
+\end{document}

templates/whitepaper/zd-whitepaper.tex

@@ -0,0 +1,54 @@
+% Zeid Data Research white paper preamble. XeLaTeX / tectonic.
+% Build from the repo root. Fonts vendored at templates/whitepaper/fonts/.
+% Fonts: OFL. DM Serif Display (headings), Inter (body).
+\usepackage{geometry}
+\geometry{a4paper, margin=1in, headheight=14pt}
+\usepackage{xcolor}
+\definecolor{zdCarbon}{HTML}{1A1A1A}
+\definecolor{zdSlate}{HTML}{4A4A4A}
+\definecolor{zdOffWhite}{HTML}{F4F1EA}
+\definecolor{zdAmber}{HTML}{E0A800}
+
+\usepackage{fontspec}
+% Vendored fonts. Build runs from the repo root, so Path is repo-root-relative.
+% Bold and italic for body text are synthesized from the single regular weight.
+% If a font file is missing, the build fails loud. Run the bootstrap to vendor
+% fonts before building. Do not ship a paper with a silently broken font.
+\setmainfont{Inter-Regular.ttf}[
+  Path = \zdroot templates/whitepaper/fonts/ ,
+  AutoFakeBold = 2.0 ,
+  AutoFakeSlant = 0.2 ]
+\newfontfamily\zdHead{DMSerifDisplay-Regular.ttf}[
+  Path = \zdroot templates/whitepaper/fonts/ ]
+
+\usepackage{titlesec}
+\titleformat{\section}{\zdHead\Large\color{zdCarbon}}{\thesection}{0.6em}{}[{\color{zdAmber}\titlerule}]
+\titleformat{\subsection}{\zdHead\large\color{zdCarbon}}{\thesubsection}{0.5em}{}
+\titlespacing*{\section}{0pt}{1.4em}{0.5em}
+
+\usepackage{fancyhdr}
+\pagestyle{fancy}\fancyhf{}
+\fancyfoot[L]{\footnotesize\color{zdSlate}Zeid Data Research Lab}
+\fancyfoot[C]{\footnotesize\color{zdSlate}Evidence first}
+\fancyfoot[R]{\footnotesize\color{zdSlate}\thepage}
+\renewcommand{\headrulewidth}{0pt}
+\renewcommand{\footrulewidth}{0.4pt}
+\renewcommand{\footrule}{\hbox to\headwidth{\color{zdAmber}\leaders\hrule height \footrulewidth\hfill}}
+
+\usepackage{microtype}
+\usepackage{booktabs}
+\usepackage{enumitem}
+\usepackage[hidelinks]{hyperref}
+\hypersetup{colorlinks=true, linkcolor=zdCarbon, urlcolor=zdAmber, citecolor=zdAmber}
+\usepackage{csquotes}
+
+\color{zdCarbon}
+
+% Title block macro. Usage: \zdtitle{Title}{Abstract text}{YYYY-MM-DD}
+\newcommand{\zdtitle}[3]{%
+  \thispagestyle{fancy}
+  \noindent\colorbox{zdCarbon}{\parbox{\dimexpr\textwidth-2\fboxsep}{%
+    \vspace{4pt}{\zdHead\LARGE\color{zdOffWhite} #1}\\[4pt]
+    {\footnotesize\color{zdAmber}Zeid Data Research Lab \quad\textbullet\quad #3}\vspace{4pt}}}%
+  \vspace{1.2em}\par\noindent{\color{zdAmber}\rule{\textwidth}{1.5pt}}\vspace{0.8em}\par
+  \noindent\textbf{Abstract.} #2\par\vspace{1.2em}}

tools/scripts/zd_whitepaper_bootstrap.sh

@@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+# One-time bootstrap for the Zeid Data white paper pipeline.
+# Vendors the OFL brand fonts, records checksums, installs tectonic if asked.
+# Run from anywhere inside the repo. Idempotent.
+# Usage: tools/scripts/zd_whitepaper_bootstrap.sh [--with-tectonic]
+
+LOG=/tmp/zd_whitepaper_bootstrap.log
+: > "$LOG"
+red()  { if [ -t 2 ]; then printf '\033[31m%s\033[0m\n' "$*" >&2; else printf '%s\n' "$*" >&2; fi; }
+say()  { printf '%s\n' "$*" | tee -a "$LOG"; }
+
+ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
+if [ -z "$ROOT" ] || [ ! -f "$ROOT/templates/whitepaper/zd-whitepaper.tex" ]; then
+  d=$(pwd)
+  while [ "$d" != "/" ]; do
+    [ -f "$d/templates/whitepaper/zd-whitepaper.tex" ] && { ROOT="$d"; break; }
+    d=$(dirname "$d")
+  done
+fi
+if [ -z "$ROOT" ] || [ ! -f "$ROOT/templates/whitepaper/zd-whitepaper.tex" ]; then
+  red "FAIL: cannot find the pipeline root (templates/whitepaper/). Run from inside the kit or repo."
+  echo exit_code=2; exit 2
+fi
+cd "$ROOT" || { red "FAIL: cannot cd to root"; echo exit_code=2; exit 2; }
+say "root=$ROOT"
+
+FD=templates/whitepaper/fonts
+mkdir -p "$FD"
+
+validate() { [ -f "$1" ] && [ "$(stat -c%s "$1" 2>/dev/null || echo 0)" -gt 20000 ]; }
+
+# DM Serif Display Regular (static TTF, OFL)
+if validate "$FD/DMSerifDisplay-Regular.ttf"; then
+  say "OK   DMSerifDisplay-Regular.ttf already vendored"
+else
+  if curl -fsSL --retry 2 --max-time 90 \
+    "https://github.com/google/fonts/raw/main/ofl/dmserifdisplay/DMSerifDisplay-Regular.ttf" \
+    -o "$FD/DMSerifDisplay-Regular.ttf" && validate "$FD/DMSerifDisplay-Regular.ttf"; then
+    say "OK   DMSerifDisplay-Regular.ttf fetched"
+  else
+    red "FAIL: could not vendor DM Serif Display"; echo exit_code=5; exit 5
+  fi
+fi
+
+# Inter Regular (instanced from the OFL variable font for a clean static weight)
+if validate "$FD/Inter-Regular.ttf"; then
+  say "OK   Inter-Regular.ttf already vendored"
+else
+  if ! python3 -c "import fontTools" >/dev/null 2>&1; then
+    say "installing fonttools"
+    pip install fonttools --break-system-packages -q >>"$LOG" 2>&1 || pip install --user fonttools -q >>"$LOG" 2>&1
+  fi
+  if curl -fsSL --retry 2 --max-time 120 \
+    "https://github.com/google/fonts/raw/main/ofl/inter/Inter%5Bopsz,wght%5D.ttf" \
+    -o /tmp/Inter-VF.ttf; then
+    python3 - "$FD" <<'PY' >>"$LOG" 2>&1
+import sys
+from fontTools.ttLib import TTFont
+from fontTools.varLib.instancer import instantiateVariableFont
+fd=sys.argv[1]
+f=TTFont('/tmp/Inter-VF.ttf')
+instantiateVariableFont(f, {'wght':400, 'opsz':14}, inplace=True)
+f.save(fd+'/Inter-Regular.ttf')
+PY
+    if validate "$FD/Inter-Regular.ttf"; then
+      say "OK   Inter-Regular.ttf instanced (wght=400)"
+    else
+      red "FAIL: could not instance Inter"; echo exit_code=6; exit 6
+    fi
+  else
+    red "FAIL: could not fetch Inter variable font"; echo exit_code=6; exit 6
+  fi
+fi
+
+sha256sum "$FD"/*.ttf | tee "$FD/SHA256SUMS" >>"$LOG"
+say "checksums written to $FD/SHA256SUMS"
+
+if [ "$1" = "--with-tectonic" ]; then
+  if command -v tectonic >/dev/null 2>&1; then
+    say "OK   tectonic already installed: $(tectonic --version)"
+  else
+    say "installing tectonic"
+    curl -fsSL --retry 2 --max-time 180 "https://drop-sh.fullyjustified.net" -o /tmp/install-tectonic.sh >>"$LOG" 2>&1
+    DEST="${HOME}/.local/bin"; mkdir -p "$DEST"
+    ( cd "$DEST" && sh /tmp/install-tectonic.sh ) >>"$LOG" 2>&1
+    if [ -x "$DEST/tectonic" ]; then
+      say "OK   tectonic installed to $DEST. Ensure $DEST is on PATH."
+    else
+      red "WARN: tectonic install did not land in $DEST. Install it manually."
+    fi
+  fi
+fi
+
+say "bootstrap complete"
+echo exit_code=0

tools/scripts/zd_whitepaper_build.sh

@@ -0,0 +1,72 @@
+#!/usr/bin/env bash
+# Build a Zeid Data white paper to PDF. Run from anywhere inside the repo.
+# Usage: tools/scripts/zd_whitepaper_build.sh research/<section>/<slug>
+#    or: tools/scripts/zd_whitepaper_build.sh research/<section>/<slug>/paper.tex
+# Resolves the repo root itself, detects the toolchain, fails loud if none.
+
+LOG=/tmp/zd_whitepaper_build.log
+: > "$LOG"
+
+red()  { if [ -t 2 ]; then printf '\033[31m%s\033[0m\n' "$*" >&2; else printf '%s\n' "$*" >&2; fi; }
+say()  { printf '%s\n' "$*" | tee -a "$LOG"; }
+
+ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
+if [ -z "$ROOT" ] || [ ! -f "$ROOT/templates/whitepaper/zd-whitepaper.tex" ]; then
+  d=$(pwd)
+  while [ "$d" != "/" ]; do
+    [ -f "$d/templates/whitepaper/zd-whitepaper.tex" ] && { ROOT="$d"; break; }
+    d=$(dirname "$d")
+  done
+fi
+if [ -z "$ROOT" ] || [ ! -f "$ROOT/templates/whitepaper/zd-whitepaper.tex" ]; then
+  red "FAIL: cannot find the pipeline root (templates/whitepaper/). Run from inside the kit or repo."
+  echo exit_code=2; exit 2
+fi
+cd "$ROOT" || { red "FAIL: cannot cd to root $ROOT"; echo exit_code=2; exit 2; }
+say "root=$ROOT"
+
+ARG="$1"
+if [ -z "$ARG" ]; then red "FAIL: pass the paper dir or paper.tex path"; echo exit_code=2; exit 2; fi
+case "$ARG" in
+  *.tex) TEX="$ARG" ;;
+  *)     TEX="${ARG%/}/paper.tex" ;;
+esac
+if [ ! -f "$TEX" ]; then red "FAIL: $TEX not found"; echo exit_code=2; exit 2; fi
+OUTDIR=$(dirname "$TEX")
+say "paper=$TEX"
+say "outdir=$OUTDIR"
+
+if [ ! -f templates/whitepaper/fonts/Inter-Regular.ttf ] || [ ! -f templates/whitepaper/fonts/DMSerifDisplay-Regular.ttf ]; then
+  red "FAIL: brand fonts missing. Run tools/scripts/zd_whitepaper_bootstrap.sh first."
+  echo exit_code=3; exit 3
+fi
+
+if command -v tectonic >/dev/null 2>&1; then
+  TOOL=tectonic
+  say "DECISION: toolchain=tectonic"
+  tectonic --chatter minimal --outdir "$OUTDIR" "$TEX" >>"$LOG" 2>&1; rc=$?
+elif command -v latexmk >/dev/null 2>&1; then
+  TOOL=latexmk
+  say "DECISION: toolchain=latexmk"
+  latexmk -xelatex -interaction=nonstopmode -outdir="$OUTDIR" "$TEX" >>"$LOG" 2>&1; rc=$?
+elif command -v xelatex >/dev/null 2>&1; then
+  TOOL=xelatex
+  say "DECISION: toolchain=xelatex"
+  ( cd "$OUTDIR" && xelatex -interaction=nonstopmode "$(basename "$TEX")" && xelatex -interaction=nonstopmode "$(basename "$TEX")" ) >>"$LOG" 2>&1; rc=$?
+else
+  red "DECISION: toolchain=MISSING"
+  red "FAIL: no LaTeX engine found. Install tectonic:"
+  red "  curl -fsSL https://drop-sh.fullyjustified.net | sh   then move ./tectonic into PATH"
+  echo exit_code=4; exit 4
+fi
+
+PDF="$OUTDIR/$(basename "${TEX%.tex}").pdf"
+if [ "$rc" -eq 0 ] && [ -f "$PDF" ]; then
+  pages=$(command -v pdfinfo >/dev/null 2>&1 && pdfinfo "$PDF" 2>/dev/null | awk '/^Pages:/{print $2}')
+  say "BUILD OK toolchain=$TOOL pdf=$PDF pages=${pages:-unknown}"
+  echo exit_code=0
+else
+  red "BUILD FAILED toolchain=$TOOL rc=$rc"
+  red "last errors:"; grep -i "error" "$LOG" | tail -5 >&2
+  echo exit_code=1; exit 1
+fi