diff --git a/docs/gotchas.md b/docs/gotchas.md new file mode 100644 index 0000000..3e13a88 --- /dev/null +++ b/docs/gotchas.md @@ -0,0 +1,78 @@ +# Gotchas & sharp edges + +The non-obvious things about BAMMM that bite people. If a conversion "loses" a +field or a spec doesn't do what you expect, start here. + +## SPLAT field names are camelCase + +SPLAT is parsed as YAML-over-JSON, so field names follow the **camelCase** json +tags: `cpusPerTask`, `memoryPerTask`, `workingDir`, `maxRetries`, `nodeSelector`, +`clientId`, … — **not** `cpus_per_task` / snake_case. + +The parser is lenient: an unknown field (including a snake_case misspelling) is +**silently ignored**, not rejected. So a typo doesn't error — it just quietly +drops that field from the job. If a conversion is missing something you set, +check the casing first. + +Guard rails: + +- Validate a SPLAT file against `schema/splat.schema.json` (strict — unknown + fields fail) before trusting it. +- `bammm convert --from splat --to splat ` round-trips through the IR; + anything you wrote that comes back missing was dropped. + +## Translation is lossy on purpose + +No two schedulers express the same things. BAMMM's contract is *honesty*, not +magic: + +- Concepts with no equivalent on the target are **dropped with a warning** or + **approximated** (see [SPEC.md § Known Lossy Translations](../SPEC.md#known-lossy-translations)). +- Anything BAMMM can't map to a first-class field is preserved verbatim under + `extensions.` so it round-trips back to the *same* scheduler — but + it does **not** cross to a different one. +- Run any conversion with `--report` to see how much of your corpus lands in + `extensions.*` (a proxy for translation loss). + +## Priority: direction and range + +Schedulers disagree on both the numeric range and the *direction* of priority +(some treat a lower number as higher priority, nice-style). BAMMM normalizes +every native priority onto one canonical band — **higher always means higher +priority** — and denormalizes on the way out. See `internal/splat/priority.go`. + +- The canonical band is `0–1000` by default. Widen it with + `bammm convert --priority-range MIN:MAX` (e.g. `0:100000`) to reduce rounding + loss when round-tripping schedulers with large native ranges. +- K8s schedulers prefer a named `priorityClass` over a number; that takes + precedence where both exist. + +## Container and script can coexist + +A SPLAT job may set **both** `execution.container` (an OCI image) and +`execution.script` (a shell script). This is intentional: K8s emitters use the +container, HPC emitters use the script. Set both if you want one spec that runs +everywhere; set only the one you have otherwise. + +## Multi-role jobs become a JobSet on K8s + +A job with `spec.tasks` (multiple roles) can't fit one `batch/v1` Job, so the +Kueue and YuniKorn emitters produce a **JobSet** (`jobset.x-k8s.io`). That means +the target cluster needs the JobSet controller installed. Supporting objects +some emitters print (a Kueue `LocalQueue`, a script `ConfigMap`) are marked as +required manual prerequisites — read the comments in the output. + +## Install channels + +- **Homebrew** ships as a **cask** and is effectively macOS-oriented. On Linux, + use the `.deb`/`.rpm`/`.apk` from the release, the release tarball, or + `go install`. +- The **container image** is on GHCR: `ghcr.io/insightsoftmax/bammm`. + +## Corpus specs aren't in the repo + +`testdata/corpus/**` holds third-party job specs scraped from GitHub and is +**gitignored** for licensing reasons — only the `manifest.json` provenance files +are committed. Regenerate the actual specs with `make corpus SCHED=` (needs +a `GITHUB_TOKEN`). See the scraper header for the query-quoting and rate-limit +gotchas that make GitHub code search cooperate. diff --git a/docs/translation-rules.md b/docs/translation-rules.md new file mode 100644 index 0000000..dab515f --- /dev/null +++ b/docs/translation-rules.md @@ -0,0 +1,91 @@ +# User-defined translation rules (design — planned) + +> **Status: proposed, not yet implemented.** This document is the design for a +> feature that lets users adjust translations *without patching BAMMM's source*. +> It exists so the feature is specified before it is built; nothing here works +> today. + +## Problem + +BAMMM maps native fields to SPLAT and back. When it meets something it doesn't +anticipate — a site-specific resource, a custom directive, a field a given +scheduler version added — the current options are: + +1. it lands in `extensions.` (preserved for round-trip, but **not** + translated to a *different* scheduler), or +2. it's dropped. + +Neither lets a user say "when converting X → Y, turn *this* into *that*" without +editing Go and rebuilding. + +## Approach: a declarative rules file + +A user supplies a rules file and points BAMMM at it: + +```sh +bammm convert --from slurm --to pbs --rules my-rules.yaml job.sh +``` + +Rules operate on the **SPLAT IR** (after parse, before emit), so one rule set +works regardless of the source/target pair. Each rule is a `match` → `action`. + +### Sketch + +```yaml +apiVersion: bammm.io/rules/v1alpha1 +rules: + # Promote a preserved Slurm extension into a first-class field for the target. + - when: + from: slurm + has: extensions.slurm.switches # dotted path into the SPLAT IR + set: + placement.constraint: "network=switch" # target-agnostic SPLAT field + + # Rename / remap a value. + - when: + has: schedule.qos + equals: { schedule.qos: debug } + set: + schedule.queue: debug-queue + + # Drop with an explicit warning instead of silent loss. + - when: + to: pbs + has: extensions.htcondor.requirements + drop: + warn: "HTCondor requirements expression cannot be expressed in PBS" +``` + +Action verbs (initial set): `set`, `rename`, `drop` (with `warn`), `default` +(set only if absent). + +### Match expressions + +Start with simple structural predicates (`has`, `equals`, `from`, `to`). If that +proves too limited, add an embedded **CEL** (`cel-go`) expression for conditions +and computed values — sandboxed, no code execution, e.g.: + +```yaml + - when: + expr: 'has(spec.resources.gpu) && spec.resources.gpu.count > 4' + set: + schedule.queue: gpu-large +``` + +Starlark or external pre/post hooks are a possible later escalation for +transforms that a declarative form can't express, but the declarative rules file +is the primary interface. + +## Open questions + +- Precedence when multiple rules match (first-wins vs. all-apply). +- Whether rules may write into `extensions.*` (target-specific escape hatch). +- How rule-driven changes surface in `--report` and the lossiness accounting. +- Packaging: site-wide default rules vs. per-invocation `--rules`. + +## When built + +This feature must ship with: a `SPEC` section (or its own `SPEC-rules.md`), CLI +help for `--rules`, worked examples under `examples/rules/`, and a note in +[gotchas.md](gotchas.md) about rule precedence. Update this document from +"proposed" to the real reference at that point.