Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions docs/gotchas.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Gotchas & sharp edges

The non-obvious things about BAMMM that bite people. If a conversion "loses" a
field or a spec doesn't do what you expect, start here.

## SPLAT field names are camelCase

SPLAT is parsed as YAML-over-JSON, so field names follow the **camelCase** json
tags: `cpusPerTask`, `memoryPerTask`, `workingDir`, `maxRetries`, `nodeSelector`,
`clientId`, … — **not** `cpus_per_task` / snake_case.

The parser is lenient: an unknown field (including a snake_case misspelling) is
**silently ignored**, not rejected. So a typo doesn't error — it just quietly
drops that field from the job. If a conversion is missing something you set,
check the casing first.

Guard rails:

- Validate a SPLAT file against `schema/splat.schema.json` (strict — unknown
fields fail) before trusting it.
- `bammm convert --from splat --to splat <file>` round-trips through the IR;
anything you wrote that comes back missing was dropped.

## Translation is lossy on purpose

No two schedulers express the same things. BAMMM's contract is *honesty*, not
magic:

- Concepts with no equivalent on the target are **dropped with a warning** or
**approximated** (see [SPEC.md § Known Lossy Translations](../SPEC.md#known-lossy-translations)).
- Anything BAMMM can't map to a first-class field is preserved verbatim under
`extensions.<scheduler>` so it round-trips back to the *same* scheduler — but
it does **not** cross to a different one.
- Run any conversion with `--report` to see how much of your corpus lands in
`extensions.*` (a proxy for translation loss).

## Priority: direction and range

Schedulers disagree on both the numeric range and the *direction* of priority
(some treat a lower number as higher priority, nice-style). BAMMM normalizes
every native priority onto one canonical band — **higher always means higher
priority** — and denormalizes on the way out. See `internal/splat/priority.go`.

- The canonical band is `0–1000` by default. Widen it with
`bammm convert --priority-range MIN:MAX` (e.g. `0:100000`) to reduce rounding
loss when round-tripping schedulers with large native ranges.
- K8s schedulers prefer a named `priorityClass` over a number; that takes
precedence where both exist.

## Container and script can coexist

A SPLAT job may set **both** `execution.container` (an OCI image) and
`execution.script` (a shell script). This is intentional: K8s emitters use the
container, HPC emitters use the script. Set both if you want one spec that runs
everywhere; set only the one you have otherwise.

## Multi-role jobs become a JobSet on K8s

A job with `spec.tasks` (multiple roles) can't fit one `batch/v1` Job, so the
Kueue and YuniKorn emitters produce a **JobSet** (`jobset.x-k8s.io`). That means
the target cluster needs the JobSet controller installed. Supporting objects
some emitters print (a Kueue `LocalQueue`, a script `ConfigMap`) are marked as
required manual prerequisites — read the comments in the output.

## Install channels

- **Homebrew** ships as a **cask** and is effectively macOS-oriented. On Linux,
use the `.deb`/`.rpm`/`.apk` from the release, the release tarball, or
`go install`.
- The **container image** is on GHCR: `ghcr.io/insightsoftmax/bammm`.

## Corpus specs aren't in the repo

`testdata/corpus/**` holds third-party job specs scraped from GitHub and is
**gitignored** for licensing reasons — only the `manifest.json` provenance files
are committed. Regenerate the actual specs with `make corpus SCHED=<name>` (needs
a `GITHUB_TOKEN`). See the scraper header for the query-quoting and rate-limit
gotchas that make GitHub code search cooperate.
91 changes: 91 additions & 0 deletions docs/translation-rules.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# User-defined translation rules (design — planned)

> **Status: proposed, not yet implemented.** This document is the design for a
> feature that lets users adjust translations *without patching BAMMM's source*.
> It exists so the feature is specified before it is built; nothing here works
> today.

## Problem

BAMMM maps native fields to SPLAT and back. When it meets something it doesn't
anticipate — a site-specific resource, a custom directive, a field a given
scheduler version added — the current options are:

1. it lands in `extensions.<scheduler>` (preserved for round-trip, but **not**
translated to a *different* scheduler), or
2. it's dropped.

Neither lets a user say "when converting X → Y, turn *this* into *that*" without
editing Go and rebuilding.

## Approach: a declarative rules file

A user supplies a rules file and points BAMMM at it:

```sh
bammm convert --from slurm --to pbs --rules my-rules.yaml job.sh
```

Rules operate on the **SPLAT IR** (after parse, before emit), so one rule set
works regardless of the source/target pair. Each rule is a `match` → `action`.

### Sketch

```yaml
apiVersion: bammm.io/rules/v1alpha1
rules:
# Promote a preserved Slurm extension into a first-class field for the target.
- when:
from: slurm
has: extensions.slurm.switches # dotted path into the SPLAT IR
set:
placement.constraint: "network=switch" # target-agnostic SPLAT field

# Rename / remap a value.
- when:
has: schedule.qos
equals: { schedule.qos: debug }
set:
schedule.queue: debug-queue

# Drop with an explicit warning instead of silent loss.
- when:
to: pbs
has: extensions.htcondor.requirements
drop:
warn: "HTCondor requirements expression cannot be expressed in PBS"
```

Action verbs (initial set): `set`, `rename`, `drop` (with `warn`), `default`
(set only if absent).

### Match expressions

Start with simple structural predicates (`has`, `equals`, `from`, `to`). If that
proves too limited, add an embedded **CEL** (`cel-go`) expression for conditions
and computed values — sandboxed, no code execution, e.g.:

```yaml
- when:
expr: 'has(spec.resources.gpu) && spec.resources.gpu.count > 4'
set:
schedule.queue: gpu-large
```

Starlark or external pre/post hooks are a possible later escalation for
transforms that a declarative form can't express, but the declarative rules file
is the primary interface.

## Open questions

- Precedence when multiple rules match (first-wins vs. all-apply).
- Whether rules may write into `extensions.*` (target-specific escape hatch).
- How rule-driven changes surface in `--report` and the lossiness accounting.
- Packaging: site-wide default rules vs. per-invocation `--rules`.

## When built

This feature must ship with: a `SPEC` section (or its own `SPEC-rules.md`), CLI
help for `--rules`, worked examples under `examples/rules/`, and a note in
[gotchas.md](gotchas.md) about rule precedence. Update this document from
"proposed" to the real reference at that point.
Loading