diff --git a/CHANGELOG.md b/CHANGELOG.md index 72cb70d..8197a63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,11 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- Prefix the HashiCorp Vault login (both `userpass` and `kubernetes`) with the Vault Enterprise namespace derived from `setup.namespace` when it ends in the KV `data` segment (e.g. `admin/ns/data` → namespace `admin/ns`). Namespace-scoped credentials previously failed authentication with "access denied" because the login always targeted the root namespace. Prefixes whose `data` segment is in the middle (the default and any custom root-namespace subpath) are unaffected and keep logging in against the root namespace. +- Fix HashiCorp Vault parameters not being retrievable after being stored on a Vault Enterprise namespace. +- Report a clear error when storing a parameter in HashiCorp Vault fails, instead of silently reporting success. ### Changed -- Rename the HashiCorp Vault `namespace` setup field label to "Namespace" and clarify its description (namespace and path prefix where parameters are stored). +- Configure the HashiCorp Vault namespace and KV path prefix as separate fields. ## [0.2.0] - 2026-07-14 ### Added diff --git a/parameters/providers/hashicorp-vault/README.md b/parameters/providers/hashicorp-vault/README.md index c413ebc..3e90626 100644 --- a/parameters/providers/hashicorp-vault/README.md +++ b/parameters/providers/hashicorp-vault/README.md @@ -11,43 +11,49 @@ Provider config (from the nullplatform provider specification): | Field | Required | Description | |------------------------|---------------------|-----------------------------------------------------------------------------| | `setup.address` | yes | Vault HTTP(S) endpoint, e.g. `https://vault.example.com:8200`. | -| `setup.namespace` | no (default `secret/data/nullplatform`) | KV v2 path prefix parameters are stored under; on Vault Enterprise it may be prefixed with a Vault namespace. | +| `setup.namespace` | no (default root) | Vault Enterprise namespace the parameters live under, e.g. `admin/eks-null-alfa-136`. Leave empty for the root namespace (Vault OSS). | +| `setup.path_prefix` | no (default `secret/data/nullplatform`) | KV v2 path prefix (relative to the namespace) parameters are stored under. Must be the full `/data/`. | | `setup.auth_mode` | yes (default `userpass`) | Authentication mode: `userpass` or `kubernetes`. | | `setup.kubernetes_role`| when `kubernetes` | Vault Kubernetes auth role bound to the agent's ServiceAccount. | -The path prefix defaults to `secret/data/nullplatform` and is configurable via -`setup.namespace` (or the `VAULT_PATH_PREFIX` env var). It must include the KV v2 -`data/` segment. The auth mounts are fixed to Vault's defaults (`auth/userpass`, -`auth/kubernetes`). +`namespace` and `path_prefix` are **two independent axes**: + +- **`setup.namespace`** (env: `VAULT_NAMESPACE`) — the Vault Enterprise namespace. + It is applied as a URL prefix (`$VAULT_ADDR/v1//…`) on the login **and** + on every store/retrieve/delete request. Empty means the root namespace, so + non-Enterprise (Vault OSS) deployments simply leave it unset. +- **`setup.path_prefix`** (env: `VAULT_PATH_PREFIX`, default + `secret/data/nullplatform`) — the KV v2 path prefix relative to the namespace. It + must be the full `/data/`: the mount name, then the KV v2 `data` + segment, then an optional subpath. `store` embeds it in the `external_id`, so + stored references survive a later `path_prefix` change. + +The auth mounts are fixed to Vault's defaults (`auth/userpass`, `auth/kubernetes`). ### Vault Enterprise namespaces On Vault Enterprise, credentials and roles are scoped to a namespace, so the login -must target that namespace or Vault returns **access denied**. Encode the namespace -in `setup.namespace` as `/data` — i.e. the Vault namespace path -**terminated by the KV v2 `data` segment**. `setup` strips that trailing `/data` to -recover the namespace and prefixes both the `userpass` and `kubernetes` login with -it (`store`/`retrieve`/`delete` already reach it because the namespace is embedded -in the request path). - -| `setup.namespace` | Vault namespace for login | KV path used by store/retrieve | -|--------------------------------|---------------------------|--------------------------------| -| `secret/data/nullplatform` *(default)* | *(root, none)* | `secret/data/nullplatform/…` | -| `secret/data/team-x` | *(root, none)* | `secret/data/team-x/…` | -| `admin/ns/data` | `admin/ns` | `admin/ns/data/…` | -| `admin/team/sub/data` | `admin/team/sub` | `admin/team/sub/data/…` | - -> **Note:** the namespace is derived **only** from a trailing `/data` segment. A -> prefix whose `data` segment is in the middle (e.g. the default, or a custom -> subpath like `secret/data/team-x`) is treated as a plain KV mount + path in the -> **root** namespace — no namespace prefix is added to the login. This keeps -> non-Enterprise deployments working unchanged. To use a Vault namespace, the -> prefix must end at `/data` (no KV subpath after `data`). +must target that namespace or Vault returns **access denied**. Inside a namespace, a +KV v2 secret lives at `//data/` — the mount name sits +between the namespace and `data`, which is why the two are configured separately. + +| `setup.namespace` | `setup.path_prefix` | Full KV path used by store/retrieve | +|-------------------------|-----------------------------|-------------------------------------------| +| *(empty — root)* | `secret/data/nullplatform` *(default)* | `secret/data/nullplatform/…` | +| *(empty — root)* | `secret/data/team-x` | `secret/data/team-x/…` | +| `admin/ns` | `secret/data/nullplatform` | `admin/ns/secret/data/nullplatform/…` | +| `admin/team/sub` | `kv/data/params` | `admin/team/sub/kv/data/params/…` | + +> **Note:** changing `setup.namespace` after parameters exist is a migration, not a +> config tweak — the namespace is not embedded in the `external_id`, and Vault does +> not move secrets between namespaces. Point it at the namespace where the secrets +> already live. > **Note:** the Vault policy examples below grant access to the default > `secret/data/nullplatform/*` (and `secret/metadata/nullplatform/*`) paths. If you -> set a custom `setup.namespace`, adjust the policy paths to match it — the KV mount -> and the `metadata/` counterpart of your configured `data/` path. +> set a custom `setup.path_prefix`, adjust the policy paths to match it — the KV +> mount and the `metadata/` counterpart of your configured `data/` path. On Vault +> Enterprise, write the policy inside the configured `setup.namespace`. ## Authentication diff --git a/parameters/providers/hashicorp-vault/delete b/parameters/providers/hashicorp-vault/delete index 0de37da..1523077 100755 --- a/parameters/providers/hashicorp-vault/delete +++ b/parameters/providers/hashicorp-vault/delete @@ -9,16 +9,21 @@ set -euo pipefail # - Any other HTTP status or network error → exit 1 with troubleshooting # # Required env: EXTERNAL_ID, VAULT_ADDR, VAULT_TOKEN +# Optional env: VAULT_NAMESPACE (Vault Enterprise namespace, applied as a URL prefix) -# EXTERNAL_ID_PATH is the full, self-contained Vault path (the KV prefix was -# embedded by store), so it is used verbatim — never recomposed against the -# current VAULT_PATH_PREFIX. This keeps references valid across setup.namespace -# changes. +# EXTERNAL_ID_PATH is the full, self-contained KV path (the KV prefix was embedded +# by store), so it is used verbatim — never recomposed against the current +# VAULT_PATH_PREFIX. This keeps references valid across setup.path_prefix changes. +# The Vault namespace is not part of the path; it is applied from the current +# config as a URL prefix. VAULT_PATH="$EXTERNAL_ID_PATH" +NS_PREFIX="" +[ -n "${VAULT_NAMESPACE:-}" ] && NS_PREFIX="${VAULT_NAMESPACE}/" + if ! RESPONSE=$(curl -s -w "\n%{http_code}" -X DELETE \ -H "X-Vault-Token: $VAULT_TOKEN" \ - "$VAULT_ADDR/v1/$VAULT_PATH" 2>/dev/null); then + "$VAULT_ADDR/v1/${NS_PREFIX}${VAULT_PATH}" 2>/dev/null); then log error "❌ Network error calling Vault at $VAULT_ADDR" log error "" log error "💡 Possible causes:" diff --git a/parameters/providers/hashicorp-vault/docs/architecture.md b/parameters/providers/hashicorp-vault/docs/architecture.md index c610738..920aa74 100644 --- a/parameters/providers/hashicorp-vault/docs/architecture.md +++ b/parameters/providers/hashicorp-vault/docs/architecture.md @@ -8,9 +8,9 @@ This document describes the `parameters/providers/hashicorp-vault/` implementati | Step | What happens | |------|---------------------------------------------------------------------------------------| -| `setup` | Reads `VAULT_ADDR`, `VAULT_AUTH_MODE` (default `userpass`) and the mode's inputs, plus `VAULT_PATH_PREFIX` (default `secret/data/nullplatform`). On Vault Enterprise, derives the Vault namespace from `VAULT_PATH_PREFIX` when it ends in the KV `data` segment (e.g. `admin/ns/data` → `admin/ns`; a `data` segment in the middle is a root-namespace KV path and derives nothing) and prefixes the login URL with it — credentials are namespace-scoped, so a root login would be denied. Authenticates against Vault and exports the derived short-lived token as `VAULT_TOKEN`. Fails fast if address or credentials/identity are missing. | -| `store` | Composes the canonical path via `build_external_id` and prepends `VAULT_PATH_PREFIX`, so the `external_id` is the full, self-contained Vault path. POSTs to `$VAULT_ADDR/v1/` with a JSON payload. Captures the new version number from Vault's response. Returns `external_id = /#`. | -| `retrieve` | Splits `EXTERNAL_ID` into path + version. GETs `$VAULT_ADDR/v1/?version=` using the path **verbatim** (the KV prefix is already embedded — never recomposed against the current `VAULT_PATH_PREFIX`, so a later `setup.namespace` change can't orphan the reference). Returns `{value}` or `{value: "value not found"}`. | +| `setup` | Reads `VAULT_ADDR`, `VAULT_AUTH_MODE` (default `userpass`) and the mode's inputs, plus two independent path axes: `VAULT_NAMESPACE` (the Vault Enterprise namespace, empty = root) and `VAULT_PATH_PREFIX` (the KV v2 `/data/` prefix, default `secret/data/nullplatform`). When a namespace is set, prefixes the login URL with it — credentials are namespace-scoped, so a root login would be denied. Authenticates against Vault and exports the derived short-lived token as `VAULT_TOKEN` plus `VAULT_NAMESPACE`. Fails fast if address or credentials/identity are missing. | +| `store` | Composes the canonical path via `build_external_id` and prepends `VAULT_PATH_PREFIX`, so the `external_id` is the full, self-contained KV path (the namespace is applied separately as a URL prefix, not embedded). POSTs to `$VAULT_ADDR/v1//` with a JSON payload. **Validates the HTTP status** (curl `-s` exits 0 on HTTP errors, so a failed write must not be reported as success). Captures the new version number from Vault's response. Returns `external_id = /#`. | +| `retrieve` | Splits `EXTERNAL_ID` into path + version. GETs `$VAULT_ADDR/v1//?version=` using the path **verbatim** (the KV prefix is already embedded — never recomposed against the current `VAULT_PATH_PREFIX`, so a later `setup.path_prefix` change can't orphan the reference). The namespace comes from the current config. Returns `{value}` or `{value: "value not found"}`. | | `delete` | Parses path from external_id. DELETEs the metadata endpoint (KV v2) — removes all versions. Idempotent. | | `notify` | Not implemented — dispatcher returns default `{success: true}`. | @@ -51,9 +51,11 @@ plus the version: /# ``` -Embedding the prefix makes the `external_id` self-contained: `retrieve`/`delete` -resolve it verbatim, so reconfiguring `setup.namespace` never orphans previously -stored references. +Embedding the KV prefix makes the `external_id` self-contained: `retrieve`/`delete` +resolve it verbatim, so reconfiguring `setup.path_prefix` never orphans previously +stored references. (The Vault namespace is applied from the current config as a URL +prefix and is deliberately not embedded — moving to a different namespace is a +migration, not a config tweak.) For Vault KV v2, `version_id` is **the literal integer version number returned by Vault** in `.data.version` — we do not invent or normalize it. Real example: diff --git a/parameters/providers/hashicorp-vault/retrieve b/parameters/providers/hashicorp-vault/retrieve index 1cb7e82..6bf5438 100755 --- a/parameters/providers/hashicorp-vault/retrieve +++ b/parameters/providers/hashicorp-vault/retrieve @@ -9,14 +9,19 @@ set -euo pipefail # - Any other HTTP status or network error → exit 1 with troubleshooting # # Required env: EXTERNAL_ID, VAULT_ADDR, VAULT_TOKEN +# Optional env: VAULT_NAMESPACE (Vault Enterprise namespace, applied as a URL prefix) # EXTERNAL_ID_PATH and EXTERNAL_ID_VERSION are set by build_context. The path is -# the full, self-contained Vault path (the KV prefix was embedded by store), so it +# the full, self-contained KV path (the KV prefix was embedded by store), so it # is used verbatim — never recomposed against the current VAULT_PATH_PREFIX. This -# keeps references valid across setup.namespace changes. +# keeps references valid across setup.path_prefix changes. The Vault namespace is +# NOT part of the path; it is applied from the current config as a URL prefix. VAULT_PATH="$EXTERNAL_ID_PATH" -URL="$VAULT_ADDR/v1/$VAULT_PATH" +NS_PREFIX="" +[ -n "${VAULT_NAMESPACE:-}" ] && NS_PREFIX="${VAULT_NAMESPACE}/" + +URL="$VAULT_ADDR/v1/${NS_PREFIX}${VAULT_PATH}" if [ -n "${EXTERNAL_ID_VERSION:-}" ]; then URL="${URL}?version=${EXTERNAL_ID_VERSION}" fi @@ -53,7 +58,7 @@ case "$HTTP_STATUS" in fi log error "" log error "🔧 How to fix:" - log error " • Verify: curl -s -H \"X-Vault-Token: \$VAULT_TOKEN\" $VAULT_ADDR/v1/$VAULT_PATH" + log error " • Verify: curl -s -H \"X-Vault-Token: \$VAULT_TOKEN\" $VAULT_ADDR/v1/${NS_PREFIX}${VAULT_PATH}" log error " • If genuinely missing, the parameter value needs to be re-stored" exit 1 ;; diff --git a/parameters/providers/hashicorp-vault/setup b/parameters/providers/hashicorp-vault/setup index 6845704..6081fd5 100755 --- a/parameters/providers/hashicorp-vault/setup +++ b/parameters/providers/hashicorp-vault/setup @@ -17,49 +17,69 @@ set -euo pipefail # store/retrieve/delete keep using the X-Vault-Token header with no knowledge of # how the token was obtained. # -# The KV path prefix defaults to `secret/data/nullplatform` but is operator- -# configurable via .setup.namespace (or the VAULT_PATH_PREFIX env var). It holds -# the KV v2 path prefix and, on Vault Enterprise, may be a Vault namespace path -# terminated by the KV `data` segment: `/data` (e.g. `admin/ns/data`). -# The auth mounts are hardcoded to Vault's defaults (auth/userpass, auth/kubernetes), -# and VAULT_ADDR is operator-configurable. +# Two independent config axes: +# +# VAULT_NAMESPACE The Vault Enterprise namespace (e.g. `admin/eks-null-alfa-136`). +# Operator-configurable via .setup.namespace / VAULT_NAMESPACE. +# Empty means the root namespace — backward compatible with +# non-Enterprise deployments. It is applied as a URL prefix +# (`$VAULT_ADDR/v1//...`) on login AND on every +# store/retrieve/delete request; Vault resolves the leading +# path segments to the namespace before routing to the mount. +# +# VAULT_PATH_PREFIX The KV v2 path prefix RELATIVE to the namespace, i.e. the +# full `/data/` (e.g. `secret/data/nullplatform`). +# It MUST include the mount and the KV v2 `data` segment. +# Defaults to `secret/data/nullplatform`. store embeds it in +# the external_id, so references survive a path_prefix change. +# +# The two are kept separate because a KV v2 path inside a namespace is +# `//data/` — the mount name sits between the namespace +# and `data`, so a single field cannot encode both without ambiguity (which broke +# read/write on Enterprise namespaces). The auth mounts are hardcoded to Vault's +# defaults (auth/userpass, auth/kubernetes); VAULT_ADDR is operator-configurable. # # On Vault Enterprise, credentials/roles are scoped to the namespace, so the login -# must target it too — otherwise Vault returns "access denied". We derive the -# namespace from VAULT_PATH_PREFIX (see below) and prefix the login URL with it, -# matching how store/retrieve/delete embed the prefix into their request paths. -# Prefixes without a trailing `/data` carry no namespace, so login stays on the -# root namespace (backward compatible). +# must target it too — otherwise Vault returns "access denied". # -# Exports: VAULT_ADDR, VAULT_TOKEN, VAULT_PATH_PREFIX +# Exports: VAULT_ADDR, VAULT_TOKEN, VAULT_NAMESPACE, VAULT_PATH_PREFIX VAULT_DEFAULT_PATH_PREFIX="secret/data/nullplatform" VAULT_ADDR=$(get_config_value --env VAULT_ADDR --provider '.setup.address') VAULT_AUTH_MODE=$(get_config_value --env VAULT_AUTH_MODE --provider '.setup.auth_mode' --default "userpass") -VAULT_PATH_PREFIX=$(get_config_value --env VAULT_PATH_PREFIX --provider '.setup.namespace' --default "$VAULT_DEFAULT_PATH_PREFIX") -# Trim a trailing slash so store/retrieve/delete don't compose a double slash. +VAULT_NAMESPACE=$(get_config_value --env VAULT_NAMESPACE --provider '.setup.namespace' --default "") +VAULT_PATH_PREFIX=$(get_config_value --env VAULT_PATH_PREFIX --provider '.setup.path_prefix' --default "$VAULT_DEFAULT_PATH_PREFIX") +# Trim surrounding slashes so URL composition never doubles them. +VAULT_NAMESPACE="${VAULT_NAMESPACE#/}" +VAULT_NAMESPACE="${VAULT_NAMESPACE%/}" VAULT_PATH_PREFIX="${VAULT_PATH_PREFIX%/}" -# Derive the Vault Enterprise namespace (if any) from the path prefix. We only -# treat it as namespaced when the prefix ENDS in the KV v2 `/data` segment -# (e.g. admin/ns/data -> namespace admin/ns): store appends the parameter path -# after `data`, so the operator configures the prefix up to `/data`. -# -# A `data` segment in the MIDDLE (secret/data/nullplatform, secret/data/team-x, -# kv/data/foo) is a plain KV mount + subpath in the ROOT namespace, textually -# indistinguishable from a namespace, so we must NOT derive one — doing so would -# break login for existing non-Enterprise deployments. Hence: trailing `/data` -# only. (Matching on the `/data` suffix is segment-safe: `.../database` does not -# end in `/data`.) -VAULT_NAMESPACE="" -case "$VAULT_PATH_PREFIX" in - */data) VAULT_NAMESPACE="${VAULT_PATH_PREFIX%/data}" ;; -esac +# Defense-in-depth: both values are interpolated raw into request URLs (the login +# call and every store/retrieve/delete). They are multi-segment paths where `/` is +# significant, so we validate rather than URL-encode (unlike the single-segment +# username, which is @uri-encoded): allow only Vault-legal path characters and +# reject `..` segments that could rewrite the request path. Values are operator +# config, so this guards against misconfiguration, not attacker input. +validate_vault_path_config() { + local name="$1" value="$2" + [ -z "$value" ] && return 0 + if [[ "$value" == *".."* ]] || [[ "$value" =~ [^A-Za-z0-9._/-] ]]; then + log error "❌ Invalid Vault $name: '$value'" + log error "" + log error "💡 Possible causes:" + log error " • .setup.$name contains characters outside [A-Za-z0-9._/-] or a '..' segment" + log error "" + log error "🔧 How to fix:" + log error " • Use only letters, digits, '.', '_', '-' and '/' path separators" + exit 1 + fi +} +validate_vault_path_config namespace "$VAULT_NAMESPACE" +validate_vault_path_config path_prefix "$VAULT_PATH_PREFIX" # API base for the login call — namespace-prefixed on Vault Enterprise, plain -# otherwise. store/retrieve/delete need no equivalent: the namespace is already -# embedded in VAULT_PATH_PREFIX, which they compose into their request paths. +# otherwise. store/retrieve/delete compose the same namespace prefix themselves. VAULT_LOGIN_BASE="$VAULT_ADDR/v1" if [ -n "$VAULT_NAMESPACE" ]; then VAULT_LOGIN_BASE="$VAULT_ADDR/v1/$VAULT_NAMESPACE" @@ -198,4 +218,4 @@ if [ -z "$VAULT_TOKEN" ]; then exit 1 fi -export VAULT_ADDR VAULT_TOKEN VAULT_PATH_PREFIX +export VAULT_ADDR VAULT_TOKEN VAULT_NAMESPACE VAULT_PATH_PREFIX diff --git a/parameters/providers/hashicorp-vault/specs/install/hashicorp-vault-configuration.json.tpl b/parameters/providers/hashicorp-vault/specs/install/hashicorp-vault-configuration.json.tpl index 01a27d4..4e7f1a0 100644 --- a/parameters/providers/hashicorp-vault/specs/install/hashicorp-vault-configuration.json.tpl +++ b/parameters/providers/hashicorp-vault/specs/install/hashicorp-vault-configuration.json.tpl @@ -52,15 +52,21 @@ "namespace": { "type": "string", "title": "Namespace", - "description": "Namespace and path prefix under which parameters are stored. Must include the KV v2 `data/` segment (e.g. secret/data/nullplatform). On Vault Enterprise, set it to the namespace path terminated by `data` (e.g. admin/ns/data) — the login targets the namespace and only a trailing `/data` is treated as a namespace.", - "order": 2, + "description": "Vault Enterprise namespace the parameters live under (e.g. admin/eks-null-alfa-136). Leave empty for the root namespace (Vault OSS / single-namespace). The login and every read/write target this namespace.", + "order": 2 + }, + "path_prefix": { + "type": "string", + "title": "KV path prefix", + "description": "KV v2 path prefix (relative to the namespace) parameters are stored under. Must be the full `/data/` including the mount name and the KV v2 `data` segment (e.g. secret/data/nullplatform).", + "order": 3, "default": "secret/data/nullplatform" }, "auth_mode": { "type": "string", "title": "Authentication mode", "description": "How the nullplatform agent authenticates to Vault.", - "order": 3, + "order": 4, "default": "userpass", "oneOf": [ { "const": "userpass", "title": "Username and password" }, @@ -71,7 +77,7 @@ "type": "string", "title": "Vault Kubernetes role", "description": "Name of the Vault Kubernetes auth role bound to the agent's ServiceAccount. Required when the authentication mode is Kubernetes.", - "order": 4 + "order": 5 } }, "allOf": [ @@ -97,6 +103,10 @@ "type": "Control", "scope": "#/properties/setup/properties/namespace" }, + { + "type": "Control", + "scope": "#/properties/setup/properties/path_prefix" + }, { "type": "Control", "scope": "#/properties/setup/properties/auth_mode", diff --git a/parameters/providers/hashicorp-vault/specs/install/locals.tf b/parameters/providers/hashicorp-vault/specs/install/locals.tf index 0f3f042..9cda165 100644 --- a/parameters/providers/hashicorp-vault/specs/install/locals.tf +++ b/parameters/providers/hashicorp-vault/specs/install/locals.tf @@ -3,7 +3,8 @@ locals { # but null-valued optional keys are dropped: the spec schema types them as # strings (with their own defaults), so emitting `null` would fail server-side # validation. Each key is only included when set — kubernetes_role when - # auth_mode = "kubernetes", and namespace when overriding the default KV prefix. + # auth_mode = "kubernetes", namespace for the Vault Enterprise namespace, and + # path_prefix when overriding the default KV prefix. instance_attributes = { for key, instance in var.instances : key => { sensibility = instance.attributes.sensibility @@ -15,6 +16,9 @@ locals { instance.attributes.setup.namespace == null ? {} : { namespace = instance.attributes.setup.namespace }, + instance.attributes.setup.path_prefix == null ? {} : { + path_prefix = instance.attributes.setup.path_prefix + }, instance.attributes.setup.kubernetes_role == null ? {} : { kubernetes_role = instance.attributes.setup.kubernetes_role } diff --git a/parameters/providers/hashicorp-vault/specs/install/terraform.tfvars.example b/parameters/providers/hashicorp-vault/specs/install/terraform.tfvars.example index 5e0d9ef..c3b7547 100644 --- a/parameters/providers/hashicorp-vault/specs/install/terraform.tfvars.example +++ b/parameters/providers/hashicorp-vault/specs/install/terraform.tfvars.example @@ -20,10 +20,11 @@ instances = { } setup = { address = "https://vault.prod.example.com:8200" - # namespace is the KV v2 mount + path prefix; it defaults to "secret/data/nullplatform" - # when omitted. Override it to store parameters under a different KV path — keep the - # KV v2 "data/" segment (e.g. "secret/data/acme-billing"): - # namespace = "secret/data/acme-billing" + # namespace is the Vault Enterprise namespace; omit it for the root namespace (Vault OSS). + # namespace = "admin/eks-null-alfa-136" + # path_prefix is the KV v2 "/data/" prefix; defaults to "secret/data/nullplatform" + # when omitted. Keep the KV v2 "data/" segment (e.g. "secret/data/acme-billing"): + # path_prefix = "secret/data/acme-billing" # auth_mode defaults to "userpass": the agent reads VAULT_USERNAME / VAULT_PASSWORD from env. } } @@ -38,7 +39,7 @@ instances = { setup = { address = "https://vault.staging.example.com:8200" # Custom KV path prefix — parameters are stored under secret/data/acme-staging/... - namespace = "secret/data/acme-staging" + path_prefix = "secret/data/acme-staging" # Kubernetes auth: the agent uses its pod ServiceAccount identity — no credentials in env. # kubernetes_role must match the Vault role bound to the agent's ServiceAccount. # Provision the cluster-side resources with the requirements/ module. diff --git a/parameters/providers/hashicorp-vault/specs/install/variables.tf b/parameters/providers/hashicorp-vault/specs/install/variables.tf index 6f2aec1..c08d181 100644 --- a/parameters/providers/hashicorp-vault/specs/install/variables.tf +++ b/parameters/providers/hashicorp-vault/specs/install/variables.tf @@ -42,8 +42,10 @@ variable "instances" { notification channel (anchored at the instance NRN). Fields: attributes — provider config matching the spec schema: sensibility.applies_to plus setup.address / setup.namespace / - setup.auth_mode / setup.kubernetes_role. namespace is the KV v2 - mount + path prefix; leave it null to use the default + setup.path_prefix / setup.auth_mode / setup.kubernetes_role. + namespace is the Vault Enterprise namespace (leave null for the + root namespace on Vault OSS). path_prefix is the KV v2 + `/data/` prefix; leave it null to use the default (secret/data/nullplatform). auth_mode is "userpass" (default) or "kubernetes"; kubernetes_role is only used (and required) when auth_mode = "kubernetes" — leave it null for userpass. @@ -63,6 +65,7 @@ variable "instances" { setup = object({ address = string namespace = optional(string) + path_prefix = optional(string) auth_mode = optional(string, "userpass") kubernetes_role = optional(string) }) diff --git a/parameters/providers/hashicorp-vault/store b/parameters/providers/hashicorp-vault/store index 69a832e..720045d 100755 --- a/parameters/providers/hashicorp-vault/store +++ b/parameters/providers/hashicorp-vault/store @@ -14,17 +14,24 @@ set -euo pipefail # and pass it back to retrieve when fetching a specific historical version. # # Required env: PARAMETER_ID, PARAMETER_VALUE, VAULT_ADDR, VAULT_TOKEN, VAULT_PATH_PREFIX +# Optional env: VAULT_NAMESPACE (Vault Enterprise namespace, applied as a URL prefix) source "$PARAMETERS_ROOT/utils/build_external_id" build_external_id -# Embed the KV path prefix so the external_id IS the full, self-contained Vault -# path. retrieve/delete then use it verbatim instead of recomposing it against the +# Embed the KV path prefix so the external_id IS the full, self-contained KV path. +# retrieve/delete then use it verbatim instead of recomposing it against the # current VAULT_PATH_PREFIX — so existing references stay valid even if -# setup.namespace is later reconfigured. +# setup.path_prefix is later reconfigured. The Vault namespace is NOT embedded: +# it is applied from the current config as a URL prefix at request time (see below). EXTERNAL_ID="$VAULT_PATH_PREFIX/$EXTERNAL_ID" VAULT_PATH="$EXTERNAL_ID" +# Namespace segment for the request URL (empty on the root namespace). Vault +# resolves it to the Enterprise namespace before routing to the KV mount. +NS_PREFIX="" +[ -n "${VAULT_NAMESPACE:-}" ] && NS_PREFIX="${VAULT_NAMESPACE}/" + PAYLOAD=$(jq -nc \ --argjson parameter_id "${PARAMETER_ID:-null}" \ --arg value "$PARAMETER_VALUE" \ @@ -32,24 +39,42 @@ PAYLOAD=$(jq -nc \ --arg external_id "$EXTERNAL_ID" \ '{data: {parameter_id: $parameter_id, value: $value, stored_at: $stored_at, external_id: $external_id}}') -if ! RESPONSE=$(curl -s -X POST \ +# Capture the HTTP status alongside the body: curl -s exits 0 even on HTTP 4xx/5xx, +# so relying on curl's exit code alone would report a failed write as success. +if ! RESPONSE=$(curl -s -w "\n%{http_code}" -X POST \ -H "X-Vault-Token: $VAULT_TOKEN" \ - "$VAULT_ADDR/v1/$VAULT_PATH" \ - -d "$PAYLOAD"); then - log error "❌ Failed to store parameter in Vault at $VAULT_ADDR" + "$VAULT_ADDR/v1/${NS_PREFIX}${VAULT_PATH}" \ + -d "$PAYLOAD" 2>/dev/null); then + log error "❌ Network error storing parameter in Vault at $VAULT_ADDR" log error "" log error "💡 Possible causes:" - log error " • Network unreachable to $VAULT_ADDR" - log error " • VAULT_TOKEN expired or lacks write permission on $VAULT_PATH" - log error " • KV mount at $VAULT_PATH_PREFIX does not exist" + log error " • Network unreachable to $VAULT_ADDR (DNS / firewall / TLS)" log error "" log error "🔧 How to fix:" log error " • Test connectivity: curl -s $VAULT_ADDR/v1/sys/health" + exit 1 +fi + +HTTP_STATUS="${RESPONSE##*$'\n'}" +HTTP_BODY="${RESPONSE%$'\n'*}" + +if [[ "$HTTP_STATUS" != 2* ]]; then + log error "❌ Vault write failed with HTTP $HTTP_STATUS at $VAULT_PATH" + log error "" + log error "💡 Possible causes:" + log error " • VAULT_TOKEN lacks write permission on $VAULT_PATH (403)" + log error " • KV v2 mount in path_prefix ($VAULT_PATH_PREFIX) does not exist (404)" + log error " • Wrong Vault namespace (${VAULT_NAMESPACE:-}) — check .setup.namespace" + log error " • path_prefix must be the full /data/ (404 if the mount/data segment is wrong)" + log error "" + log error "🔧 How to fix:" log error " • Verify token: curl -s -H \"X-Vault-Token: \$VAULT_TOKEN\" $VAULT_ADDR/v1/auth/token/lookup-self" + log error " • Confirm the mount: curl -s -H \"X-Vault-Token: \$VAULT_TOKEN\" $VAULT_ADDR/v1/${NS_PREFIX}sys/mounts" + log error "Vault response: $HTTP_BODY" exit 1 fi -VERSION_ID=$(echo "$RESPONSE" | jq -r '.data.version // empty') +VERSION_ID=$(echo "$HTTP_BODY" | jq -r '.data.version // empty') # Encode version into external_id (the canonical handle nullplatform persists). [ -n "$VERSION_ID" ] && EXTERNAL_ID="${EXTERNAL_ID}#${VERSION_ID}" diff --git a/parameters/tests/providers/hashicorp-vault/delete.bats b/parameters/tests/providers/hashicorp-vault/delete.bats index 80e1a69..a457f23 100644 --- a/parameters/tests/providers/hashicorp-vault/delete.bats +++ b/parameters/tests/providers/hashicorp-vault/delete.bats @@ -95,6 +95,16 @@ EOF assert_contains "$captured" "https://vault.example.com/v1/secret/data/nullplatform/abc-123" } +@test "vault delete: applies the Vault namespace as a URL prefix" { + export VAULT_NAMESPACE="admin/eks-null-alfa-136" + + run bash -c "$DEPS; MOCK_HTTP_STATUS=204 source $SCRIPT" + + assert_equal "$status" "0" + captured=$(cat "$CURL_LOG") + assert_contains "$captured" "https://vault.example.com/v1/admin/eks-null-alfa-136/secret/data/nullplatform/abc-123" +} + @test "vault delete: uses external_id path verbatim, ignoring current VAULT_PATH_PREFIX" { # Regression: simulates setup.namespace being reconfigured AFTER this secret was # stored. The full path lives in the external_id, so it must win over the current diff --git a/parameters/tests/providers/hashicorp-vault/retrieve.bats b/parameters/tests/providers/hashicorp-vault/retrieve.bats index 79458ac..5f63128 100644 --- a/parameters/tests/providers/hashicorp-vault/retrieve.bats +++ b/parameters/tests/providers/hashicorp-vault/retrieve.bats @@ -105,6 +105,17 @@ EOF assert_contains "$captured" "https://vault.example.com/v1/secret/data/nullplatform/abc-123" } +@test "vault retrieve: applies the Vault namespace as a URL prefix" { + export VAULT_NAMESPACE="admin/eks-null-alfa-136" + body='{"data":{"data":{"value":"x"}}}' + + run bash -c "$DEPS; MOCK_HTTP_STATUS=200 MOCK_HTTP_BODY='$body' source $SCRIPT" + + assert_equal "$status" "0" + captured=$(cat "$CURL_LOG") + assert_contains "$captured" "https://vault.example.com/v1/admin/eks-null-alfa-136/secret/data/nullplatform/abc-123" +} + @test "vault retrieve: uses external_id path verbatim, ignoring current VAULT_PATH_PREFIX" { # Regression: simulates setup.namespace being reconfigured AFTER this secret was # stored. The full path lives in the external_id, so it must win over the current diff --git a/parameters/tests/providers/hashicorp-vault/setup.bats b/parameters/tests/providers/hashicorp-vault/setup.bats index 10b861a..646e76d 100644 --- a/parameters/tests/providers/hashicorp-vault/setup.bats +++ b/parameters/tests/providers/hashicorp-vault/setup.bats @@ -40,7 +40,7 @@ EOF } teardown() { - unset VAULT_ADDR VAULT_AUTH_MODE VAULT_TOKEN VAULT_PATH_PREFIX PROVIDER_CONFIG \ + unset VAULT_ADDR VAULT_AUTH_MODE VAULT_TOKEN VAULT_NAMESPACE VAULT_PATH_PREFIX PROVIDER_CONFIG \ VAULT_USERNAME VAULT_PASSWORD VAULT_K8S_ROLE VAULT_K8S_JWT_PATH \ MOCK_LOGIN_BODY MOCK_CURL_EXIT } @@ -80,10 +80,10 @@ teardown() { assert_contains "$output" "PREFIX=secret/data/nullplatform" } -@test "vault setup: path_prefix from PROVIDER_CONFIG (.setup.namespace)" { +@test "vault setup: path_prefix from PROVIDER_CONFIG (.setup.path_prefix)" { export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"secret/data/team-x"}}' + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","path_prefix":"secret/data/team-x"}}' run bash -c "$DEPS; source $SCRIPT && echo PREFIX=\$VAULT_PATH_PREFIX" @@ -107,7 +107,7 @@ teardown() { export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" export VAULT_PATH_PREFIX="secret/data/from-env" - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"secret/data/from-config"}}' + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","path_prefix":"secret/data/from-config"}}' run bash -c "$DEPS; source $SCRIPT && echo PREFIX=\$VAULT_PATH_PREFIX" @@ -118,7 +118,7 @@ teardown() { @test "vault setup: path_prefix trailing slash is trimmed" { export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"secret/data/team-x/"}}' + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","path_prefix":"secret/data/team-x/"}}' run bash -c "$DEPS; source $SCRIPT && echo PREFIX=\$VAULT_PATH_PREFIX" @@ -126,6 +126,28 @@ teardown() { assert_contains "$output" "PREFIX=secret/data/team-x" } +@test "vault setup: rejects a namespace with illegal characters" { + export VAULT_USERNAME="agent" + export VAULT_PASSWORD="pw" + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"admin/ns?injected=1"}}' + + run bash -c "$DEPS; source $SCRIPT" + + [ "$status" -ne 0 ] + assert_contains "$output" "❌ Invalid Vault namespace" +} + +@test "vault setup: rejects a path_prefix containing a .. segment" { + export VAULT_USERNAME="agent" + export VAULT_PASSWORD="pw" + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","path_prefix":"secret/../data"}}' + + run bash -c "$DEPS; source $SCRIPT" + + [ "$status" -ne 0 ] + assert_contains "$output" "❌ Invalid Vault path_prefix" +} + @test "vault setup: unknown auth_mode fails" { export VAULT_ADDR="https://vault.example.com" export VAULT_AUTH_MODE="ldap" @@ -208,11 +230,11 @@ teardown() { assert_contains "$captured" "login/ns%2Fagent" } -@test "vault setup: userpass login is NOT namespace-prefixed for the default prefix" { +@test "vault setup: userpass login is NOT namespace-prefixed by default (root namespace)" { export VAULT_ADDR="https://vault.example.com" export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - # default prefix (secret/data/nullplatform) → root namespace, no prefix + # no .setup.namespace → root namespace, no prefix run bash -c "$DEPS; source $SCRIPT" @@ -221,78 +243,73 @@ teardown() { assert_contains "$captured" "https://vault.example.com/v1/auth/userpass/login/agent" } -@test "vault setup: userpass login is prefixed with the Vault namespace (prefix ends in /data)" { +@test "vault setup: userpass login is prefixed with the configured Vault namespace" { export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - # admin/ns/data → Vault namespace = admin/ns - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"admin/ns/data"}}' + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"admin/eks-null-alfa-136"}}' run bash -c "$DEPS; source $SCRIPT" assert_equal "$status" "0" captured=$(cat "$CURL_LOG") - assert_contains "$captured" "https://vault.example.com/v1/admin/ns/auth/userpass/login/agent" + assert_contains "$captured" "https://vault.example.com/v1/admin/eks-null-alfa-136/auth/userpass/login/agent" } -@test "vault setup: VAULT_PATH_PREFIX export keeps the trailing /data segment when a namespace is derived" { +@test "vault setup: VAULT_NAMESPACE is exported for store/retrieve/delete" { export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - # Namespace derivation must not mutate VAULT_PATH_PREFIX itself — store/retrieve/ - # delete compose their Vault paths from the full prefix (including /data). - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"admin/ns/data"}}' + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"admin/ns"}}' - run bash -c "$DEPS; source $SCRIPT && echo PREFIX=\$VAULT_PATH_PREFIX" + run bash -c "$DEPS; source $SCRIPT && echo NS=\$VAULT_NAMESPACE" assert_equal "$status" "0" - assert_contains "$output" "PREFIX=admin/ns/data" + assert_contains "$output" "NS=admin/ns" } -@test "vault setup: userpass login is NOT namespace-prefixed for a non-default root prefix (custom subpath, no Enterprise namespace)" { +@test "vault setup: VAULT_NAMESPACE from env var prefixes the login" { + export VAULT_ADDR="https://vault.example.com" export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - # secret/data/team-x is a plain KV mount + subpath in the root namespace — the - # `data` segment is in the MIDDLE, so no namespace is derived (backward compat). - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"secret/data/team-x"}}' + export VAULT_NAMESPACE="admin/env-ns" run bash -c "$DEPS; source $SCRIPT" assert_equal "$status" "0" captured=$(cat "$CURL_LOG") - assert_contains "$captured" "https://vault.example.com/v1/auth/userpass/login/agent" + assert_contains "$captured" "https://vault.example.com/v1/admin/env-ns/auth/userpass/login/agent" } -@test "vault setup: userpass login is NOT namespace-prefixed when a subpath follows the data segment" { +@test "vault setup: namespace surrounding slashes are trimmed" { export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - # Only a TRAILING /data derives a namespace; admin/ns/data/foo has a subpath - # after data, so it is treated as a root KV path (no namespace prefix). - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"admin/ns/data/foo"}}' + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"/admin/ns/"}}' - run bash -c "$DEPS; source $SCRIPT" + run bash -c "$DEPS; source $SCRIPT && echo NS=\$VAULT_NAMESPACE" assert_equal "$status" "0" + assert_contains "$output" "NS=admin/ns" captured=$(cat "$CURL_LOG") - assert_contains "$captured" "https://vault.example.com/v1/auth/userpass/login/agent" + assert_contains "$captured" "https://vault.example.com/v1/admin/ns/auth/userpass/login/agent" } -@test "vault setup: userpass login namespace does not treat 'database' as the data segment" { +@test "vault setup: path_prefix is independent of the namespace" { export VAULT_USERNAME="agent" export VAULT_PASSWORD="pw" - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"myns/database/x"}}' + # namespace and path_prefix are separate axes: setting a namespace must not + # alter the KV path prefix that store/retrieve/delete compose their paths from. + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","namespace":"admin/ns","path_prefix":"secret/data/nullplatform"}}' - run bash -c "$DEPS; source $SCRIPT" + run bash -c "$DEPS; source $SCRIPT && echo PREFIX=\$VAULT_PATH_PREFIX" assert_equal "$status" "0" - captured=$(cat "$CURL_LOG") - # 'database' is not the KV 'data' segment → no namespace derived → root login - assert_contains "$captured" "https://vault.example.com/v1/auth/userpass/login/agent" + assert_contains "$output" "PREFIX=secret/data/nullplatform" } -@test "vault setup: kubernetes login is prefixed with the Vault namespace" { +@test "vault setup: kubernetes login is prefixed with the configured Vault namespace" { export VAULT_AUTH_MODE="kubernetes" export VAULT_K8S_ROLE="nullplatform-agent" export VAULT_K8S_JWT_PATH="$SA_TOKEN_FILE" - export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","auth_mode":"kubernetes","kubernetes_role":"nullplatform-agent","namespace":"admin/ns/data"}}' + export PROVIDER_CONFIG='{"setup":{"address":"https://vault.example.com","auth_mode":"kubernetes","kubernetes_role":"nullplatform-agent","namespace":"admin/ns"}}' run bash -c "$DEPS; source $SCRIPT" diff --git a/parameters/tests/providers/hashicorp-vault/store.bats b/parameters/tests/providers/hashicorp-vault/store.bats index fb36d05..42e999e 100644 --- a/parameters/tests/providers/hashicorp-vault/store.bats +++ b/parameters/tests/providers/hashicorp-vault/store.bats @@ -15,14 +15,17 @@ setup() { mkdir -p "$BATS_TEST_TMPDIR/bin" - # Mock curl + # Mock curl. store passes -w "\n%{http_code}", so append the HTTP status on a + # trailing line; store validates it (curl -s alone exits 0 even on HTTP errors). export CURL_LOG="$BATS_TEST_TMPDIR/curl.log" - cat > "$BATS_TEST_TMPDIR/bin/curl" << EOF + cat > "$BATS_TEST_TMPDIR/bin/curl" << 'EOF' #!/bin/bash -echo "ARGS: \$@" >> "$CURL_LOG" -if [ "\${MOCK_CURL_EXIT:-0}" -ne 0 ]; then exit \$MOCK_CURL_EXIT; fi -# Vault KV v2 returns the new version number in response body -echo '{"data":{"created_time":"2026-06-23T00:00:00Z","version":3,"deletion_time":"","destroyed":false}}' +echo "ARGS: $@" >> "$CURL_LOG" +if [ "${MOCK_CURL_EXIT:-0}" -ne 0 ]; then exit "$MOCK_CURL_EXIT"; fi +# Vault KV v2 returns the new version number in the response body. +default_body='{"data":{"created_time":"2026-06-23T00:00:00Z","version":3,"deletion_time":"","destroyed":false}}' +printf '%s' "${MOCK_HTTP_BODY:-$default_body}" +printf '\n%s' "${MOCK_HTTP_STATUS:-200}" EOF chmod +x "$BATS_TEST_TMPDIR/bin/curl" @@ -107,10 +110,42 @@ EOF run bash -c "$DEPS; MOCK_CURL_EXIT=22 source $SCRIPT" [ "$status" -ne 0 ] - assert_contains "$output" "❌ Failed to store parameter in Vault" + assert_contains "$output" "❌ Network error storing parameter in Vault" assert_contains "$output" "💡 Possible causes:" } +@test "vault store: fails when Vault returns a non-2xx status (silent-write regression)" { + # Regression: a failed KV write (wrong mount / namespace / permission) must NOT + # be reported as success. curl -s exits 0 on HTTP 4xx, so store must inspect the + # HTTP status, not just curl's exit code. + run bash -c "$DEPS; MOCK_HTTP_STATUS=404 MOCK_HTTP_BODY='{\"errors\":[]}' source $SCRIPT" + + [ "$status" -ne 0 ] + assert_contains "$output" "❌ Vault write failed with HTTP 404" + assert_contains "$output" "🔧 How to fix:" +} + +@test "vault store: fails when Vault returns 403 (no write permission)" { + run bash -c "$DEPS; MOCK_HTTP_STATUS=403 MOCK_HTTP_BODY='{\"errors\":[\"permission denied\"]}' source $SCRIPT" + + [ "$status" -ne 0 ] + assert_contains "$output" "❌ Vault write failed with HTTP 403" +} + +@test "vault store: applies the Vault namespace as a URL prefix" { + export VAULT_NAMESPACE="admin/eks-null-alfa-136" + + run bash -c "$DEPS; source $SCRIPT" + + assert_equal "$status" "0" + captured=$(cat "$CURL_LOG") + assert_contains "$captured" "https://vault.example.com/v1/admin/eks-null-alfa-136/secret/data/nullplatform/organization=acme-1255165411" + # The namespace is NOT baked into the external_id (it comes from current config). + external_id=$(echo "$output" | jq -r '.external_id') + assert_contains "$external_id" "secret/data/nullplatform/organization=acme-1255165411" + [[ "$external_id" != admin/* ]] +} + @test "vault store: external_id embeds a custom VAULT_PATH_PREFIX" { export VAULT_PATH_PREFIX="kv/data/custom-mount"