Bring Docker Compose target to v0.5.0 parity with Kubernetes

[fbeneventi]

Context

In v0.5.0 the Kubernetes (Helm) target has moved well ahead of the Docker Compose target. Compose is positioned as a first-class single-host option (see doc11 reframing and compose.trino.yml shipped in 14341cd), but the underlying docker-compose.yml is still essentially the v0.4.0 stack. This issue tracks the work to bring it to parity.

Gaps to close

1. Split the monolithic examon service

Today the examon service in docker-compose.yml bundles Mosquitto, mqtt2kairosdb, random_pub, and the examon-server Flask API under one supervisord container (examonhpc/examon:latest). The Helm chart already ships these as separate workloads with separate subcharts (charts/mosquitto, charts/mqtt2kairosdb, charts/random-pub, charts/examon-server, charts/grafana, charts/kairosdb).

Compose should mirror that decomposition:

• mosquitto service (publish 1883)
• mqtt2kairosdb service (no host port; talks to mosquitto + kairosdb)
• examon-server service (publish 5000)
• random-pub service (no host port; publishes to mosquitto)
• Keep cassandra, kairosdb, grafana as today (already separate)

This unblocks per-component lifecycle, log isolation, and image-version alignment with the chart.

2. Align image versions with the Kubernetes chart

The Compose stack currently pins:

• examonhpc/examon:latest (monolithic)
• examonhpc/grafana:7.3.10 (legacy AngularJS plugin era)
• examonhpc/kairosdb:1.2.2
• examonhpc/cassandra:3.0.19

The K8s chart targets newer Grafana (with the arpnetworking-kairosdb-datasource plugin), the K8ssandra operator for Cassandra, and an updated KairosDB image. Compose should track the same image pins per component, so an operator does not face two different stacks behind the same release tag.

3. Automatic datasource and dashboard provisioning

The Helm chart auto-provisions the KairosDB datasource and ships the Random Sensor test dashboard via grafana.plugins + grafana.grafana.ini + a dashboards ConfigMap. The Compose stack still requires the operator to add the datasource manually (steps 1-3 in with-docker-compose.md) and to import the legacy dashboards/legacy/...json snapshot by hand.

Compose should provision both automatically using Grafana's file-based provisioning (/etc/grafana/provisioning/datasources/*.yaml and /etc/grafana/provisioning/dashboards/*.yaml) mounted from deploy/docker/grafana/provisioning/. Once Grafana is on the same version as the chart, the dashboard JSON used should be the React-plugin one from dashboards/, not the AngularJS snapshot.

4. Smoke test parity

The K8s target ships scripts/k8s-local-setup.sh + scripts/k8s-smoke-test.sh. The Compose target has no equivalent. Add scripts/compose-smoke-test.sh exercising the same layers: container health, KairosDB health endpoint, Grafana reachability + datasource provisioning, dashboard load, ExaMon API reachability, MQTT pub/sub round-trip, and an end-to-end metric count from random_pub.

5. Compose overlays directory structure

Today deploy/docker/ only holds image-build subfolders (examon-server, kairosdb, mosquitto, mqtt2kairosdb, random-pub). The Trino overlay added a deploy/docker/trino/ peer. Once provisioning files and split services exist, normalize the layout so deploy/docker/ is the Compose counterpart of deploy/helm/, including:

• deploy/docker/grafana/provisioning/{datasources,dashboards}/
• deploy/docker/mosquitto/ (config)
• Any per-component env or config files extracted out of the umbrella image.

6. Documentation follow-up (on the next docN branch)

Once 1-5 are in place:

• Rewrite docs/administrators/deploy/with-docker-compose.md to reflect the split services and drop the manual Grafana datasource / dashboard steps.
• Update docs/users/dashboards/index.md to remove the "dashboards under dashboards/ are not compatible with Compose" caveat once Grafana versions match.
• Drop the dashboards/legacy/ reference path from the Compose flow (kept on disk for users still running v0.4.0).
• Verify the Trino Quickstart Docker section against the new Compose shape.

Out of scope

• Production hardening for Compose (TLS, secret rotation, NetworkPolicy-equivalent): Compose is intentionally a single-host quick-deployment target, not a production target. That stays a Kubernetes-only concern.
• Multi-host Compose / Swarm: not a supported shape.

Related

• release/v0.5.0 commit 14341cd: ships compose.trino.yml (already aligned with the K8s Trino overlay shape).
• doc11 commit d1b17d5: reframes Compose as a first-class single-host target in the docs.