feat: As a user, I want a Valkey config_provider for the data-plane role, so that I can run a decoupled data plane without etcd

[daric93]

Description

As a user running Valkey/Redis-compatible infrastructure, I want to run APISIX data-plane nodes that sync their configuration from Valkey, so that I can operate a decoupled data plane without also running an etcd cluster.

Context

APISIX already supports multiple config_provider options for the data-plane role (etcd, yaml, json, xds). This proposal adds one more — valkey — following the same read-only sync pattern that json and xds established. It is purely additive: etcd remains the default and nothing about the existing providers changes.

Many teams already standardize on Valkey/Redis for their infrastructure (APISIX's own redis-backed plugins are widely used against Valkey today, e.g. #13584, #12282). Letting a data plane sync config from Valkey removes etcd as a hard dependency for those deployments.

Why Valkey can satisfy the config-provider contract

The historical concern with non-etcd config stores (see the FAQ) is the need for resumable watch streams, prefix reads, and atomic updates. Valkey now provides primitives that map cleanly onto these:

• Change notification → Valkey Streams (XREADGROUP with consumer groups): resumable, at-least-once delivery from the last processed ID, which is exactly the "watch stream" semantic etcd provides. This is stronger than the yaml provider's 1s polling.
• Prefix / directory reads → one Hash per collection ({apisix}:routes, {apisix}:services, ...): HGETALL returns all items in a collection atomically.
• Revision tracking → an application-managed counter key ({apisix}:revision), INCRed on each change, used to synthesize the conf_version / modifiedIndex that core.config consumers rely on.
• Hash-tag {apisix} keeps keys co-located on one shard so the design is Valkey-cluster-safe.

Proposed design (data-plane, read-only)

• New provider module apisix/core/config_valkey.lua implementing the existing core.config read contract (new, get, fetch_created_obj, close, init, init_worker, plus values / values_hash / conf_version / clean_handlers and filter / checker / item_schema handling) — the same surface config_yaml.lua and config_xds.lua satisfy today.
• Connect via the existing api7-lua-resty-redis-connector dependency (already used by the redis plugins).
• Add "valkey" to the data-plane config_provider enum + a valkey connection schema (host, port, password, database, tls) in apisix/cli/schema.lua, with the corresponding edits in apisix/cli/file.lua, apisix/cli/ops.lua, and the config_ready_check allowlist in apisix/init.lua.
• Config gets written to Valkey by an external process (or a Valkey-backed standalone controller); the data plane only reads and watches. Admin API writes are explicitly out of scope for this issue — this is data-plane sync only, matching how xds works.

Example configuration:

# conf/config.yaml
deployment:
  role: data_plane
  role_data_plane:
    config_provider: valkey
    valkey:
      host: "valkey.example.com"
      port: 6379
      password: "secret"
      database: 0
      tls:
        enabled: true

Testing

• New t/config-center-valkey/ directory mirroring t/config-center-yaml/ and t/config-center-json/, using --- yaml_config blocks to set the data-plane role + config_provider: valkey.
• Add a valkey/valkey service to ci/pod/docker-compose.common.yml and wire t/config-center-valkey into the corresponding test group in .github/workflows/build.yml.

Scope note

This is deliberately scoped to the data-plane read path — the pattern with clear precedent. Extending the option to the Admin API / control-plane write path is a larger, separate discussion (it would need a write-side storage abstraction, since the Admin API calls core.etcd directly today) and is not part of this proposal.

I'd like to gauge maintainer interest before writing code, per CONTRIBUTING.md. Does this direction sound acceptable, and are there constraints I should account for in the design? I'm happy to implement it. Thank you!

[daric93]

I'd like to gauge maintainer interest before writing code, per CONTRIBUTING.md. Does this direction sound acceptable, and are there constraints I should account for in the design? I'm happy to implement it. Thank you!