feat(gcp): bootstrap two data centers with --multi-dc - #631
Open
NJona wants to merge 3 commits into
Open
Conversation
Wires up multi-data-center support: --multi-dc bootstraps a second data center in the same project, sharing the VPC, jumpbox and PostgreSQL server but running its own Kubernetes and Ceph cluster. The secondary data center's config and vault are derived from the primary's after its secrets exist, which is why Bootstrap handles it in a loop after the primary's config is written: - the config is cloned and its data-center-scoped fields cleared, so the installer's dataCenters topology, the domains and the shared registry are inherited while the ingress CA and cephadm key are regenerated; - the vault is derived through DeriveDataCenterVault, keeping the postgres roles and token keys and dropping the per-cluster secrets; - postgres becomes mode: external pointing at the shared server's internal IP, which is what its certificate's only SAN carries, and "postgres" is appended to operations.skip in the config so manual re-runs on the jumpbox skip it too. A post-generation check fails the bootstrap if secrets that must match across data centers diverged, or if a per-cluster secret was inherited instead of regenerated — both would otherwise surface long after the fact, against a live shared database. Installs run strictly sequentially in ascending data center order, since the primary's install creates the database, roles and schema the others reuse. validateMultiDC rejects the combinations that cannot work (--write-config=false, an explicit --datacenter-id, an empty datacenter name), and the CLI prints one install command per data center in the required order. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Jona Neef <Jona.Neef.97@gmail.com>
Signed-off-by: NJona <25478046+NJona@users.noreply.github.com>
Signed-off-by: NJona <25478046+NJona@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wires up multi-data-center support:
--multi-dcbootstraps a second data center in the same project, sharing the VPC, jumpbox and PostgreSQL server but running its own Kubernetes and Ceph cluster. 14 VMs (~100 vCPU) and 6 static IPs, so the region's CPU quota may need raising.The secondary data center's config and vault are derived from the primary's after its secrets exist, which is why
Bootstraphandles it in a loop after the primary's config is written:DeriveDataCenterVault, keeping the postgres roles and token keys and dropping the per-cluster secrets;mode: externalpointing at the shared server's internal IP — which is what its certificate's only SAN carries — andpostgresis appended tooperations.skipin the config, so manual re-runs on the jumpbox skip it too.Review notes
validateMultiDCrejects what cannot work:--write-config=false, an explicit--datacenter-id(the IDs are derived and drive the domains), an empty--datacenter-name(the k0s cluster is named after it).--multi-dcwith--recover-configis supported: a secondary data center missing from the jumpbox is tolerated, so a second data center can be added to an existing single-DC project.--multi-dc=falsestays byte-identical to today's behaviour.Part of the
oms beta bootstrap-gcp --multi-dcstack (10 PRs). Merge in order; each PR is based on its predecessor.