Update log collector docs for the fluentd to fluent-bit migration (EV-6164)#2815
Draft
hjiawei wants to merge 1 commit into
Draft
Update log collector docs for the fluentd to fluent-bit migration (EV-6164)#2815hjiawei wants to merge 1 commit into
hjiawei wants to merge 1 commit into
Conversation
✅ Deploy Preview for calico-docs-preview-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for tigera failed. Why did it fail? →Built without sensitive environment variables
|
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the next-version Calico Enterprise and Calico Cloud documentation to reflect the migration from the Fluentd log collector to Fluent Bit (calico-fluent-bit in calico-system), including updated filtering configuration examples, monitoring/metrics guidance, and component naming/namespace changes.
Changes:
- Replace Fluentd-specific resource names, namespaces, and architecture references with Fluent Bit equivalents across Enterprise and Cloud docs.
- Rewrite flow/DNS log filtering docs from Fluentd
<filter>syntax to Fluent Bit YAML filter lists, including the ConfigMap rename tofluent-bit-filtersand upgrade notes. - Update Prometheus monitoring docs and alert examples to Fluent Bit metrics and ServiceMonitor manifests, including new/different alert signals.
Reviewed changes
Copilot reviewed 27 out of 27 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| calico-enterprise/reference/component-resources/configure-resources.mdx | Updates LogCollector resource-override examples and verification commands to Fluent Bit. |
| calico-enterprise/reference/clis/calicoctl/cluster/diags.mdx | Updates diags output example to include calico-fluent-bit pods in calico-system. |
| calico-enterprise/reference/architecture/overview.mdx | Replaces Fluentd architecture section/link with Fluent Bit. |
| calico-enterprise/operations/monitor/prometheus/byo-prometheus.mdx | Updates BYO Prometheus scraping instructions for Fluent Bit and new ServiceMonitor reference. |
| calico-enterprise/operations/monitor/metrics/elasticsearch-and-fluentd-metrics.mdx | Retitles and updates alert examples from Fluentd metrics to Fluent Bit metrics. |
| calico-enterprise/operations/license-options.mdx | Updates license-expiration behavior text to reference Fluent Bit collector scale-down. |
| calico-enterprise/operations/comms/index.mdx | Updates certificates/services table entries to Fluent Bit-related names/namespaces. |
| calico-enterprise/observability/review-unused-network-policies.mdx | Updates example output to Fluent Bit-related policy naming. |
| calico-enterprise/observability/packetcapture.mdx | Updates packet capture retrieval steps to use calico-node pods in calico-system. |
| calico-enterprise/observability/kube-audit.mdx | Updates wording to reflect audit logs being consumed by Fluent Bit collector. |
| calico-enterprise/observability/elastic/overview.mdx | Updates Elastic logging overview to reference Fluent Bit installation/behavior. |
| calico-enterprise/observability/elastic/flow/filtering.mdx | Rewrites flow log filtering from Fluentd to Fluent Bit YAML list format; renames filters ConfigMap. |
| calico-enterprise/observability/elastic/dns/filtering-dns.mdx | Rewrites DNS log filtering docs to Fluent Bit YAML list format; renames filters ConfigMap and adds upgrade note. |
| calico-enterprise/_includes/content/_default-install.mdx | Updates default install table to mention Fluent Bit for log collection. |
| calico-enterprise/_includes/components/ReqsSys.js | Updates system requirements table wording for log storage. |
| calico-cloud/reference/component-resources/configure-resources.mdx | Updates LogCollector resource-override examples and verification commands to Fluent Bit. |
| calico-cloud/operations/monitor/prometheus/byo-prometheus.mdx | Updates BYO Prometheus scraping instructions for Fluent Bit and new ServiceMonitor reference. |
| calico-cloud/operations/monitor/metrics/elasticsearch-and-fluentd-metrics.mdx | Retitles and updates alert examples from Fluentd metrics to Fluent Bit metrics. |
| calico-cloud/operations/comms/index.mdx | Updates certificates/services table entries to Fluent Bit-related names/namespaces. |
| calico-cloud/observability/packetcapture.mdx | Updates packet capture retrieval steps to use calico-node pods in calico-system. |
| calico-cloud/observability/kube-audit.mdx | Updates wording to reflect audit logs being consumed by Fluent Bit collector. |
| calico-cloud/observability/elastic/overview.mdx | Updates Elastic logging overview to reference Fluent Bit installation/behavior. |
| calico-cloud/observability/elastic/flow/filtering.mdx | Rewrites flow log filtering from Fluentd to Fluent Bit YAML list format; renames filters ConfigMap. |
| calico-cloud/observability/elastic/dns/filtering-dns.mdx | Rewrites DNS log filtering docs to Fluent Bit YAML list format; renames filters ConfigMap and adds upgrade note. |
| calico-cloud/get-started/operator-checklist.mdx | Updates operator checklist steps/output to Fluent Bit pods/namespace and per-node component list. |
| calico-cloud/get-started/cc-arch-diagram.mdx | Updates Cloud architecture diagram table entries (component name + metrics port) for Fluent Bit. |
| calico-cloud/_includes/components/ReqsSys.js | Updates system requirements table wording for log storage. |
| ### CalicoFluentBitDaemonSet. | ||
|
|
||
| To configure resource specification for the [FluentdDaemonSet](../../reference/installation/api.mdx#fluentddaemonset), patch the LogCollector CR using the below command: | ||
| To configure resource specification for the [CalicoFluentBitDaemonSet](../../reference/installation/api.mdx#fluentbitdaemonset), patch the LogCollector CR using the below command: |
| ### CalicoFluentBitDaemonSet. | ||
|
|
||
| To configure resource specification for the [FluentdDaemonSet](../../reference/installation/api.mdx#fluentddaemonset), patch the LogCollector CR using the below command: | ||
| To configure resource specification for the [CalicoFluentBitDaemonSet](../../reference/installation/api.mdx#fluentbitdaemonset), patch the LogCollector CR using the below command: |
…-6164) Calico Enterprise removes the fluentd log collector in favour of Fluent Bit (calico-fluent-bit in calico-system; tigera/operator#4910). Update the next-version Calico Enterprise and Calico Cloud docs to match: - Rewrite "Filter flow logs" / "Filter DNS logs": examples move from fluentd <filter> syntax to Fluent Bit YAML filter lists, the ConfigMap is renamed fluentd-filters -> fluent-bit-filters, and an upgrade note explains that old fluentd-syntax filters are not translated (the operator raises a tigera status warning naming the offending key). - Metrics pages: Fluent Bit's built-in Prometheus endpoint (port 2020, /api/v2/metrics/prometheus, fluentbit_* metrics); the fluentd buffer-space alert becomes fluentbit_output_chunk_available_capacity_percent plus a dropped-chunks alert on fluentbit_output_retries_failed_total. - BYO Prometheus: the fluent-bit tab needs no client TLS (plain HTTP behind the allow-calico-fluent-bit policy); ServiceMonitor sample is fluent-bit-metrics-service-monitor.yaml. - Sweep of remaining pages: tigera-fluentd namespace -> calico-system, fluentd-node -> calico-fluent-bit (labels, secrets, packet capture retrieval via calico-node, diags output, resource override examples now use calicoFluentBitDaemonSet, architecture/overview descriptions, cc-arch-diagram ports, operator checklist). Not touched: versioned docs, releases.json (historical), and the generated installation API reference (regenerates from the operator release). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hjiawei
force-pushed
the
EV-6164-fluent-bit-docs
branch
from
5e860df to
f923e00
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Calico Enterprise replaces the fluentd log collector with Fluent Bit (
calico-fluent-bitincalico-system; tigera/operator#4910, tigera/calico-private#12538). This updates the next-version Calico Enterprise and Calico Cloud docs to match:<filter>syntax to Fluent Bit YAML filter lists; the user-filters ConfigMap is renamedfluentd-filters→fluent-bit-filters; an upgrade note explains that old fluentd-syntax filters are not read or translated and that the operator raises atigera statuswarning naming any key that fails to parse.fluentbit_output_chunk_available_capacity_percent, plus a new dropped-chunks alert onfluentbit_output_retries_failed_total. (File path kept for URL stability — happy to rename + add a redirect if preferred.)/api/v2/metrics/prometheus, guarded by theallow-calico-fluent-bitpolicy) and points at the newfluent-bit-metrics-service-monitor.yamlsample.tigera-fluentd→calico-system, daemonset/label/secret renames, packet capture retrieval via the calico-node pod,calicoFluentBitDaemonSetresource-override examples, diags output, architecture overviews, Calico Cloud arch diagram ports and operator checklist.Not touched: versioned docs,
releases.json(historical), and the generated installation API reference (_api.mdxregenerates from the operator release — until then the#fluentbitdaemonsetanchor is dangling, which the build ignores peronBrokenAnchors: ignore).Validation:
yarn build-nextreports exactly the same three broken links asmain— this PR introduces no new ones. Every resource/label/metric name was verified against the operator branch (hjiawei/operator:fluent-bit-deploy) or upstream Fluent Bit docs.Should land together with the calico-private/operator PRs for the release that ships the migration.
🤖 Generated with Claude Code