Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .agents/skills/project-create-topology/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,13 @@ For SNMP managed device actor modals:
- `l3_subnet` links represent shared-subnet logical L3 adjacency between
resolved managed SNMP device actors. They must use explicit L3 link/evidence
types and must not be presented as discovery, physical, or L2 links.
- Protocol-specific L3 adjacency, such as `ospf_adjacency`, is control-plane
logical adjacency. It must not be presented as discovery, physical, L2, or
port-neighbor evidence.
- Preserve protocol-neighbor diagnostics that do not become graph links in
actor-owned detail tables, such as `actor_ospf_neighbors`. Non-full or
unresolved OSPF neighbors should remain visible there without creating loose
router/IP actors.
- Keep generic graph-link `Links` sections only for endpoint, segment, or
custom actors that do not own port inventory.
- Build link endpoint port labels only from real port fields: `port_name`,
Expand Down
26 changes: 24 additions & 2 deletions .agents/sow/specs/topology-function-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -604,6 +604,24 @@ source/destination interface IPs, subnet, network, netmask, prefix, source, and
logical inference/attachment metadata. The evidence is an actor-modal source
for L3 adjacency drilldowns.

SNMP `ospf_adjacency` links are logical OSPF control-plane adjacency, not
physical, L2, discovery, or port-neighbor evidence. They use
`orientation: observed_bidirectional`, `direction_role: observation`, and
`semantic_role: control`. Their graph presentation should use a dashed line
style and `purple` color slot so operators do not read them as physical cabling
or generic success/health state. The producer emits
graph links only for full OSPF neighbors between resolved managed SNMP device
actors. Non-full or unresolved OSPF neighbor rows remain diagnostic actor-owned
detail rows and must not create loose router/IP graph actors.
Current SNMP OSPF topology scope is OSPFv2 non-virtual neighbors from
`ospfNbrTable`; OSPFv3 and OSPF virtual-neighbor tables are future scope unless
a later topology feature explicitly adds their distinct semantics.

When `ospf_adjacency` and `l3_subnet` describe the same resolved actor pair and
endpoint/subnet relationship, OSPF is the stronger protocol-specific signal for
the graph. The producer may suppress the matching `l3_subnet` graph link while
preserving typed OSPF evidence and actor-owned neighbor detail rows.

SNMP modal composition must be port-centric for managed device actors. A managed
device modal uses actor-label identification for important device facts, a
primary `Ports` section over `actor_ports`, and a `Port Neighbors` section over
Expand Down Expand Up @@ -631,8 +649,12 @@ It exists so device modals can align neighbor rows with the same port identity
shown in `actor_ports`; it is not a second copy of raw evidence.

SNMP `actor_port_links` must remain port-neighbor oriented. It must not include
`l3_subnet` links, because shared-subnet L3 adjacency does not prove a physical
or L2 port neighbor.
`l3_subnet` or `ospf_adjacency` links, because shared-subnet and
routing-protocol L3 adjacency do not prove a physical or L2 port neighbor.

SNMP protocol-neighbor detail sections, such as OSPF neighbors, should be
actor-owned detail tables. They may include unresolved or non-full neighbor
observations for diagnostics even when no graph link is emitted.

SNMP polished UI must not depend on raw `actor_metadata` and endpoint JSON.
Important scalar/count summary values live in typed actor or actor-detail
Expand Down
27 changes: 27 additions & 0 deletions .agents/sow/specs/topology-modes-correlation-aggregation.md
Original file line number Diff line number Diff line change
Expand Up @@ -515,6 +515,21 @@ state collected from IP-MIB `ipAddrTable`, limited to IPv4 point-to-point
same point-to-point subnet; it does not prove that the devices are physically
connected.

`ospf_adjacency` represents OSPF control-plane adjacency between two resolved
managed SNMP device actors. It is a logical L3 routing-protocol relationship,
not physical, L2, discovery, or port-neighbor evidence. The producer emits graph
links only for full OSPF neighbors with resolved managed endpoints. Non-full or
unresolved OSPF neighbor rows remain diagnostic actor-owned detail rows and
must not create loose router/IP graph actors.
Current SNMP OSPF topology scope is OSPFv2 non-virtual neighbors from
`ospfNbrTable`; OSPFv3 and OSPF virtual-neighbor tables are future scope unless
a later topology feature explicitly adds their distinct semantics.

When `ospf_adjacency` and `l3_subnet` describe the same resolved actor pair and
endpoint/subnet relationship, OSPF is the stronger protocol-specific signal for
the graph. The producer may suppress the matching `l3_subnet` graph link while
preserving typed OSPF evidence and actor-owned neighbor detail rows.

SNMP is not a loose-side topology. Every graph link should have two actors in
both Agent and aggregator views. `l3_subnet` follows the same rule: unmatched
or ambiguous L3 endpoints are diagnostic/suppression state, not materialized
Expand Down Expand Up @@ -563,6 +578,14 @@ evidence type. Replacement may rewire endpoint actors to stronger managed
device actors, but it MUST NOT reinterpret `l3_subnet` as discovery protocol
evidence or port-neighbor evidence.

For `ospf_adjacency`, aggregation MUST preserve the explicit `ospf_adjacency`
link and evidence type, including its `semantic_role: control`. Replacement may
rewire endpoint actors to stronger managed device actors, but it MUST NOT
reinterpret OSPF adjacency as discovery, physical, L2, or port-neighbor
evidence. Actor-owned OSPF neighbor detail rows should remain attached to the
local device actor and are not loose-side graph materialization input unless a
future producer contract explicitly defines such a policy.

### UI SNMP

The UI should not expose a detailed/aggregated toggle for SNMP unless the
Expand All @@ -581,6 +604,10 @@ Device modals should remain port-centric:
relationship rows so local port identity never contradicts the port table.
- L3 adjacency information: derived from typed `l3_subnet` relationship
evidence, not from `actor_port_links`.
- routing protocol neighbor information: derived from typed actor-owned detail
rows such as `actor_ospf_neighbors`, not from `actor_port_links`. The modal
may show unresolved or non-full protocol neighbors as diagnostics without
creating graph links.

## Streaming

Expand Down
20 changes: 13 additions & 7 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,10 @@ if(ENABLE_NETDATA_JOURNAL_FILE_READER OR ENABLE_PLUGIN_OTEL OR ENABLE_PLUGIN_OTE

if(ENABLE_PLUGIN_NETFLOW)
set(NETFLOW_PLUGIN_RUSTFLAGS "")
set(NETFLOW_PLUGIN_ENV_VARS
"NETDATA_BUILD_CACHE_DIR=${CACHE_DIR}"
"NETDATA_BUILD_LIB_DIR=${VARLIB_DIR}"
"NETDATA_BUILD_STOCK_DATA_DIR=${STOCK_DATA_DIR}")

# We depend (transitively) on the `io-uring` crate which doesn't provide
# prebuilt bindings for 32-bit arches. Considering this is a compile-time
Expand All @@ -318,8 +322,7 @@ if(ENABLE_NETDATA_JOURNAL_FILE_READER OR ENABLE_PLUGIN_OTEL OR ENABLE_PLUGIN_OTE
# The workspace release profile still propagates fat LTO, single-codegen-unit,
# and debuginfo settings through Rust dependencies on 32-bit builds. Override
# Cargo's release profile too so the entire dependency graph stays linkable.
corrosion_set_env_vars(
netflow-plugin
list(APPEND NETFLOW_PLUGIN_ENV_VARS
"CARGO_PROFILE_RELEASE_DEBUG=0"
"CARGO_PROFILE_RELEASE_LTO=off"
"CARGO_PROFILE_RELEASE_CODEGEN_UNITS=8")
Expand All @@ -332,6 +335,8 @@ if(ENABLE_NETDATA_JOURNAL_FILE_READER OR ENABLE_PLUGIN_OTEL OR ENABLE_PLUGIN_OTE
if(NETFLOW_PLUGIN_RUSTFLAGS)
corrosion_add_target_rustflags(netflow-plugin ${NETFLOW_PLUGIN_RUSTFLAGS})
endif()

corrosion_set_env_vars(netflow-plugin ${NETFLOW_PLUGIN_ENV_VARS})
endif()
endif()

Expand Down Expand Up @@ -519,6 +524,7 @@ endif()
include(NetdataJSONC)
include(NetdataYAML)
include(NetdataBacktrace)
include(NetdataSQLite)
include(NetdataDlib)

if(ENABLE_LEGACY_EBPF_PROGRAMS)
Expand Down Expand Up @@ -639,6 +645,8 @@ if(ENABLE_LIBBACKTRACE)
netdata_bundle_libbacktrace()
endif()

netdata_bundle_sqlite3()

#
# check source compilation
#
Expand Down Expand Up @@ -1747,11 +1755,6 @@ set(RRD_PLUGIN_FILES
src/database/sqlite/sqlite_aclk_node.h
src/database/sqlite/sqlite_aclk_alert.c
src/database/sqlite/sqlite_aclk_alert.h
src/database/sqlite/vendored/sqlite3.c
src/database/sqlite/vendored/sqlite3.h
src/database/sqlite/vendored/sqlite3recover.c
src/database/sqlite/vendored/sqlite3recover.h
src/database/sqlite/vendored/dbdata.c
src/web/api/queries/KolmogorovSmirnovDist.c
src/web/api/queries/KolmogorovSmirnovDist.h
src/database/rrdfunctions-inflight.c
Expand Down Expand Up @@ -1904,6 +1907,7 @@ set(STREAMING_PLUGIN_FILES
src/streaming/stream-parents.c
src/streaming/stream-handshake.c
src/streaming/protocol/command-function.c
src/streaming/protocol/command-function-del.c
src/streaming/protocol/command-host-labels.c
src/streaming/protocol/command-chart-definition.c
src/streaming/protocol/command-begin-set-end-v2.c
Expand Down Expand Up @@ -2493,6 +2497,8 @@ if(HAVE_LIBBACKTRACE)
netdata_add_libbacktrace_to_target(libnetdata)
endif()

netdata_add_sqlite3_to_target(libnetdata)

if(OS_WINDOWS)
set(HAVE_ETW True)
set(HAVE_WEL True)
Expand Down
2 changes: 1 addition & 1 deletion docs/network-flows/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The static install (the kickstart `--static-only` path) bundles the plugin autom

- A working Netdata Agent on the host that will receive flow data.
- That host must be reachable on UDP from your routers and switches (default port `2055`).
- Linux. The plugin is Linux-only.
- A Netdata installation that includes `netdata-plugin-netflow`. Native Linux packages install it as a separate package; static installs bundle it automatically; source builds need a Rust toolchain.

## Install on Debian / Ubuntu / Mint

Expand Down
6 changes: 3 additions & 3 deletions docs/snmp-traps/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ Job names must be 64 characters or shorter, start with a letter or digit, and ma
| Enrichment | `reverse_dns.enabled` | `false` | Adds best-effort PTR annotation as `TRAP_REVERSE_DNS`. Never used as authoritative identity. |
| Storm controls | `rate_limit` | disabled | Optional per-source token-bucket rate limiting. |
| Storm controls | `dedup` | disabled | Optional suppression of repeated identical traps inside a window. |
| Outputs | `journal.enabled` | `true` | Writes decoded traps to local journal-compatible files and exposes listener jobs through the `snmp:traps` Function. Linux only. |
| Outputs | `journal.enabled` | `true` | Writes decoded traps to local journal-compatible files and exposes listener jobs through the `snmp:traps` Function. |
| Outputs | `otlp` | disabled | Optional OTLP/gRPC Logs export. |
| Storage | `retention` | `max_size: 10GB` | Per-job direct journal retention and rotation. Ignored when `journal.enabled` is `false`. |
| Meaning | `overrides` | `[]` | Per-OID category, severity, and label overrides on top of profile defaults. |
Expand Down Expand Up @@ -319,7 +319,7 @@ The listener can write to the direct journal, export to OTLP/gRPC, or do both.

| Backend | Enabled by default | Local querying | Notes |
|---|---:|---:|---|
| Direct journal | yes | yes | Stores journal-compatible files under the Netdata log directory and exposes the job through the `snmp:traps` Function. Linux only. |
| Direct journal | yes | yes | Stores journal-compatible files under the Netdata log directory and exposes the job through the `snmp:traps` Function. |
| OTLP/gRPC Logs | no | no | Exports traps as OTLP LogRecords to an external collector. |

When both backends are enabled, traps go to both outputs and the local journal is authoritative: an OTLP export failure does not affect what was already written to the journal. When journal is disabled and OTLP is enabled, no local journal files are created and the job does not appear as a local SNMP trap log source.
Expand All @@ -328,7 +328,7 @@ Trap rows can contain sensitive operational payloads, not only credentials. Revi

### Direct journal

Direct journal storage is enabled by default and requires Linux:
Direct journal storage is enabled by default:

```yaml
journal:
Expand Down
2 changes: 1 addition & 1 deletion docs/snmp-traps/forwarding-to-siem.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ OTLP-only jobs create no local trap journal files and do not appear as job sourc

## Direct journal backend

Direct journal output is enabled by default for listener jobs and requires Linux. On non-Linux systems, jobs with `journal.enabled: true` fail validation; use OTLP-only mode instead.
Direct journal output is enabled by default for listener jobs. It writes local trap log files that can be read with `journalctl --directory` on systems where `journalctl` is available. Use OTLP-only mode when the external OTLP receiver is the system of record and local trap files are not needed.

```yaml
journal:
Expand Down
8 changes: 4 additions & 4 deletions docs/snmp-traps/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,16 @@ endmeta-->

Netdata receives SNMP traps with the **snmp_traps** collector in `go.d.plugin`. Trap receiving is explicit: Netdata does not create a trap listener job or configure devices to send traps by itself. You must create an explicit listener job before Netdata can receive traps.

The default direct-journal backend requires Linux. OTLP-only jobs are not blocked by the Linux direct-journal requirement, but they do not create local journal files or local SNMP trap log sources.
The default direct-journal backend writes local trap log files on every supported platform. OTLP-only jobs do not create local journal files or local SNMP trap log sources.

The default `listen.endpoints` template uses UDP `0.0.0.0` on port `162`. The stock `go.d/snmp_traps.conf` file ships as a commented template; Netdata does not bind any UDP port until a job is uncommented, created, or applied with Dynamic Configuration.

## Prerequisites

- A Netdata Agent host with the `go.d.plugin` component installed.
- A Linux host when you want the default direct-journal backend or local SNMP trap log sources.
- At least one output backend enabled per listener job: `journal.enabled: true` on Linux (the default), `otlp.enabled: true`, or both.
- For direct-journal jobs on Linux, the files `/etc/machine-id` and `/proc/sys/kernel/random/boot_id` must exist, be readable, and contain non-empty values. Minimal containers, chroots, and embedded systems should check these before relying on local journal storage.
- A host with writable Netdata log and state directories when you want the default direct-journal backend or local SNMP trap log sources.
- At least one output backend enabled per listener job: `journal.enabled: true` (the default), `otlp.enabled: true`, or both.
- For direct-journal jobs, Netdata must be able to resolve a stable local Agent identity and boot identity. Minimal containers, chroots, and embedded systems should validate local journal storage before relying on it.
- UDP reachability from each trap sender to the Netdata host.
- Network devices configured to send SNMP Trap or INFORM notifications to the Netdata host IP and listener port.
- Firewall, security group, ACL, NAT, and host firewall rules that allow the selected UDP port.
Expand Down
2 changes: 1 addition & 1 deletion docs/snmp-traps/validation-and-data-quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Work through these checks in order during first deployment. Use [Quick Start](/d
| Varbind quality | `TRAP_VAR_*`, `TRAP_JSON`, `snmp.trap.errors` `template_unresolved`, `binary_encoded` | Important event payload is queryable through indexed fields, with structured payload and audit details available in `TRAP_JSON`; binary-encoded journal fields are counted and need review. |
| Deduplication | `TRAP_REPORT_TYPE=deduplication_summary`, `TRAP_SUPPRESSED_COUNT`, `TRAP_SUPPRESSED_FINGERPRINTS`, `snmp.trap.dedup_suppressed` | Repeated traps are summarized only when deduplication is intentionally enabled and the fingerprint matches operational expectations. |
| Rate limiting | `snmp.trap.errors` `rate_limited`, job `rate_limit.mode` | Over-limit traffic is counted by `rate_limited`. In `drop` mode the packet is discarded; in `sample` mode it is counted and still enters the pipeline. |
| Journal health | `snmp.trap.pipeline` `committed`, `write_failed`, `snmp.trap.errors` `journal_write_failed`, direct journal path | Healthy direct-journal jobs commit rows and do not report write failures. Direct journal output is Linux-only. |
| Journal health | `snmp.trap.pipeline` `committed`, `write_failed`, `snmp.trap.errors` `journal_write_failed`, direct journal path | Healthy direct-journal jobs commit rows and do not report write failures. Local `journalctl` validation requires `journalctl` on the host where you query the files. |
| OTLP health | `snmp.trap.errors` `otlp_export_failed`, downstream receiver records, OTLP resource attributes | OTLP export succeeds, records have `service.name=netdata-snmptrap` and the expected `service.instance.id`, and downstream records contain the expected trap attributes. |
| Retention | Job `retention` settings and direct journal files | Local retention keeps enough trap history for investigation without unexpectedly evicting needed rows. |
| Downstream SIEM fields | Ingested field names or OTLP attributes | SIEM rules use the fields that actually arrive, especially report type, source, OID/name, category, severity, and payload fields. |
Expand Down
1 change: 1 addition & 0 deletions packaging/cmake/Modules/NetdataGoTools.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ set(GO_LDFLAGS "${GO_LDFLAGS} -X ${BUILDINFO_PKG}.PluginsDir=${PLUGINS_DIR}")
set(GO_LDFLAGS "${GO_LDFLAGS} -X ${BUILDINFO_PKG}.UserConfigDir=${CONFIG_DIR}")
set(GO_LDFLAGS "${GO_LDFLAGS} -X ${BUILDINFO_PKG}.StockConfigDir=${LIBCONFIG_DIR}")
set(GO_LDFLAGS "${GO_LDFLAGS} -X ${BUILDINFO_PKG}.CacheDir=${CACHE_DIR}")
set(GO_LDFLAGS "${GO_LDFLAGS} -X ${BUILDINFO_PKG}.VarLibDir=${VARLIB_DIR}")
set(GO_LDFLAGS "${GO_LDFLAGS} -X ${BUILDINFO_PKG}.LogDir=${LOG_DIR}")

# add_go_target: Add a new target that needs to be built using the Go toolchain.
Expand Down
Loading
Loading