Linux support with automated detection validation#7
Open
Murat-Oruntak wants to merge 3 commits into
Open
Conversation
…uthored-By: Claude
- Align map to ATT&CK v19 (T1562.001/.006 -> T1685), re-validated - Fix ESCU rule links for Linux (develop branch + forward-slash paths) - ESCU mitre_attack_id top-level lookup for schema change (Linux-guarded) - stdin execution for multiline/quoted bash commands - Linux coverage stats count Sigma OR ESCU detections - Guard platform-suffixed output filenames so Windows keeps original names - Regenerate dist/ Linux map + MITRE Navigator layers Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds Linux support to AttackRuleMap's detection pipeline. Atomic Red Team Linux
tests run on an Ubuntu 24.04 VM (VirtualBox), auditd forwards the logs to Splunk, and a rule
counts as validated only if it actually triggers on the log the real test produced — the
same execution-based approach the project already uses for Windows.
What we start from — the full Linux surface
Two separate sets define the space:
1. Atomic Red Team — the tests
supported_platforms: linux).Not all are runnable in a log-collection lab: some are
manual(no command), some compilecode / kernel modules on the target, and some need a second machine (C2). These are noted
individually in the findings.
2. Sigma + ESCU — the rules
Sigma ships 210 Linux rules, but they split by the telemetry they require:
135 of the 210 rules (~64%) require Sysmon-for-Linux (
Image,CommandLinefields).Sysmon-for-Linux is experimental and not widely deployed, so this PR uses auditd — the
native, standard Linux audit source. That leaves 75 non-Sysmon Sigma rules (auditd +
builtin), plus Splunk's 55 ESCU
linux_auditddetections, as the usable rule surface.So the two sides are 407 tests / 118 techniques and 75 Sigma + 55 ESCU rules.
Running the tests shows which of them actually overlap.
How the validation runs
tests never contaminate each other. The Splunk search is strictly time-boxed to each
test's execution window.
Universal Forwarder ships them to Splunk, where both the Sigma-compiled SPL and the ESCU
linux_auditddetections are evaluated.it — never rewrite the rule or test to force a match.
Results
making 130 test↔rule matches in total.
Detected techniques (37):
What changed
Code (
automation/):config.py— platform-aware configuration (PLATFORM,VM_SSH_PORT,USE_PROXMOX,VM_INTERFACE,ESCU_AGG_WINDOW_SECONDS).execution_handler.py— SSH/bash executor for Linux (sudo, prereq handling, stdinexecution for multiline/quoted commands, dependency staging to the VM).
vm_handler.py— VirtualBox (VBoxManage) support alongside the existing Proxmox path.dynamic_generator.py— Linux Sigma-rule filtering, CIM-mapping skip for auditd, andcorrect ESCU rule links for the Linux ruleset.
sigma_handler.py— Linux-only Sigma-rule filtering.report_handler.py— platform-specific MITRE Navigator layers and Linux coverage stats.dependency_handler.py— dependency staging path for Linux tests.Config / helpers:
.env.example— new environment variables documented..gitignore— ignore local run artifacts.add_splunk_macros.py— helper to install the ESCU filter macros in Splunk.Outputs (
dist/):attack_rule_map_linux.json— the Linux detection map (37 techniques, 112 entries).mitre_layer_{sigma,splunk,combined}_linux.json— MITRE ATT&CK Navigator layers.Key design decisions
if config.PLATFORM == "linux"; the Windows path is unchanged (I could not test Windows,so each such spot has a comment saying so).
USE_PROXMOXflag.could be added later to unlock the 135 Sysmon Sigma rules).
VM / deployment setup
at,xclip,tcpdump,steghide,imagemagick).execve_commandand decodeproctitlewhen auditdis ingested as raw
audit.log(sourcetype=linux_audit) — exact config in the Deploymentnotes of the write-up below.
Limitations & future work
135 Sigma rules that rely on Sysmon-for-Linux fields are out of scope. Adding that telemetry
later would let them be validated too.
those cases are noted in the findings.
need a compiler/kernel toolchain, and some need a second machine. These are listed
per-technique in the findings.
it was, but I could not run the Windows pipeline to confirm.
Findings
Execution surfaces techniques a paper mapping calls "covered" but which do not detect when
run. Each was investigated and resolved into a real rule/ecosystem gap (reported, not
patched) or could not be validated on this lab (with an evidenced reason). Highlights:
a0short name while auditd logs the full path (Finding 1).(Finding 3).
CONFIG_CHANGE(Finding 5).
rule's procedure, and rules that require the (unpopulated) CIM Endpoint datamodel.
The full write-up — with end-to-end evidence for every non-detected technique, plus the exact
Splunk onboarding config — is in the collapsible section below.
Full findings write-up — end-to-end evidence for every non-detected technique (click to expand)
This document records what surfaced while validating Linux detection coverage (Atomic Red
Team tests → auditd → Splunk → Sigma/ESCU rules). It falls into a few kinds: real gaps in
the detection rules themselves; data-onboarding / deployment lessons about how auditd
has to reach Splunk before existing rules can fire; and engineering notes on detections
that only fire once the lab runs the test faithfully. Each item says clearly which one it is.
Scope and method
Sigma's Linux rule set (210 rules total) splits by required telemetry:
135 of 210 rules (~64%) are written against Sysmon-for-Linux fields (
Image,CommandLine, ...). Sysmon-for-Linux is not production-mature and not widely deployed,so this lab uses auditd — the native, standard Linux audit source. That leaves the
75 non-Sysmon Sigma rules (auditd + builtin), plus Splunk's 55 ESCU
linux_auditddetections, as the realistic detection surface. On the test side, Atomic Red Team ships
407 Linux tests across 118 techniques.
Method. We ran every test and checked which rules actually fired. When a technique had a
usable (auditd/ESCU) rule and a runnable Linux test but still didn't detect on the first
pass, we dug into it one by one. A paper mapping of ID-to-ID would call these "covered";
running them shows they are not. Each one ended up in exactly one of three buckets:
detected after a lab/onboarding fix, a real rule/ecosystem gap (reported, not
patched), or could not be validated on this lab (each with a concrete reason).
Result: 37 ATT&CK techniques detected (aligned to ATT&CK v19), entirely through lab and
data-onboarding correctness, without altering any rule or test.
Detections that required lab/onboarding correctness
Every fix here follows the same rule: run the real test, watch the real detection rule
fire, and change neither of them. We only fix (a) how the pipeline runs the test, and
(b) how the auditd logs are loaded into Splunk, so the data looks the way the (unchanged)
rules already expect.
A. Test-execution fixes (the pipeline now runs the test as written)
(e.g.
if [ "$(uname)" = 'FreeBSD' ]; then cmd="netstat -ant"; fi) broke when wrapped insh -c "…". The Linux path now feeds such scripts over stdin (sh -s). UnblockedT1016 and is a prerequisite for many multiline tests.
Atomic
get_prereqonly echoes a reminder for; installing it and re-snapshottingenabled T1053.002 (
at/atd).B. Detection data-onboarding — reconciling an ingestion difference (not a rule gap)
Why they didn't fire. The ESCU
linux_auditdrules expect auditd data in the exactshape the official Splunk Add-on for *nix produces. That add-on ingests auditd through
a script (
bin/rlog.sh) that runsausearch -i, which decodes the hex-encodedproctitleinto readable text, and its[auditd]stanza builds a joinedexecve_commandfield — all under
sourcetype=auditd. This lab does something simpler and very commoninstead: a Universal Forwarder tails the raw
/var/log/audit/audit.log(
sourcetype=linux_audit). In that setupproctitlestays hex-encoded andexecve_commandis never built, so the ESCU rules silently do nothing — not because the rules are wrong,
but because the data reached Splunk in a different shape than they expect. (Two signs this
is a known issue: the
linux_auditdmacro ships as a fill-in-yourself placeholder —"Replace the macro definition with configurations for your Splunk Environment" — and ESCU
even shipped a now-deprecated
linux_auditd_normalized_proctitle_processmacro thathex-decodes
proctitlefor exactly this situation.)So this is a deployment issue, not a rule or ecosystem gap — but worth documenting,
because tailing the raw
audit.logis a common setup and these rules fail silently under it.We rebuilt the same fields the official ingestion produces, in Splunk props/transforms for
the
linux_auditsourcetype (rules and tests untouched):execve_commandrebuilt — the joined command lineausearch -i/ the Add-on's[auditd]stanza produce; enabled T1083, T1115, T1140, T1552.004, T1548.001.proctitlehex-decoded — the same decodeausearch -i(and the deprecated ESCUmacro) perform, done here in pure SPL (
printf("%c", …)); enabled T1485, T1003.008,T1053.006, T1548.003.
Two adjacent fixes here are genuine (not ingestion-specific):
mitre_attack_idmoved to top-level — a Splunksecurity_contentschema changethat broke ESCU technique matching (returned zero rules) for every consumer of the
current repo; corrected the lookup.
bucket span=15m); thepipeline's per-test search window was widened so a test falls inside a full bucket.
C. Keeping the map current with ATT&CK
promoted/moved to T1685 ("Disable or Modify Tools") in ATT&CK v19. The map was
re-aligned to the current ID and re-validated by a fresh T1685 run (ASLR, Auditing,
and Logging config-change tests all fired). Same detections, current taxonomy.
The rest of this document explains each residual finding: what the rule expected, what
actually happened in auditd/Splunk, and why detection did not fire — all verified
end-to-end.
Finding 1 — auditd
a0full-path vs. short-name mismatchMany Sigma auditd rules match the EXECVE
a0field with an exact short name:But
a0is simplyargv[0]— the exact name the program was launched with. When atool is run through
sudoor by its absolute path (which is howiptablesand otheradmin tools are normally invoked),
a0holds the full path, not the short name.Evidence (Splunk, 7-day window):
a0="/usr/sbin/iptables"→ 252 recordsa0="iptables"(short name) → 0 recordsBecause the rule requires an exact match on
a0: 'iptables', it never matchesthe real records (
/usr/sbin/iptables), producing a silent false negativefor these tools.
Assessment: The weakness is in the rule, not in our pipeline. A better rule would use
a0|endswith: 'iptables'. We deliberately do not rewritea0inside the pipeline: thewhole question we are answering is "does this rule fire as written?", and cleaning up the
data ourselves would hide the real answer. So we run the rule as-is and report the gap.
Finding 2 — Orphan rules (rule exists, but no Atomic test exercises it)
lnx_auditd_capabilities_discovery.yml(taggedattack.t1083andattack.t1548) matches:getcap -r /manuallyproduced one EXECVE record and the rule matched it in Splunk (verified at
15s/60s/120s after execution).
getcap— not under T1083, not underT1548, nowhere in the test set.
Conclusion: This
getcapSigma rule is sound but unverifiable with availabletests — no Atomic test runs
getcap, so we do not claim it as validated (doing so wouldrequire editing the rule or hand-crafting a command, both of which break the contract).
Update (post-recovery): the technique T1083 is now detected — via a different,
ESCU rule (Linux Auditd File and Directory Discovery) that fired once
execve_commandwas onboarded (see §B). So T1083 is covered; the observation above narrows to
"this one
getcapSigma rule remains an orphan," not "the technique is uncovered."Finding 3 — auditd argument-position assumption (
a1vsa2)Sigma auditd rules often pin a match to a specific positional argument field
(
a1,a2, ...). But auditd records command arguments positionally, and theposition depends on the order the test author wrote them — which is not fixed.
Example:
lnx_auditd_dd_delete_file.yml(T1485) compiles to this SPL:The rule expects
if=/dev/zeroina1. But the Atomic test runs:Because
of=comes first, auditd records:Evidence (verified end-to-end):
a1 IN (...if=/dev/zero...)→ 0 results.a1=of=/var/log/syslog,a2=if=/dev/zero.So
if=/dev/zerolands ina2, nota1, and the rule never fires — a silentfalse negative. The command genuinely ran and was logged and forwarded; only the
rule's positional assumption is wrong.
Variant: combined short flags (T1040)
The same brittleness appears with how flags are written.
lnx_auditd_network_sniffing.yml(T1040) requires:
The Atomic test runs
tcpdump -c 5 -nnni enp0s3, which auditd records as:a0/a1match, buta3|contains: '-i'fails: the test combines the flags into-nnni, so the substring-i(dash immediately followed byi) never appears.The rule assumes
-iis a separate token; the test bundles it. Verified: tcpdump ran(
listening on enp0s3), the record reached Splunk (a0=tcpdump), but the rulereturned 0.
Variant: arguments split across two commands (T1115)
lnx_auditd_clipboard_collection.yml(T1115) requires all four constraints in asingle EXECVE:
i.e. one invocation
xclip -sel clip -o. But the Atomic test runs xclip twice:auditd records two separate EXECVE events:
Evidence (verified end-to-end): running the rule SPL against each record
individually returns 0 and 0; combined 0. DEBUG confirms the pipeline really issued
the two separate xclip commands. xclip ran and was logged (not a headless/X11 issue) —
the rule's "all args in one command" assumption simply doesn't hold.
Assessment: Same family as Finding 1 — these Sigma auditd rules are too strict about
where an argument sits (
a1vsa2), how a flag is written (-ivs-nnni), and whethereverything lands in one command. A more robust rule would match on the whole EXECVE content
instead of a fixed
aNslot. As before, we leave the rule as-is and report the gap ratherthan rewriting it to force a match.
Update (post-recovery): two of the three techniques above are now detected — not
by fixing the brittle Sigma rule, but because each also has an ESCU rule that keys on a
whole-command field rather than a fixed
aN, and that field is now onboarded:proctitle(oncehex-decoded, §B); the command line contains all needed tokens regardless of order. The
Sigma
a1-vs-a2gap still stands for the Sigma rule.execve_command; eachxclip invocation now matches on its own. The Sigma "all args in one command" gap still
stands for the Sigma rule.
-iflag token,which the test bundles as
-nnni. Genuine residual finding.A useful takeaway: rules that match on the whole command line (
proctitle/execve_command) are not thrown off by argument order or flag spacing — the very thingsthat break Sigma rules pinned to a fixed
aNslot.Finding 4 — distro-family assumption (RHEL paths on a Debian-family VM)
Some Sigma rules hard-code file paths that only exist on one Linux distro family.
lnx_auditd_keylogging_with_pam_d.yml(T1056.001) matches:password-authandsystem-authare RHEL-family files (RHEL, CentOS, Fedora,Rocky, Alma). The lab VM is Debian-family (Ubuntu 24.04), which uses
common-auth,common-password,common-accountinstead — the RHEL files do not exist.Evidence (verified directly on the VM):
test -f /etc/pam.d/password-auth→ absent;system-auth→ absent.common-*PAM files exist (ls /etc/pam.d/).password-auth; 177pam.dPATHrecords, all
common-*; 0TTY/USER_TTYrecords.The Atomic test itself guards with
if sudo test -f /etc/pam.d/password-auth; then ...,so on Ubuntu it simply does nothing — no file is touched, the rule never fires.
Assessment: This is a real coverage blind spot, not a lab artifact. On Debian-family
hosts (very common in production) this rule silently misses PAM-based keylogging because
it does not include the Debian PAM paths (
common-auth, etc.). Static mapping wouldcount T1056.001 as covered; validation shows it is not covered on this distro family.
Per project policy we report the gap rather than editing the rule.
Finding 5 — rule queries the wrong telemetry source (Sysmon fields for an auditd-native action)
Technique: T1685.004 (Disable or Modify Tools — disable/clear the Linux Audit System).
The two Atomic tests run and succeed:
auditd captures both perfectly, in its native
CONFIG_CHANGErecord:So the telemetry needed to detect this technique exists and is unambiguous. The problem is
the rules mapped to T1685.004:
Image="*/auditctl" | regex CommandLine="-D"— i.e. Sysmon process fields (
Image,CommandLine). On an auditd host these are notpopulated, so the rule cannot fire even though the action was fully logged.
type=DAEMON_ABORT/END/START— butauditctl -D/-e 0change configuration; they do not stop the daemon, so thoselifecycle events never appear for this test.
Assessment: Not a "cannot validate" case (the test runs and the action is logged) and
not a missing-data case (the
CONFIG_CHANGEevent is right there). It is a rule/telemetry-source mismatch: the only auditctl-specific rule was written against
Sysmon-for-Linux telemetry, while the native auditd
CONFIG_CHANGEevent that perfectlycaptures "disable/clear auditd" is queried by no mapped rule. Per project policy we
report the gap rather than authoring a new
CONFIG_CHANGE-based rule.Methodology note
These findings rely on a key infrastructure fact established during the
investigation: the lab chain (VM → auditd → Splunk Universal Forwarder → Splunk →
Sigma/ESCU) is sound. This was proven by re-running a previously-detected,
non-EXECVE technique (T1489, Magic SysRq) which detected correctly, and by
confirming EXECVE records forward into Splunk normally. The non-detections above
are therefore content/mapping issues, not lab failures.
Outcome summary
Every investigated technique was resolved into one of three buckets:
Detected — via pipeline/lab fixes, not by changing any rule:
(C source for the shared library was never staged to the VM, so the prereq
gccstep failed; it now compiles and the rule fires).
Two rules watched the
connectsyscall with different keys:atomic.ruleshad
-S connect -k networkandsigma.ruleshad-S connect -k network_connect_4.auditd tags an event with the first matching rule's key only (it stops at the
first match), and
atomic.rulesloads first (alphabetical), so every nmapconnectwas written to the log as
key=network. The Sigma rule searches forkey=network_connect_4, which therefore never appeared in the log — not because thesearch missed it, but because auditd never wrote that key. No Sigma rule uses
key=network, so deleting that line is safe; afterwardsconnectevents carrynetwork_connect_4and the rule fires (Count 11 and 75). This is a VM/auditd-configfix, not a code change — it lives in the lab snapshot, so reproducing it requires the
same auditd rule set.
ingestion produces (see §B):
execve_commandrebuilt → T1083, T1115, T1140, T1552.004, T1548.001proctitlehex-decoded → T1485, T1003.008, T1053.006, T1548.003mitre_attack_idtop-level + aggregation window + stdin execution → T1016at).Auditing + Logging config-change tests re-fired under the current ID).
Findings (rule/ecosystem gaps, reported not patched):
-nnni); no whole-commandrule to fall back on, so still uncovered.
a0=/usr/sbin/iptables).native auditd
CONFIG_CHANGEevent that captures it is queried by no rule).Could not be validated — the test cannot be exercised on this lab, each for a concrete
reason (not a rule gap):
native code (or a kernel module) on the target before running: a C
src/directorybuilt with
make(output into a siblingbin/the Makefile assumes exists), and forT1014/T1547.006 an
insmodof a freshly built module into the running kernel. Validatingthem needs a full build toolchain (compiler, kernel headers, the
bin/layout,module-loading) — beyond the auditd-telemetry validation this lab performs, so out of
scope for this PR. (A staging fix — recursively uploading the
src/dir — was prototypedand confirmed necessary-but-not-sufficient; reverted to keep the pipeline minimal.)
manualexecutor(no command) or run
python http.server, whose only rule is Sysmon-shaped(
Image/CommandLine); the auditd rule looks forwget --post-file, which no test runs.systemctl … start/enable, but the only test runspsexec.py(Impacket SMB to127.0.0.1), which never issues
systemctl startand has no SMB service to reach.by touching files (
rm -rf,truncate,unlink,cat /dev/null >), while the onlyauditd rule matches a different procedure:
SYSCALL=syslog a0 IN (4,5,6)=dmesg -c-styleclearing of the kernel ring buffer. No test runs
dmesg; the file-deletion rules areCIM. Independent ecosystems picked non-overlapping procedures of the same technique.
authorized-keys rules are all Endpoint/CIM (not populated here); the one auditd rule
watches the sshd config file, whereas the test modifies
~/.ssh/authorized_keys(a different file — its access was logged: 3 PATH events).
(ICMP Reverse Shell) uses the
manualexecutor and needs a second machine (C2) plusdownloaded binaries; the mapped rules are Network/CIM (not populated).
supported_platforms: containers, notlinux, so the pipeline'slinuxfilter never runsthem; they also need a Docker host with privileged containers, which is outside this VM lab.
Its one Linux-tagged rule (Docker Root Directory Mount) is an Endpoint/CIM rule (not
populated here).
the new IDs and removed the old folders, so the pipeline finds no Linux test under the
old IDs (they only appeared via a stale ID reference in the coarse filter).
Deployment notes (reproducing the onboarding fixes)
Two of the fixes in §B (
execve_command,proctitle) are Splunk-side onboarding config,not pipeline code, so they must be present in any Splunk instance that ingests auditd as
raw
audit.log/sourcetype=linux_audit(as this lab does) rather than through theofficial
Splunk_TA_nixscripted input (bin/rlog.sh→ausearch -i, which alreadydecodes
proctitleand joins the command line, landing assourcetype=auditd). The configbelow reproduces that transform for
sourcetype=linux_auditso the unmodified ESCU rulesfire. Both were applied via the Splunk REST API and shared globally; the equivalent
props.conf/transforms.confis:(Splunk
evalhas no native hex→ASCII, butprintf("%c", n)acts aschr(), which is whatmakes the pure-SPL proctitle decode possible.)
CIM / Endpoint datamodel (out of scope). Several ESCU rules (and the authorized-keys
rules under T1098.004, plus T1027, T1036.004, T1037.004, T1070.004, T1572, T1611) query
datamodel=Endpoint.*viatstats. That datamodel is not populated here. CIM issource-agnostic (it can be fed from auditd), but doing so is a full data-engineering effort
— mapping auditd's split SYSCALL/EXECVE/PATH records into single CIM process/file events,
including
parent_process, which auditd does not provide natively — i.e. re-implementing whatSysmon-for-Linux gives for free. Left out of scope for this auditd-focused PR.