Network Flow Traffic Monitoring and Analysis with Wazuh
Lab-based network security monitoring project that ingests NetFlow/IPFIX flow telemetry into Wazuh SIEM also adding network layer visibility to complement endpoint centric log monitoring through flow-based anomaly detection and threat hunting.
Network Flow Traffic Monitoring and Analysis with Wazuh
Project Description
Most SIEM deployments focus on endpoint logs: Windows Events, Linux syslog, application logs. This project adds a complementary network visibility layer by ingesting NetFlow/IPFIX flow metadata into Wazuh.
NetFlow captures communication metadata who spoke to whom, on what port, for how long, how many bytes without capturing packet payloads. That's often exactly what's needed for behavioral anomaly detection:
- "User logged in from IP" + "That IP sent 50GB outbound last night"
- "Process created on host" + "That host connected to the same IP every 5 minutes for 3 days"
- "Authentication failure" + "Followed by lateral SMB connections to 12 internal hosts"
Goals & Objectives
Build NetFlow Ingestion Pipeline
Deploy pmacctd as flow collector and build Python normalizer to convert raw flow records into Wazuh-compatible JSON format
Write Custom Wazuh Decoder
Create XML decoder to identify and parse NetFlow JSON events ingested via localfile log collection
Implement 9 Detection Rules
Write custom Wazuh rules 117000–117009 covering port scanning, data exfiltration, beaconing, lateral movement, suspicious DNS, and DoS patterns
Map to MITRE ATT&CK
Map all detection rules to corresponding MITRE ATT&CK techniques including T1046, T1041, T1071, T1021, and T1498
Validate with Synthetic Events
Generate synthetic attack scenario events using test script and confirm all rules fire correctly in Wazuh Dashboard
Architecture & Workflow
flowchart TD INTERNET(["🌐 Internet\nExternal Traffic"])
subgraph VM2["💻 VM 2 — NetFlow Collector Host"]
direction TB
NIC[/"enp1s0\nNetwork Interface"/]
PMACCT[["pmacctd\nDirect Capture"]]
RAW[("netflow-raw.json")]
NORM[["normalize_netflow_to_wazuh.py\nCron · every 1 min"]]
WAZUH_JSON[("netflow-wazuh.json")]
AGENT["Wazuh Agent\nlog_format: json"]
NIC -->|"live packets"| PMACCT
PMACCT -->|"flush 60s"| RAW
RAW --> NORM
NORM -->|"overwrite"| WAZUH_JSON
WAZUH_JSON --> AGENT
end
subgraph VM1["🖥️ VM 1 — Wazuh Server All-in-One"]
direction TB
MANAGER["Wazuh Manager\nDecoder + Rules 117000–117009"]
INDEXER[("Wazuh Indexer\nOpenSearch")]
DASHBOARD["Wazuh Dashboard\nrule.groups:netflow"]
MANAGER -->|"index"| INDEXER
INDEXER --> DASHBOARD
end
ANALYST(["🔍 Analyst\nThreat Hunting"])
INTERNET -->|"inbound/outbound"| NIC
AGENT -->|"port 1514"| MANAGER
DASHBOARD --> ANALYST
Implementation
Pipeline
- pmacctd captures live packets from
enp1s0, flushes JSON records every 60 seconds tonetflow-raw.json - Python normalizer (
normalize_netflow_to_wazuh.py) runs via cron every 1 minute, converts pmacct flat JSON to Wazuh compatible nested JSON with ECS-like field naming - Wazuh Agent reads
netflow-wazuh.jsonvialocalfilewithlog_format: json - Custom decoder (
netflow_decoders.xml) identifies NetFlow events by@timestampprefix - 9 custom rules (117000–117009) trigger on flow anomalies mapped to MITRE ATT&CK techniques
Key Technical Challenges
- Cloud hypervisor packet filtering • softflowd/nfcapd failed due to hypervisor-level packet processing; resolved by switching to pmacctd for direct interface capture
- Nested JSON conflict • Wazuh decoder field prefix mismatch with flat dot-notation JSON; resolved by restructuring output to nested JSON format
- Numeric aggregation •
data.network.bytesindexed as keyword by default; resolved via custom OpenSearch index template to enable Sum/Avg aggregation in dashboards - Internal network classification •
flow.directionmisconfiguration caused all traffic to be classified asexternal_to_external; resolved viaINTERNAL_NETWORKSenvironment variable
Tech Stack Used
SIEM platform — Manager, Indexer, and Dashboard all-in-one deployment
Direct packet capture from network interface, outputs flow records as JSON
Flow normalizer, anomaly detector, and synthetic test event generator
Wazuh Indexer backend for alert storage and aggregation queries
OS for both Wazuh Server VM and NetFlow Collector VM
Custom Wazuh decoder and 9 detection rules in XML format
Threat intelligence framework for mapping detection rules to adversary techniques
Key Features & Deliverables
normalize_netflow_to_wazuh.py
Python normalizer converting pmacct flat JSON to Wazuh-compatible nested JSON with ECS-like field naming and flow direction classification
netflow_decoders.xml
Custom Wazuh decoder identifying NetFlow JSON events and extracting source IP, destination IP, protocol, bytes, and anomaly tags
netflow_rules.xml
9 custom detection rules (117000–117009) covering port scanning, high outbound traffic, beaconing, lateral movement, suspicious DNS, and DoS patterns
detect_flow_anomalies.py
Pre-ingestion anomaly tagger that enriches flow records with behavioral tags before Wazuh processing
generate_safe_netflow_test_events.py
Synthetic test event generator for all 9 attack scenarios — validates rule firing without real malicious traffic
collect_netflow_evidence.sh
Evidence collection script for incident investigation — captures flow artifacts, alerts, and system state
Key Metrics
Results & Outcome
All 9 custom detection rules confirmed firing via synthetic test event generation. Full pipeline validated end-to-end: pmacctd → normalizer → Wazuh Agent → Manager → Dashboard. NetFlow data successfully enriched with MITRE ATT&CK context and visualized in Wazuh Dashboard for threat hunting.
Screenshots & Demo
System Architecture • 2 Virtual Machine NetFlow Pipeline
End-to-end architecture diagram showing the 2 Virtual Machine setup: VM 2 as the NetFlow Collector Host running pmacctd and Python normalizer, connected to VM 1 running Wazuh Server All-in-One (Manager, Indexer, Dashboard).
Custom NetFlow Dashboard • Wazuh OpenSearch
Custom Wazuh Dashboard showing 103,780 total flow events and 328 anomaly alerts over 24 hours. Includes visualizations for top source IPs, external inbound sources, internal-to-external traffic, destination port distribution, protocol distribution, flow direction, alert distribution by rule ID, service distribution, and MITRE ATT&CK technique mapping.
Wazuh Discover • NetFlow Events (103,224 hits)
Wazuh Discover view filtered by rule.groups:netflow showing 103,224 flow event hits over 24 hours. Each event contains enriched fields including source/destination IP, flow direction, protocol, network bytes, anomaly tags, and MITRE ATT&CK context.
Wazuh Logtest • Rule 117001 Port Scan Detection
Wazuh logtest validation output showing successful three-phase processing: pre-decoding, JSON decoding with all NetFlow fields extracted, and rule filtering triggering rule 117001 (level 9) mapped to MITRE ATT&CK T1046 Network Service Discovery with alert generation confirmed.
Interested in This Project?
Discuss your security needs or explore collaboration opportunities.