NetFlow Network Traffic Monitoring
Network flow visibility integrated into Wazuh SIEM using pmacctd, Python normalization, and 24 custom detection rules validated against real internet traffic.
NetFlow Network Traffic Monitoring
Project Description
Built a complete network flow monitoring pipeline that integrates traffic metadata into Wazuh SIEM on a two VM architecture. The system captures raw flow data using pmacctd, normalizes it via a custom Python script, and forwards it to Wazuh for detection using 24 custom rules covering 10 threat categories.
Validated against real internet traffic on a live cloud VM. Within hours of deployment, the pipeline detected automated RDP scanners, Telnet crawlers, MySQL and PostgreSQL port scanners, NetBIOS broadcasts, and C2 beaconing patterns.
Goals & Objectives
Build NetFlow Pipeline
Integrate pmacctd traffic capture with Wazuh SIEM using Python normalization on a two-VM architecture
Write Detection Rules
Develop 24 custom Wazuh rules covering 10 threat categories including RDP, Telnet, database scanning, and C2 beaconing
Validate Against Real Traffic
Expose VM to internet and confirm detections firing against real automated scanners and threat actors
Build SOC Dashboard
Create 15+ visualizations in OpenSearch Dashboard including heatmap, timeline, and attack detail table
Architecture & Workflow
flowchart TD subgraph VM2["VM 2 - Linux Agent + NetFlow Collector"] A["Network Traffic"] --> B["pmacctd\n(Traffic Metadata Capture)"] B -->|"Raw JSON + timestamps"| C["Raw Flow Log\n/var/log/netflow/netflow_raw.json"] C --> D["Python Normalization Script\n(filter + flatten + normalize)"] D -->|"Flat normalized JSON"| E["Normalized Log\n/var/log/netflow/netflow_wazuh.json"] E --> F["Wazuh Agent\n(localfile monitor + forward)"] end
subgraph VM1["VM 1 - Wazuh All-in-One Server"]
G["Wazuh Manager\n(Log Ingestion)"]
G --> H["Built-in JSON Decoder\n(flat field extraction)"]
H --> I["Custom Rules\n(117001 – 117024)"]
I --> J["Wazuh Indexer\n(Alert Storage)"]
J --> K["Wazuh Dashboard\n(Alert Visualization)"]
end
F -- "Agent Connection\n(1514/TCP)" --> G
Implementation
- pmacctd 1.7.6 configured with timestamp_start/timestamp_end aggregate fields
- Python normalization script with multicast/broadcast/internal subnet filtering
- Flat JSON output (nf_src_ip, nf_dst_port, etc.) required due to Wazuh 4.x limitation on dot notation in rule field tags
- 24 custom rules using pcre2 syntax, frequency-based detection, and field matching
- OpenSearch scripted fields for numeric aggregation on bytes/packets
- Custom dashboard with 15+ visualizations including heatmap and traffic correlation timeline
Tech Stack Used
Log ingestion, custom decoder, detection rules, and alert dashboard
Network traffic metadata capture with timestamp aggregation
Flow normalization, filtering, and flat JSON output script
15+ custom visualizations including heatmap and correlation timeline
Base OS for both VMs on Eranya Cloud
pmacctd, cron, systemd service management
Key Features & Deliverables
normalize_netflow_to_wazuh.py
Python script with multicast/broadcast/internal subnet filtering, timestamp parsing, and flat integer JSON output
netflow_rules.xml
24 custom Wazuh detection rules (117001-117024) using pcre2 and frequency-based matching
netflow_decoder.xml
Custom Wazuh JSON decoder for NetFlow field extraction
Wazuh Dashboard
15+ visualizations: correlation timeline, threat heatmap, attack table, top IPs, protocol breakdown
Real Traffic Validation
4,472 alerts in ~5 hours against live internet traffic - RDP, Telnet, DB scan, NetBIOS, C2 beaconing confirmed
Full Documentation
Architecture, installation, configuration, detection logic, and troubleshooting guides
Key Metrics
Results & Outcome
Fully working detection pipeline from traffic capture to dashboard alert. In ~5 hours of real internet exposure:
- 4,472 total NetFlow alerts generated
- 982 high severity alerts (Level 9+)
- 326 unique external source IPs detected
- 10 threat categories identified
Confirmed detections: - RDP scanner (87.251.64.25) - 4 hits in <1 second, Level 12 - Telnet crawlers from 8 different external IPs, Level 10 - MySQL port 3306 scanner (45.156.87.127), Level 10 - PostgreSQL port 5432 scanner (64.89.163.133), Level 10 - NetBIOS broadcasts (103.153.61.85) - 2,470 hits, Level 9 - C2 beaconing patterns from multiple external IPs, Level 11
Screenshots & Demo
NetFlow Dashboard Overview
A centralized Wazuh dashboard for monitoring NetFlow telemetry, showing traffic correlation, firing events, threat categories, alert severity, protocol distribution, source and destination activity, and representative network events for security analysis.
External Threat Detection Events
Filtered Wazuh Discover view showing NetFlow security events with external source traffic, focusing on medium to high severity alerts such as NetBIOS activity, suspicious port scanning, and potential network anomaly indicators.
High Severity NetFlow Alerts
Wazuh Discover view filtered to NetFlow alerts with severity level 9 and above, highlighting high-risk network events such as NetBIOS traffic detection, potential lateral movement, and suspicious external communication patterns.
Top Attacker IP Investigation
Wazuh Discover view filtered by NetFlow source IP 103.153.61.85, showing repeated security events related to NetBIOS traffic detection, network anomaly indicators, and potential lateral movement activity within the monitored traffic.
Wazuh Logtest Database Port Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for external access to database port 3306, generating a level 10 alert categorized as network anomaly and database activity.
Wazuh Logtest NetBIOS Traffic Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for NetBIOS UDP traffic on port 138, generating a level 9 alert categorized as network anomaly and potential lateral movement activity.
Wazuh Logtest RDP Flow Event
Validation of NetFlow JSON parsing using wazuh-logtest, showing successful decoding of TCP traffic to destination port 3389 and rule matching as a general NetFlow event with rule ID 117001 and severity level 3.
Wazuh Logtest Suspicious Port Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for TCP traffic to suspicious destination port 4444, generating a level 7 alert categorized as a network anomaly.
Wazuh Logtest Telnet Connection Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for TCP traffic to Telnet port 23, generating a level 10 alert categorized as a network anomaly and cleartext protocol activity.
Interested in This Project?
Discuss your security needs or explore collaboration opportunities.