Network Flow Monitoring and Anomaly Detection with Wazuh

Network visibility is an important layer in security monitoring. Endpoint logs can show what happened on a host, but network flow metadata helps answer a different question: who is communicating with whom, how often, through which service, and how much data is being transferred.

This project demonstrates how to build a lab-based network traffic monitoring pipeline using pmacctd, Python log normalization, and Wazuh. The goal is to ingest NetFlow like metadata into Wazuh, enrich the events into structured JSON, and create custom detection rules for network anomalies.

Unlike full packet capture, this project does not inspect payload content. Instead, it focuses on flow metadata such as source IP, destination IP, ports, protocol, byte count, packet count, duration, direction, and anomaly tags.

Project Objective

The objective of this project is to build a practical network monitoring lab that can detect suspicious flow based behavior using Wazuh.

This project focuses on:

• Capturing network flow metadata from a Linux collector host.
• Normalizing raw pmacctd JSON output into Wazuh-compatible JSON.
• Enriching events with flow direction, service name, and anomaly tags.
• Ingesting normalized flow logs into Wazuh using the agent localfile configuration.
• Creating custom Wazuh decoders and rules for NetFlow-based detection.
• Mapping selected detections to MITRE ATT&CK techniques.
• Building dashboard filters and threat hunting queries for investigation.
• Producing sample reports for network anomaly and incident investigation.

The result is a small but practical SIEM pipeline that adds network behavior visibility into Wazuh.

Why Network Flow Monitoring Matters

Traditional SIEM use cases often depend on endpoint telemetry such as Windows Event Logs, Linux syslog, Sysmon, auditd, application logs, and authentication logs. These logs are useful, but they do not always provide a complete picture of network behavior.

NetFlow-style metadata adds another layer of visibility.

Network flow data is useful because it captures communication patterns, not payloads. This makes it lightweight, scalable, and suitable for continuous monitoring.

What Is NetFlow?

NetFlow is a method for collecting network communication metadata. A network flow represents a conversation between two endpoints over a period of time.

A typical flow contains:

Example flow:

src=192.168.56.10  dst=203.0.113.50  sport=52341  dport=443  proto=TCP
bytes=52428800  packets=420  duration=45s

This flow does not show the content of the communication. However, it tells us that a host transferred around 50 MB to an external destination over HTTPS.

That kind of metadata is useful for detecting suspicious behavior such as possible data exfiltration, beaconing, port scanning, and lateral movement.

NetFlow vs Packet Capture

NetFlow and packet capture are different monitoring approaches.

NetFlow is not a replacement for packet capture. Instead, it is a practical always-on monitoring layer that helps analysts understand network behavior at scale.

Lab Architecture

This project uses a two-VM lab architecture.

The general pipeline is:

Network Interface
      ↓
pmacctd direct capture
      ↓
netflow-raw.json
      ↓
Python normalizer
      ↓
netflow-wazuh.json
      ↓
Wazuh Agent localfile
      ↓
Wazuh Manager
      ↓
Custom decoder and rules
      ↓
Wazuh Indexer
      ↓
Wazuh Dashboard
      ↓
Threat hunting and investigation

The collector host captures network metadata from the network interface. The raw output is written to /var/log/netflow/netflow-raw.json. A Python script then converts the raw data into nested JSON fields that Wazuh can parse properly.

Main Components

Repository Structure

The project is structured into several folders:

netflow-network-traffic-monitoring/
├── README.md
├── .env.example
├── collectors/
│   └── pmacct/
│       ├── pmacct-collector.sh
│       ├── nfacctd-sample.conf
│       └── pmacct-json-output-notes.md
├── scripts/
│   ├── install_netflow_tools.sh
│   ├── setup_cron.sh
│   ├── normalize_netflow_to_wazuh.py
│   ├── detect_flow_anomalies.py
│   ├── generate_safe_netflow_test_events.py
│   ├── rotate_netflow_logs.sh
│   └── collect_netflow_evidence.sh
├── wazuh/
│   ├── ossec-localfile-netflow-snippet.xml
│   ├── agent-group-netflow-snippet.xml
│   ├── decoders/
│   │   └── netflow_decoders.xml
│   └── rules/
│       └── netflow_rules.xml
├── samples/
│   ├── sample-pmacct-json-flow.json
│   ├── sample-normalized-netflow-event.json
│   └── sample-wazuh-alert-*.json
├── dashboards/
│   ├── dashboard-fields-and-filters.md
│   ├── saved-searches.md
│   └── visualization-guide.md
├── docs/
│   ├── 01-overview.md
│   ├── 03-netflow-concept.md
│   ├── 04-netflow-vs-packet-capture.md
│   ├── 12-detection-use-cases.md
│   ├── 13-dashboard-and-threat-hunting-queries.md
│   ├── 15-incident-investigation-playbook.md
│   └── 17-troubleshooting.md
└── reports/
    ├── sample-netflow-monitoring-report.md
    ├── sample-network-anomaly-detection-report.md
    └── sample-incident-investigation-report.md

Step 1 — Install Wazuh Agent on Collector Host

The collector host sends normalized NetFlow events to the Wazuh server through the Wazuh Agent.

Example installation command:

WAZUH_MANAGER_IP="<WAZUH_SERVER_IP>"

curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | \
  gpg --no-default-keyring --keyring gnupg-ring:/usr/share/keyrings/wazuh.gpg \
  --import && chmod 644 /usr/share/keyrings/wazuh.gpg

echo "deb [signed-by=/usr/share/keyrings/wazuh.gpg] \
  https://packages.wazuh.com/4.x/apt/ stable main" | \
  sudo tee /etc/apt/sources.list.d/wazuh.list

sudo apt update
sudo WAZUH_MANAGER="$WAZUH_MANAGER_IP" apt install wazuh-agent -y
sudo systemctl enable wazuh-agent
sudo systemctl start wazuh-agent

Verify the agent:

sudo systemctl status wazuh-agent

Step 2 — Install NetFlow Tools

The project includes a helper script to install required tools.

sudo bash scripts/install_netflow_tools.sh

The collector uses pmacctd to capture network flow metadata from the selected interface.

Step 3 — Start pmacctd Collector

Before starting the collector, identify the network interface.

ip a

Example interface:

enp1s0

Start the collector:

bash collectors/pmacct/pmacct-collector.sh

Or run it manually:

sudo pmacctd -i enp1s0 \
  -c src_host,dst_host,src_port,dst_port,proto \
  -P print -O json \
  -o /var/log/netflow/netflow-raw.json \
  -r 60 -D

This command captures flow metadata and writes JSON output every 60 seconds.

Verify the raw output:

cat /var/log/netflow/netflow-raw.json | head -3

Expected result:

JSON lines containing source IP, destination IP, protocol, packets, and bytes.

Step 4 — Configure Internal Networks

The normalizer uses INTERNAL_NETWORKS to classify the flow direction.

Example .env configuration:

INTERNAL_NETWORKS=192.168.56.0/24,10.10.10.0/24

This helps classify traffic as:

This configuration is important because several detections depend on traffic direction. For example, lateral movement detection requires internal_to_internal traffic.

Step 5 — Configure Wazuh Agent Localfile

The Wazuh Agent reads the normalized JSON file using log_format: json.

Add this configuration to the Wazuh Agent configuration file:

<localfile>
  <log_format>json</log_format>
  <location>/var/log/netflow/netflow-wazuh.json</location>
</localfile>

Then restart the agent:

sudo systemctl restart wazuh-agent

This allows Wazuh to ingest each normalized NetFlow event as a JSON log.

Step 6 — Normalize Raw Flow Logs

The raw pmacctd output is converted into Wazuh-compatible JSON using the Python normalizer.

python3 scripts/normalize_netflow_to_wazuh.py \
  --pmacct /var/log/netflow/netflow-raw.json \
  --output /var/log/netflow/netflow-wazuh.json

The normalized output uses nested JSON fields such as:

{
  "@timestamp": "2026-04-26T11:45:20Z",
  "netflow": "true",
  "collector": {
    "name": "lab-collector-01"
  },
  "source": {
    "ip": "192.168.56.30",
    "port": 48739
  },
  "destination": {
    "ip": "203.0.113.99",
    "port": 443
  },
  "flow": {
    "protocol": "TCP",
    "direction": "internal_to_external"
  },
  "network": {
    "bytes": 52428800,
    "packets": 420
  },
  "event": {
    "duration": 45.0,
    "type": "network",
    "category": "network_traffic"
  },
  "service": {
    "name": "HTTPS"
  },
  "anomaly": {
    "tags": [
      "high_outbound_bytes"
    ]
  }
}

This structure makes the data easier to query in Wazuh Dashboard using fields such as:

data.source.ip
data.destination.ip
data.destination.port
data.flow.direction
data.network.bytes
data.network.packets
data.anomaly.tags

Step 7 — Automate Normalization with Cron

The project includes a script to automate the normalizer every minute.

bash scripts/setup_cron.sh

Example cron logic:

* * * * * rm -f /var/log/netflow/netflow-wazuh.json && python3 /path/to/scripts/normalize_netflow_to_wazuh.py --pmacct /var/log/netflow/netflow-raw.json --output /var/log/netflow/netflow-wazuh.json

This keeps the normalized Wazuh log updated from the latest pmacctd output.

Step 8 — Deploy Wazuh Custom Decoder

The custom decoder identifies NetFlow JSON events and allows Wazuh to extract fields from the normalized JSON.

Deploy the decoder on the Wazuh server:

sudo cp wazuh/decoders/netflow_decoders.xml /var/ossec/etc/decoders/
sudo chown wazuh:wazuh /var/ossec/etc/decoders/netflow_decoders.xml

The decoder uses Wazuh JSON parsing to extract fields from each event.

Simplified decoder logic:

<decoder name="netflow-json">
  <prematch>{"@timestamp"</prematch>
</decoder>

<decoder name="netflow-json-fields">
  <parent>netflow-json</parent>
  <use_own_name>true</use_own_name>
  <plugin_decoder>JSON_Decoder</plugin_decoder>
</decoder>

Step 9 — Deploy Wazuh Custom Rules

The project uses custom Wazuh rules in the ID range 117000–117009.

Deploy the rules:

sudo cp wazuh/rules/netflow_rules.xml /var/ossec/etc/rules/
sudo chown wazuh:wazuh /var/ossec/etc/rules/netflow_rules.xml
sudo systemctl restart wazuh-manager

The base rule identifies NetFlow events:

<rule id="117000" level="0">
  <decoded_as>json</decoded_as>
  <field name="netflow">^true$</field>
  <description>NetFlow: Network flow event received from collector</description>
  <group>netflow,network_monitoring,</group>
</rule>

Specific anomaly rules then match against fields such as anomaly.tags, flow.direction, and destination.port.

Detection Coverage

The project includes several custom detection use cases.

These detections are not meant to confirm compromise by themselves. They are designed to generate investigation leads based on suspicious network behavior.

Detection Example 1 — Port Scanning

Port scanning can be detected when one source IP attempts to connect to many destination ports in a short period.

Example behavior:

192.168.56.10 → 192.168.56.20:22
192.168.56.10 → 192.168.56.20:23
192.168.56.10 → 192.168.56.20:80
192.168.56.10 → 192.168.56.20:443
192.168.56.10 → 192.168.56.20:3389

The project marks this pattern with:

possible_port_scan

Wazuh rule:

117001 — NetFlow: Possible port scanning activity detected

MITRE ATT&CK mapping:

T1046 — Network Service Discovery

Possible false positives:

• Authorized vulnerability scanners.
• Network discovery tools.
• Asset inventory scans.
• Monitoring or health check systems.

Detection Example 2 — High Outbound Traffic

High outbound traffic can indicate possible data exfiltration, especially when the destination is external and the volume is unusual for the host.

Example flow:

src=192.168.56.10
dst=203.0.113.99
dport=443
bytes=52428800
direction=internal_to_external

The project marks this pattern with:

high_outbound_bytes

Wazuh rule:

117002 — NetFlow: High outbound traffic volume detected

MITRE ATT&CK mapping:

T1041 — Exfiltration Over C2 Channel
T1567 — Exfiltration Over Web Service

Possible false positives:

• Cloud backup.
• File synchronization.
• Software update.
• Video conferencing.
• Authorized data transfer.

Detection Example 3 — Beaconing

Beaconing is a repeated communication pattern from one source to the same destination at a regular interval. This can be associated with command-and-control check-in behavior.

Example pattern:

T+0    192.168.56.30 → 203.0.113.99:443
T+300  192.168.56.30 → 203.0.113.99:443
T+600  192.168.56.30 → 203.0.113.99:443
T+900  192.168.56.30 → 203.0.113.99:443

The project marks this pattern with:

possible_beaconing

Wazuh rule:

117003 — NetFlow: Possible beaconing pattern detected

MITRE ATT&CK mapping:

T1071 — Application Layer Protocol

Possible false positives:

• Agent heartbeat.
• Monitoring tools.
• Cloud service polling.
• Scheduled API checks.

Detection Example 4 — Lateral Movement

Lateral movement can be detected when an internal host communicates with other internal systems using administrative or remote access ports.

Example ports:

Detection condition:

flow.direction = internal_to_internal
destination.port in 22, 445, 3389, 5985, 5986, 1433, 3306

Wazuh rule:

117004 — NetFlow: Possible lateral movement detected

MITRE ATT&CK mapping:

T1021 — Remote Services
T1021.002 — SMB/Windows Admin Shares

Possible false positives:

• Administrator activity.
• Patch management.
• Backup systems.
• Internal automation.
• Database administration.

Detection Example 5 — Suspicious DNS Flow

Suspicious DNS flow volume can indicate malware polling, DNS tunneling, or abnormal name resolution behavior.

Example pattern:

192.168.56.50 → 8.8.8.8:53 UDP
192.168.56.50 → 8.8.8.8:53 UDP
192.168.56.50 → 8.8.8.8:53 UDP
... repeated many times

The project marks this pattern with:

suspicious_dns_flow

Wazuh rule:

117005 — NetFlow: Suspicious DNS flow volume detected

MITRE ATT&CK mapping:

T1071.004 — DNS

Possible false positives:

• DNS-heavy applications.
• Misconfigured resolver.
• Browser activity.
• Internal DNS forwarding.
• Security tools performing DNS lookups.

Testing with Synthetic Events

The project includes a safe event generator for testing detection rules without using real attack traffic.

Generate all scenarios:

python3 scripts/generate_safe_netflow_test_events.py \
  --scenario all \
  --output /var/log/netflow/netflow-wazuh.json

Generate only port scan scenario:

python3 scripts/generate_safe_netflow_test_events.py \
  --scenario port_scan \
  --count 25 \
  --output /var/log/netflow/netflow-wazuh.json

Generate only beaconing scenario:

python3 scripts/generate_safe_netflow_test_events.py \
  --scenario beaconing \
  --output /var/log/netflow/netflow-wazuh.json

Important note:

If the cron normalizer is active, pause it temporarily before generating synthetic events.
The cron job may overwrite netflow-wazuh.json before the Wazuh Agent reads the test events.

Validating Rules with wazuh-logtest

Before relying on custom rules, the detection logic should be tested using wazuh-logtest.

Run on the Wazuh server:

sudo /var/ossec/bin/wazuh-logtest

Paste a single normalized JSON event:

{"@timestamp":"2026-04-26T11:45:20Z","netflow":"true","source":{"ip":"192.168.56.30","port":48739},"destination":{"ip":"192.168.56.28","port":47775},"flow":{"protocol":"TCP","direction":"internal_to_internal"},"network":{"bytes":60,"packets":1},"event":{"duration":0.001,"type":"network","category":"network_traffic"},"tcp":{"flags":"SYN"},"service":{"name":"OTHER"},"anomaly":{"tags":["possible_port_scan"]}}

Expected result:

**Phase 2: Completed decoding.
        name: 'json'
        netflow: 'true'
        anomaly: '{"tags": ["possible_port_scan"]}'

**Phase 3: Completed filtering (rules).
        id: '117001'
        level: '9'
        description: 'NetFlow: Possible port scanning activity detected — multiple destination ports from one source'
**Alert to be generated.

If the rule appears in Phase 3, the custom rule is working.

Viewing Alerts in Wazuh Dashboard

To view all NetFlow-related events, use this filter:

rule.groups: netflow

Useful filters:

Common dashboard fields:

data.source.ip
data.source.port
data.destination.ip
data.destination.port
data.flow.protocol
data.flow.direction
data.network.bytes
data.network.packets
data.event.duration
data.service.name
data.anomaly.tags
rule.id
rule.level
rule.description
agent.name

Dashboard Visualization Ideas

This project also includes dashboard guidance for building NetFlow visualizations in Wazuh Dashboard.

Useful visualizations include:

Example visualization query:

rule.groups:netflow AND rule.id:[117001 TO 117009]

Threat Hunting Queries

Here are some useful hunting queries based on the normalized fields.

Find All NetFlow Events

rule.groups:netflow

Find High-Severity Network Anomalies

rule.groups:network_anomaly AND rule.level >= 10

Find Internal Lateral Movement Candidates

data.flow.direction:internal_to_internal AND data.destination.port:(22 OR 445 OR 3389 OR 5985 OR 5986)

Find Possible Data Exfiltration

rule.id:117002

Find Suspicious DNS Activity

rule.id:117005 OR data.destination.port:53

Find Beaconing Indicators

rule.id:117003

Find External Inbound Access to Sensitive Services

rule.id:117006

Find Unusual Outbound Ports

rule.id:117007

Incident Investigation Workflow

When a NetFlow alert triggers, the investigation can follow this workflow.

1. Identify the Alert

Review the Wazuh alert fields:

rule.id
rule.description
rule.level
data.source.ip
data.destination.ip
data.destination.port
data.network.bytes
data.flow.direction
data.anomaly.tags

2. Determine the Communication Direction

Check whether the traffic is:

internal_to_external
internal_to_internal
external_to_internal
external_to_external

This helps determine whether the case is more likely to involve exfiltration, lateral movement, external probing, or general internet traffic.

3. Review Source Host Behavior

Investigate the source IP:

data.source.ip:<SOURCE_IP>

Look for:

• Other alerts from the same host.
• Repeated destinations.
• High byte volume.
• Multiple destination ports.
• Communication with sensitive internal services.

4. Review Destination Behavior

Investigate the destination IP:

data.destination.ip:<DESTINATION_IP>

Look for:

• Multiple internal sources contacting the same destination.
• External destinations with high outbound traffic.
• Repeated beaconing-like connections.
• Suspicious service ports.

5. Correlate with Endpoint Logs

NetFlow alone shows communication metadata. For deeper investigation, correlate with endpoint logs such as:

• Process creation logs.
• Authentication events.
• PowerShell activity.
• Sysmon events.
• Linux audit logs.
• EDR telemetry.

6. Decide the Response

Depending on severity, possible response actions include:

• Validate whether the activity is authorized.
• Check whether the source host is a scanner, backup server, or admin workstation.
• Block suspicious external destinations.
• Isolate the endpoint if compromise is suspected.
• Collect forensic artifacts.
• Escalate to incident response if needed.

Example Investigation: Possible Beaconing

An alert is generated:

Rule ID: 117003
Description: Possible beaconing pattern detected
Source: 192.168.56.30
Destination: 203.0.113.99:443
Pattern: Repeated connections every 5 minutes

Analyst assessment:

The source host repeatedly connects to the same external destination using HTTPS.
The timing interval is consistent and the byte count is similar across events.
This behavior may indicate command-and-control beaconing, but it may also be caused by legitimate agent heartbeat or polling behavior.

Recommended follow-up:

1. Check endpoint process tree on the source host.
2. Review DNS and proxy logs for the destination.
3. Validate whether the destination is known and authorized.
4. Check whether other hosts are contacting the same destination.
5. Escalate if no legitimate business justification is found.

Common Troubleshooting

Wazuh Manager Fails to Start

Possible causes:

• Invalid XML syntax in custom rules or decoders.
• Unsupported decoder configuration.
• Incorrect rule field names.
• Duplicate rule IDs.

Check the manager log:

sudo tail -f /var/ossec/logs/ossec.log

Validate XML before restarting the manager.

NetFlow Events Do Not Appear in Dashboard

Check whether the Wazuh Agent is reading the file:

sudo systemctl status wazuh-agent
sudo tail -f /var/ossec/logs/ossec.log

Check whether the normalized file exists:

ls -lah /var/log/netflow/netflow-wazuh.json
head -1 /var/log/netflow/netflow-wazuh.json

Check the localfile configuration:

sudo grep -A4 -B2 "netflow-wazuh.json" /var/ossec/etc/ossec.conf

Rules Do Not Trigger

Use wazuh-logtest and validate:

• The log is a single JSON line.
• The field netflow is set to true.
• The anomaly tag exists.
• The rule field name matches the decoded field name.
• The Wazuh Manager was restarted after deploying rules.

Example field:

"anomaly": {
  "tags": ["possible_port_scan"]
}

Example rule condition:

<field name="anomaly.tags" type="pcre2">possible_port_scan</field>

Flow Direction Is Incorrect

If all traffic appears as:

external_to_external

Then INTERNAL_NETWORKS may not match the lab subnet.

Check the collector IP:

ip a | grep "inet " | grep -v "127.0.0"

Then update .env:

INTERNAL_NETWORKS=192.168.56.0/24

Re-run the normalizer afterward.

Numeric Fields Cannot Be Aggregated

If data.network.bytes or data.network.packets cannot be used for sum or average aggregation, the index mapping may treat the field as keyword instead of number.

Apply an index template for numeric fields:

curl -k -u admin:<password> \
  -X PUT "https://localhost:9200/_index_template/wazuh-netflow-numeric" \
  -H "Content-Type: application/json" \
  -d '{
    "index_patterns": ["wazuh-alerts-*"],
    "priority": 200,
    "template": {
      "mappings": {
        "properties": {
          "data.network.bytes":   {"type": "long"},
          "data.network.packets": {"type": "long"}
        }
      }
    }
  }'

This mapping applies to new indexes created after the template is added.

Limitations

This project is designed for portfolio project, so there are several limitations:

• The test events use synthetic data.
• Flow metadata does not include packet payload.
• The detection thresholds need tuning before production use.
• NetFlow alone cannot confirm malware execution.
• MITRE ATT&CK mapping is contextual, not definitive proof of compromise.
• GeoIP and threat intelligence enrichment are not included in this version.
• Additional correlation with endpoint logs is required for stronger investigation.

Future Improvements

This project can be improved further by adding:

• GeoIP enrichment for external destination IPs.
• Threat intelligence enrichment for suspicious destinations.
• DNS log correlation.
• Proxy log correlation.
• Endpoint telemetry correlation with Sysmon or auditd.
• Alert suppression for authorized vulnerability scanners.
• Baseline-based anomaly detection.
• SOAR workflow for high-confidence detections.
• Dashboard export and reusable visualization templates.
• Integration with case management or incident ticketing.

Key Takeaways

This project helped me understand how network flow metadata can add valuable visibility to a SIEM environment.

The most important lessons from this project are:

• NetFlow is useful for identifying communication patterns without collecting packet payloads.
• Flow metadata can reveal suspicious behavior such as scanning, beaconing, exfiltration, and lateral movement.
• Wazuh can ingest custom JSON logs using the agent localfile configuration.
• Python can be used to normalize and enrich raw telemetry before SIEM ingestion.
• Custom Wazuh decoders and rules are essential for turning raw data into actionable alerts.
• Threat hunting becomes easier when fields are normalized consistently.
• Network-based visibility is stronger when correlated with endpoint telemetry.

Conclusion

This project demonstrates how Wazuh can be extended beyond endpoint monitoring by ingesting and analyzing network flow metadata. By combining pmacctd, Python-based normalization, Wazuh Agent log collection, custom decoders, and custom detection rules, the lab provides a practical example of flow-based detection engineering.

The result is a lightweight network monitoring pipeline that can detect suspicious behavior such as port scanning, high outbound traffic, beaconing, lateral movement, suspicious DNS volume, exposed services, unusual outbound ports, and DoS-like patterns.

For me, this project represents a strong bridge between networking, log engineering, SIEM implementation, and detection engineering. It also provides a foundation for more advanced blue team use cases such as threat hunting, incident investigation, network anomaly detection, and telemetry enrichment.

Disclaimer

This project is for lab, learning, and portfolio purposes only. All IP addresses, hostnames, and sample events use dummy or documentation values. Do not capture, monitor, or analyze traffic from networks that you do not own or do not have explicit authorization to monitor.

References

• Wazuh Documentation
• pmacct Project
• MITRE ATT&CK T1046 — Network Service Discovery
• MITRE ATT&CK T1071 — Application Layer Protocol
• MITRE ATT&CK T1021 — Remote Services
• MITRE ATT&CK T1071.004 — DNS
• MITRE ATT&CK T1498 — Network Denial of Service