NetFlow, sFlow and IPFIX for Security Monitoring: Detection and Sampling Trade-offs

What a flow record is

A switch or router forwards millions of packets a second. Recording every packet is too expensive, and most of the content is not needed to answer common security questions. A flow record keeps the summary. It names a pair of endpoints and the conversation between them: source and destination IP address, source and destination port, protocol, interface, start and end times, and counts of bytes and packets, often with the TCP flags seen. RFC 7011 defines a flow as a set of packets observed at an observation point during a time interval, all sharing common properties, and calls the properties that define the flow Flow Keys, such as the classic five-tuple. RFC 3954 for NetFlow v9 describes a flow as a unidirectional sequence of packets passing an observation point.

Direction matters. A conversation between a client and a server is two unidirectional flows, one in each direction, and tools usually join them into a single bidirectional view. A flow ends when the exporter sees a TCP FIN or RST, when it has been idle for a configurable timeout, when a long-lived flow reaches a periodic export interval, or when internal limits such as memory force an early export, as RFC 3954 lists. That last point explains why a long download appears as several records.

NetFlow v9: templates

Early NetFlow versions had fixed record layouts. Version 9, documented in RFC 3954 as an informational description of the Cisco export format, introduced templates. The exporter sends a Template FlowSet that lists the fields and their lengths, then Data FlowSets that carry records in that layout. Template IDs for data run from 256 to 65535 and are unique per exporter and observation domain. The collector must store each template, and a data FlowSet cannot be decoded until its template has arrived. Because exports normally travel over UDP, which can lose packets, the exporter must resend templates periodically, either every N packets or every N minutes, and after a restart or a configuration change.

NetFlow v9 also defines Options Template FlowSets that describe the exporter itself and not individual flows. A typical use is to report the sampling configuration, scoped to an interface or line card, so the collector knows how to scale the counts. The field registry covers byte and packet counters, protocol, type of service, TCP flags, addresses and ports for IPv4 and IPv6, interface indexes, next hop, BGP AS numbers, timestamps, MPLS labels, MAC addresses and VLAN IDs. Because the format is extensible, vendors can add fields without changing the packet format.

IPFIX: the standard

IPFIX is the IETF's standardized descendant, defined in RFC 7011. It keeps the template idea and adds structure. A message has a header followed by Sets: Template Sets, Options Template Sets or Data Sets. Fields come from the IANA IPFIX Information Element registry, which also lets vendors define enterprise-specific elements, and a variable-length field is possible. An Observation Domain groups the observation points whose flows the same metering process can aggregate, with a 32‑bit ID. RFC 7011 names the sender an Exporter and the receiver a Collector.

Transport is where IPFIX is more careful than its predecessor. SCTP is mandatory to implement, and the RFC says it should be used where links are prone to congestion. TCP is optional. UDP is allowed, but the RFC says it must not be used unless the application tolerates some message loss, that the packet size must not exceed the path MTU, and that it may be used only on dedicated, over-provisioned links. Collectors should infer loss from gaps in the sequence numbers. For UDP, exporters must retransmit each active template periodically and must not send template withdrawals; collectors may discard a template that is not refreshed, with a lifetime that should default to at least three times the refresh interval.

Security is spelled out as well. RFC 7011 requires support for TLS on TCP and DTLS on UDP and SCTP, requires mutual authentication so that an exporter does not send to an unverified collector and a collector does not accept data from an unverified exporter, forbids the null cipher, and says that UDP traffic not protected by DTLS should be segregated onto a dedicated network. This matters because flow data is sensitive. It is a map of who talks to whom in your company, and a forged flow stream can hide an attack or send an analyst the wrong way.

sFlow: sampling instead of caching

sFlow takes a different route. RFC 3176, published as an informational document in 2001 by InMon, describes an agent in the switch or router that does two things. It takes packet samples: a counter decrements for each packet, and when it reaches zero the agent copies the packet and picks a new random skip count, so that over time one packet in N is sampled. The RFC suggests a uniform random generator with variation of about plus or minus 10 percent around the mean. And it polls interface counters on a schedule. Both go out in datagrams, XDR-encoded, over UDP to port 6343 by default.

A flow sample contains the sequence number, the sampling rate, the sample pool, the number of drops and the input and output interfaces, plus the first bytes of the packet itself: by default 128 bytes of header, with a protocol maximum of 256. Optional extended data can add switch, router, BGP and user information. Because the sample carries real packet headers, a collector can decode any protocol it understands, including layer 2 and VXLAN or MPLS encapsulation, without waiting for a vendor to define a field. The agent builds no flow cache, which keeps memory and CPU low and is why sFlow works on high-speed switch silicon. The RFC accepts the cost: UDP is unreliable, and lost samples only slightly reduce the effective sampling rate. The sFlow.org site describes the technology as an industry standard supported by many vendors, though the RFC itself states it does not specify an Internet standard of any kind.

How sampling changes what you see

Many devices also sample when they export NetFlow or IPFIX, and sFlow always samples. The sFlow.org packet sampling basics page explains the statistics clearly. A sampled count scales up by the rate: if you sampled c packets of a class out of n total samples at one in N, the estimate of the class is about N times c. Accuracy depends on the number of samples in that class, not on the total traffic. The page's chart puts the error at about 6 percent when a class has 1,000 samples, and our arithmetic with the usual 95 percent formula, about 196 divided by the square root of the sample count, agrees (that formula is general statistics; the page's equations did not come through in our fetch).

This has direct security consequences. A large flow is easy to see. A UDP flood of hundreds of thousands of packets per second produces many samples in a second, and a sampled view is accurate enough to identify the source prefixes, the destination and the protocol. A slow flow is hard. The page gives an example: a flow of one packet per second sampled at 0.25 percent for an hour has about 65 percent error, because only a handful of samples exist; over a month the error falls to about 2.4 percent. A command-and-control beacon that calls home once a minute may never be sampled at all in a short window. Sampling suits volume questions, such as top talkers, floods, scans that touch many hosts, and bulk transfers. It is weak for low-and-slow behavior.

Unsampled flow export, 1:1, avoids the problem but costs memory on the device and storage in the collector. Many security teams use unsampled or lightly sampled flow at key choke points (internet edge, data-center ingress and egress, the cloud virtual network) and heavier sampling on high-speed internal links. That is a sound compromise, and the sampling rate should be recorded in an options template or in the collector so analysts know the scaling.

Detections that flow data supports

• Scanning and reconnaissance. One source contacting many destinations or many ports in a short time, mostly with SYN-only flows and small byte counts. Internal scans stand out because normal hosts rarely touch more than a few dozen peers.
• Data exfiltration. A host sending an unusual volume to an external address, especially to a first-seen destination or at an odd hour. Compare outbound bytes with the host's own baseline, not a global threshold.
• Command-and-control beaconing. Regular, repeated small connections to one external address with similar byte counts and intervals. Needs unsampled or lightly sampled data to be reliable.
• Floods and amplification. Many sources to one destination, or reflected traffic with the telltale source ports of services such as DNS, NTP and memcached. Flow data identifies the traffic that a FlowSpec rule or blackhole should match.
• Lateral movement. Workstations connecting to each other over SMB, RDP or WinRM when they normally only talk to servers. This is where flow data pays off for microsegmentation.
• Policy violations. Direct connections from a user network to a restricted zone, a database reachable from the internet, or legacy protocols that were supposed to be retired.
• Shadow infrastructure. New hosts and services that appear in the flow map without an inventory record.

Flow data cannot see inside encrypted traffic, cannot prove what was transferred, and says nothing about a single packet. It pairs well with DNS logs, firewall and proxy logs, endpoint telemetry and, for high-value segments, packet capture. A flow alert is usually a lead, not a verdict.

Design choices that matter

Place exporters where the questions are. The internet edge answers what is entering and leaving. The data-center core sees east-west traffic. Cloud providers offer their own flow logs for virtual networks, and these feed the same analysis. Choose one collection point or a small set of collectors, keep clocks synchronized with NTP so timestamps line up across devices, and avoid double counting when a flow crosses several exporters by choosing which interfaces export, ingress only being a common rule.

Secure the pipeline. Send flow data over a management network or, where supported, over DTLS or TLS with mutual authentication, as RFC 7011 requires for IPFIX. Restrict who can reach the collector port. Treat the stored records as sensitive data with a retention period set by your policy and any regulatory requirement, and keep enough history to look back. Many investigations start weeks after the first event, and a few weeks of raw flow plus longer aggregates is a common compromise. And plan collector capacity: record rates grow with link speed and with the number of short flows, and a flood can multiply them just when you need the data.

Enrich the data. Flow records have addresses, not names. Add the asset inventory, the user behind the address (from DHCP or the identity system), geolocation and threat-intelligence lists, and the BGP AS number the exporter already knows. With that context an alert reads a finance server sent 40 GB to a first-seen address in a foreign network, which is something an analyst can act on.

Where to start

A pragmatic start has three steps. First, turn on IPFIX or NetFlow v9 or sFlow at the internet edge and the data-center core, using whichever format the devices support well. Second, deploy a collector with at least a basic set of detections: top talkers, new external destinations per host, scan thresholds and flood alerts. Third, tune against a baseline of a few weeks, and feed what you learn into the segmentation and firewall policy. Our team designs and operates monitoring of this kind as part of managed security services, and connects it to incident response so that an alert leads to a defined action. Related reading: MPLS, SD-WAN or EVPN: Choosing a Backbone for a Multi-Site Company; Network Time for Security and Trading: NTP, NTS and PTP (accurate clocks make flow records correlate).

Sources

• RFC 3954 - Cisco Systems NetFlow Services Export Version 9
• RFC 3176 - InMon Corporation's sFlow: A Method for Monitoring Traffic in Switched and Routed Networks
• RFC 7011 - Specification of the IP Flow Information Export (IPFIX) Protocol
• sFlow.org - About sFlow
• sFlow.org - Packet Sampling Basics

This article is general technical information, not a monitoring design for your network. We read the RFCs and sFlow.org pages through a summarizing fetch tool (the RFC 7011 text was longer than one fetch and was read in two parts), so confirm section details against the RFC text. The 95 percent error formula, the detection use cases, the placement, retention and enrichment advice are general engineering and security-operations knowledge, not quoted from the sources listed. Device support for sampling, templates and export differs by vendor and model. Last reviewed November 8, 2026.

Keep reading

Next · InfrastructureMPLS, SD-WAN or EVPN: Choosing a Backbone for a Multi-Site CompanyA company with several offices, data centers and clouds has to decide how those sites talk to each other. Three families of answer dominate: a provider's MPLS VPN, an SD-WAN overlay running over ordinary internet and carrier links, and an EVPN-based fabric that the company runs itself. They are not three brands of the same thing. They sit at different layers and solve different problems. This article explains what each one is, what the standards say, where each fits and how a mid-size firm can choose without buying more than it needs.10 min read

• Penetration Testing Explained for Executives: Scope, Reports and How Often to Test8 min
• Ransomware Readiness and Incident Response: A Checklist Based on the CISA #StopRansomware Guide9 min