NETFLOW INFO to TRACK NETWORK EVENTS

Netflow information to track network events

Lead: How to use netflow information to track network events

What you will learn...

●     How netflows are generated

●     Sampling rates can be used to reduce cpu loads

●     fingerprints can be shared amongst users

●     anomaly reports can help detect malicious traffic

●     types of data objects that can be tracked

●     report generation

●     netflow version characteristics

What you should know...

●     Basic knowledge of routing protocols

●     Knowledge of the TCP/IP stack

●     Experience in packet capture interpretation would be useful

●     Network security mindset.

About the author

The author has worked for over 30 years for a large Internet service provider in various

capacity, retiring as a network security manager involved in creating and administrating a large netflow deployment and is now a freelance contractor as a network support engineer.

 

Imagine if you will, a peaceful night’s sleep interrupted by a pager beeping, telephone calls

and confused operators trying to describe what is happening in the network and as you slowly wake up and try to log on to your laptop, you realize that all your

preparations and hard work  deployed in the previous months were worth it as you
are able to have a high level view of the situation and even know which type of traffic
and origin of the malicious traffic.
Within minutes, you can have an access list applied to your devices or even a BGP

black holing  statement issued to take care of the problem.The operators are amazed at the speed and ease with which you were able to resolve the issue and you even have nice graphs and reports produced about the event that the higher management will need to see , the next day , as they will Question you about what happened and how to mitigate such events.

This is all happening because you implemented netflow collectors and a controller in
your network and created a baseline about the normal traffic patterns.
Your next step is to train the operators to do the same tasks so you won’t have your
sleep  interrupted , in case it happens again.

 Continuous monitoring may not be necessary if you program alerts to be sent via

email or pager calls sent, once filtering on high alert settings with a duration
threshold reached, for instance or by initiating a SNMP trap to alert the network
administrators.
Is this sounding too good to be true ?  Not at all, these actions occur on a regular basis and new
techniques and procedures are produced as time goes by and as the experience level grows or
new features are introduced by the netflow device manufacturers.
The following article can be used as a starting point and is not intended to be a full
Recipe to manage network events, network device and netflow collector

manufacturers have further descriptions and data sheets about their offerings, if you so choose to go ahead in implementing netflow collection in your network, by using commercial solutions as this will permit a faster deployment and the learning curve will be smoother.You need to realize, though the solution has numerous merits, it cannot be used as a cure-all to the situations you may experience, but it is a very useful adjunct to your toolkit.We will therefore describe suggested arrangements and try to describe various elements  that are either part of the issues or the solution, this will give you a good starting point and hopefully guide you towards establishing the ideal situation and reduce the number of catastrophic events that may occur on a more or less regular basis.

 Network Management

 in order to be able to protect your network, there are a few techniques to make the task a bit more easier and help us focus our attention and resources in the right direction.

Although we should not rely on any one tool or technique to do the work, netflows along with active and passive DNS, data captures, information exchanges within a network security community, ingress and egress filtering, DDOS mitigation techniques such as access list filtering or BGP black holing or even diverting the traffic to packet scrubber devices should be sufficient to handle most of the events encountered during regular operations.

What is Netflow ?

A netflow packet is usually generated from a router or a switch processing traffic and generating netflow packets containing several values about various parameters that are part of the data packets transiting the interfaces under scrutiny Various manufacturers will have Jflow, Cflow or Sflow equivalents to netflow and their documentation can provide specific information on how to configure and use the flows generated. We will use the term netflow in this article as a generic designation for all types of variants mentioned above.
Keep in mind that some of the capabilities and details would not be supported in some flavors of netflow equivalents or not in the same format.This protocol can be used to perform network and security monitoring as well as network capacity planning, ip accounting and traffic analysis.Depending on the platform used and the ip version (ipv4 or ipv6), the netflow version will vary and may contain more or less of the interesting data contained in the netflow packets.Typically, these packets are forwarded to a collector using the UDP protocol with a particular port designated along with a sampling rate and direction of capture,  software programs are used to generate reports and track conversations between the targeted ip addresses and its remote connections. Some platforms support SCTP reliable exporting of these flows. These flows can enable you to track misuse on your network or track specific patterns or protocols.Another use is to detect traffic that exceeds the usual volume of traffic  destined to an interface or ip address, some platforms would contain applications with built-in signatures that will detect and report malicious traffic.
yet another way to track specific flows is to use the fingerprinting capability by specifying a trigger rate or a signature (TCPdump syntax) These fingerprints are created from events that are specific to the conditions seen or searched from the system and can be shared with other users.

 

 Netflow versions

The most prevalent versions are v5 or v9 , although we can find versions such as v7 or v8 on some platforms.Versions 5 & 9 will typically carry the following information fields:
ñ  Source address
ñ  Destination address
ñ  Source port
ñ  Destination port
ñ  Protocol type
ñ  Input logical interface
ñ  Output logical interface
ñ  TOS field
ñ  AS name
ñ  TCP flag
ñ  MPLS label (v9)
Version 5 is the type mostly encountered and has a fixed export format whereas version 9 has a flexible export format and is the basis for the IETF Ipfix protocol. This version is mostly used in newer generations of network devices.

http://datatracker.ietf.org/wg/ipfix/charter/

 

Even if you do not export these flows to a collector, they can still be useful in tracking activity within the router or switch internally and can show you the top ten talkers, for instance or anomalies affecting your devices, but if you use netflows in this manner , just ensure that you turn off the netflow generation process within the device when you are done as this will impact the resources of the device, Built-in show commands are usually part of the operating system being used within the device.

It must be understood that netflows being generated should be one of the several tools at your disposal to monitor, detect and capture malicious traffic transiting your network, then data captures can validate and provide useful information about this traffic, for traceback or forensic uses, for instance

Sampling rates

Sampling rates can be specified to be 1 to 1 or 1 to ten thousand and everything in between the above values, meaning either one netflow packet generated for every data packet crossing the inspected interface up to one netflow packet to ten thousand data packets, impacting the cpu cycles accordingly ie: 1:1000 would impact the router cycles much less than a 1:10 configuration

Always monitor the cpu loading before and after enabling netflow generation.

Some applications will also do sampled data captures as high capacity links can easily overwhelm monitoring platforms. The same netflow packets can be shared with several applications that can do storage, billing, accounting capacity planning or visual representations in order to better comprehend what goes on within your network.

 

Collector placement

We should deploy netflow at the edge of networks, for instance , or aggregation points to better characterize the traffic flows. Data centres can also benefit from netflow analysis.

Source: wikipedia.org Author: Pazder

Alternate network design

Source: wikipedia.org Author : helix84

 

 

Network Capacity Planning

 

Netflows can simply be used to monitor links and applications present on your network in order to plan ahead and add capacity to links for instance, it can also serve as an ip accounting application for peering or transit agreements or for billing purposes

Another nice feature that is usually part of the capabilities of netflow collectors is to store information about events in a database to be able to retrieve historical data and to document trends                                             

Other protocols can supplement the information gathered with netflows such as SNMP, Syslogs , IDS alerts and monitoring, active or passive DNS and data captures where available would provide the granularity required when dealing with a particular threat.

Generating a baseline

  Using netflows would enable us to perform a baseline of network traffic to be referenced , in case of an DOS attack or unusual event occurring, this data will be very useful to enable specific actions to mitigate malicious traffic.

Open source solutions are available to get you started at little cost but you will need to spend some time setting up your system to collect and analyze the data collected, commercial offerings will allow you to profit from the manufacturer's expertise and experience in setting up your system, at a cost, but it usually comes with pre-configured reports and analysis tools as well as fingerprint creation functions. Once your baseline is created, profiles can be established and serve as tracking points or elements used as managed objects or fingerprint input criteria.

The netflow devices can also track routing instability within your network and will alert you if you encounter misconfigurations or peering anomalies

Classifying anomalies

We also need to adjust sensitivity levels to detect and classify anomalies as low, medium or high severities.

The classification of these alerts are dependent on several conditions such as trigger rates, event duration or threat patterns.

A combination of patterns such as TCP SYN traffic at a high volume during a few minutes would trigger a High Alert .

Also the number of routers and interfaces involved would also influence the classification of these alerts. This system would enable the network administrator to visualize the event and take appropriate measures such as applying an access list for instance or generate a BGP route injection either manually or via the monitoring console or application.

 

 

Denial Of Service Attack

What is a Denial Of Service attack ?

An attempt to overwhelm resources either of the network provider or the end users thus affecting network capacity or availability and if the attack is distributed, the effects would be amplified and the collateral damage can we worse than the initial impact.

Types of malicious traffic

The following list contains some of the attack vectors

Traffic rates towards specific hosts that deviate from normal internet practices as is often seen on the Internet to paralyze a company’s operations or even the network provider’s devices.

 

Misuse anomalies cover the following types of traffic

icmp anomaly   (ICMP types and data rates)

tcp null flag anomaly (mostly scanning activity)

tcp syn flag anomaly (flooding rate)

tcp rst flag anomaly (flooding rate)

ip null (protocol 0) anomaly (flooding rate)

ip fragmentation anomaly (flooding rate)

ip private address space anomaly (spoofed traffic and rate)

dns (tcp and udp port 53) anomaly (flooding rate)

total traffic bps and pps deployed against common attacks targeted

at individual network hosts including syn, smurf, fraggle (well known attack signatures)

Mitigation Techniques

You can help defend against spoofed traffic by filtering ingress and egress traffic streams by using different techniques (bogon filtering, urpf....etc...)

Commercial units would also create anomaly-specific access lists to counter that particular event , these access lists can be customized to work on specific devices or across your network and in a format type recognized by different router brands. They can also generate black holing BGP announcements towards border routers.

You first create a BGP route such as 192.168.1.1 with a next hop of 192.0.2.1 with a no_export community to ensure the routing instruction does not leave the Autonomous System, then on the edge routers a static route stating 192.0.2.1 goes to Null0, thus the final result is that traffic directed to the prepared ip address will be blackholed at the edge routers

When you have a source ip address or a list of addresses to blackhole, you then direct that traffic to the BGP route described above and within a minute, all borders routers should have the modified routing information. This can be done manually or from within the netflow device or the application

 

 

 

 

 

 

 

 

Report Generation

           

From within the netflow controller’s application or console, you can zero in a particular alert or severity pattern and generate a report that could be a high level description of the event, but also gives you the possibility to drill down to a second or third level of details to better comprehend the attack, impact of such events.

An example of second level of details can show subnets involved and to what degree whereas a third level may indicate FQDN and whois information about the hosts involved.

 

You could also generate reports based on your own created fingerprint or from a shared fingerprint to track a particular protocol and who uses it within your network or see connections talking to a specific host , for instance.

 

 

Another type of report can be created simply to outline the current state of traffic across the network and used for capacity planning or for accounting purposes as well as tracking other autonomous systems flows across our network. Other reports can give you a big picture about bandwidth hogging applications such as video traffic as an example.

 

 

Yet another type of report is generated by using the built in dos signatures that are part of many netflow devices to track well known malware activity like slammer...etc...

Specific customer profiles can also be created to track their activities and events and thus could be formatted to present them during status meetings

An example of a simple report can be seen below:

 

Summary

.

 

If one network team or individual has benefited from this article, then the author would have reached his goal. To implement such a structure is not easy, a lot of canvassing teams and individuals in various departments can be tedious, the analyzing of your network topology can be daunting, but I assure you, the benefits are well worth the efforts as the author discovered over several years of overseeing the netflow based administrative duties.

It is important not to panic if a catastrophic event occurs as you won’t be of any use to your colleagues and cannot think clearly in order to analyze and suggest a mitigation effort.

 

Establish a network of key people that can help you and to whom you can also return the favor, keeping abreast of new developments in malicious traffic techniques is also recommended.

Training sessions that can increase your knowledge and confidence level are not to be overlooked. Once you are ready, then you can provide training to other participants in your project.

If you are already involved in network management or operational management, you have the necessary basic skills to investigate if this tool is useful and pertinent to your day to day operations or planned endeavors

The end goal is to reach a point where your infrastructure will be protected and the end users will benefit from this state and although you cannot block every attempt or anticipate all future forms of malicious activity, you can be as prepared as possible with a good model of defense in depth and to present yourself as a difficult target rather than an easy victim, so netflow processing is a very good way to prepare for this and to monitor if you are successful.

On the Web

 

http://en.wikipedia.org/wiki/NetFlow  - Good description of netflows and associated topics

http://www.iana.org/assignments/icmp-parameters/icmp-parameters.xml - ICMP types listing

http://en.wikipedia.org/wiki/Transmission_Control_Protocol - TCP description

http://en.wikipedia.org/wiki/User_Datagram_Protocol - UDP description

http://en.wikipedia.org/wiki/Border_Gateway_Protocol - BGP Decryption

http://en.wikipedia.org/wiki/Bogon_(address) – Bogon Description

http://en.wikipedia.org/wiki/Ddos#Distributed_attack – DOS & DDOS Description

http://en.wikipedia.org/wiki/Domain_Name_System - DNS Description

http://en.wikipedia.org/wiki/Fqdn - FQDN Description

http://en.wikipedia.org/wiki/Intrusion_Detection_System - IDS Description

http://en.wikipedia.org/wiki/SCTP - SCTP Description

http://en.wikipedia.org/wiki/Snmp - SNMP Description

http://en.wikipedia.org/wiki/Syslog - Syslog Description

http://en.wikipedia.org/wiki/URPF - URPF Description

 

 

 

 

 

 

 

 

 

 

Glossary

 

BGP ..........................................................................................Border Gateway Protocol

 

Bogon ...................................................................................................Bogus Ip Address

 

DDOS................................................................................. Distributed Denial Of Service

 

DNS ..............................................................................................Domain Name System

 

DOS...................................................................................................... Denial Of Service

 

FQDN .................................................................................Fully Qualified Domain Name

 

IDS ........................................................................................Intrusion Detection System

 

SCTP.....................................................................Stream Control Transmission Protocol

 

SNMP....................................................................Simple Network Management Protocol

 

SYSLOG...................................................................................................System Logging

 

TCP ...................................................................................Transmission Control Protocol

 

UDP ............................................................................................User Datagram Protocol

 

URPF ...........................................................................Unicast Reverse Path Forwarding