[AusNOG] Best Open-Source Flow analyzer tools

[John Edwards]

On 13/12/2010, at 11:30 PM, Sean K. Finn wrote:

> Hence why I'm looking for an open-source alternative that I can rip the guts out of.

I've had good results before ripping the guts out of this netflow collector:

http://iagu.net/software/netflow-collector.html

It's written in Perl, but since it compiles once and then runs in a loop the performance is ok - in my application I added in per-IP-address in-memory 64-bit counters for a variety of subnet and ToS matching rules, and deprecated the logging/writing features to take disk out of the equation. On a pentium-4 class machine, it could handle the throughput of 2x Cisco 7301 broadband aggregation routers.

I added in a special type of netflow packet to make it answer queries on the counters. We later gave it the ability to dump its in-memory counters to a file and read them back in on restart, so that the software could be upgraded or reconfigured for new subnets with minimal data loss. 

This was around 2003, in response to the lack of accounting options available for L3 Bridged DSL, and long-running L3 PPP sessions through the same network. I understand that the same system is still running today supporting tens of thousands of broadband users.

John