[AusNOG] User-Aware Netflow

[Beeson, Ayden]

We handle our traffic enforcement using L7 capable inspection boxes, such as the cisco SCE and Procera PacketLogic boxes.

We are talking about a lot of data volume with us so we need a system that can handle the throughput we often see, which is substantial...

In essence, other than the L7 inspection stuff (which is really just signature analysis) the box is basically a username -> IP mapping system so its nothing hugely complex.

We used to use a system that used a webpage keep alive with authentication to insert entries into our front gate ACL as well as a database of entries. Netflow was then used to charge the data back to the users (as well as log what they did), this was AGES ago though and gave nowhere near the detail the enforcement engines we have now give.

It was however built in house and free which was nice :)

Thanks,
Ayden Beeson


-----Original Message-----
From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Scott O'Brien
Sent: Thursday, 27 March 2014 11:53 AM
To: ausnog at lists.ausnog.net
Subject: [AusNOG] User-Aware Netflow

G'Day Noggers,

Long time loiterer, first time poster here.  At the organisation I've been working at, we've had a requirement to attribute traffic (and the type of traffic) back to a user.  Not being able to find any open source stuff to do this, I decided to build one.


I've been building a tool that makes use of pmacct to put netflow and BGP attributes (namely community and AS Path) into a central message queue (RabbitMQ).  In basic, the tool is basically a set of consumers that listen on a user-auth message exchange and have access to auth history in my MongoDB cluster.   When a flow comes in, I'm able to add the user who had the destination IP address at the time to the netflow record before storing it on my database and increment the appropriate counters in Mongo.  I'm now working on a front-end (in Meteor) that shows information on the traffic and per user usage in near real-time.

There's a little bit of work now to abstract the tools I've built such that it's easy to use for the wider community.  I'm curious, is this style of IP based user-attribution something that people want/need?  How are others tackling this problem? (I know proxies are popular.)  If there's a demand for it, I'll abstract it, clean it up a bit and put it up on Github but only if it's an area people have found lacking.  Ideas and suggestions welcome :-)

Cheers,
- Scotty O'Brien


_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog
Charles Sturt University

| ALBURY-WODONGA | BATHURST | CANBERRA | DUBBO | GOULBURN | MELBOURNE | ONTARIO | ORANGE | PORT MACQUARIE | SYDNEY | WAGGA WAGGA |

LEGAL NOTICE
This email (and any attachment) is confidential and is intended for the use of the addressee(s) only. If you are not the intended recipient of this email, you must not copy, distribute, take any action in reliance on it or disclose it to anyone. Any confidentiality is not waived or lost by reason of mistaken delivery. Email should be checked for viruses and defects before opening. Charles Sturt University (CSU) does not accept liability for viruses or any consequence which arise as a result of this email transmission. Email communications with CSU may be subject to automated email filtering, which could result in the delay or deletion of a legitimate email before it is read at CSU. The views expressed in this email are not necessarily those of CSU.

Charles Sturt University in Australia  http://www.csu.edu.au  The Grange Chancellery, Panorama Avenue, Bathurst NSW Australia 2795  (ABN: 83 878 708 551; CRICOS Provider Numbers: 00005F (NSW), 01947G (VIC), 02960B (ACT)). TEQSA Provider Number: PV12018

Charles Sturt University in Ontario  http://www.charlessturt.ca 860 Harrington Court, Burlington Ontario Canada L7N 3N4  Registration: www.peqab.ca

Consider the environment before printing this email.

Disclaimer added by CodeTwo Exchange Rules 2007
http://www.codetwo.com