[AusNOG] User-Aware Netflow

[Jonathan Thorpe]

Hi Scotty,

This is an interesting idea.

The main challenge with this is determining what user is using a given IP address at a particular point in time. If a mechanism can be used to determine this, tools such as pmacct/nfacct/sfacct can be used (and export to a RabbitMQ queue if need be) without having to start from scratch with the collection and classification.

I've had a somewhat related requirement to attribute traffic to a virtual machine where a given VM may have multiple IP addresses on different subnets and collection is not done at a hyervisor level.

I'm not sure how to do this with something like MongoDB, however with a regular SQL database, you could achieve this as follows:

1. Database table maintains an association of users and their respective IP. Obviously some mechanism has to maintain this table.
2. Upon inserting flows into the database, a pre-insert/update trigger looks up a user ID for a given IP and sets this for the flow record.

There are a few ways to handle the resulting record. For example, you could set a fake ingress/egress interface that corresponds with each user based on a numeric user ID and then use existing flow tools to filter by user based on an interface identifier that corresponds with the user id.

As for mapping an individual user to an IP address without an authenticated proxy is a tricky one. How would you do this? The only way I can think of is to enforce 802.1x and use it with DHCP; Doing a lookup of 802.1x authenticated sessions and map that MAC address to an IP in DHCP could work?

Kind Regards,
Jonathan





-----Original Message-----
From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Scott O'Brien
Sent: Thursday, 27 March 2014 11:53 AM
To: ausnog at lists.ausnog.net
Subject: [AusNOG] User-Aware Netflow

G'Day Noggers,

Long time loiterer, first time poster here.  At the organisation I've been working at, we've had a requirement to attribute traffic (and the type of traffic) back to a user.  Not being able to find any open source stuff to do this, I decided to build one.

I've been building a tool that makes use of pmacct to put netflow and BGP attributes (namely community and AS Path) into a central message queue (RabbitMQ).  In basic, the tool is basically a set of consumers that listen on a user-auth message exchange and have access to auth history in my MongoDB cluster.   When a flow comes in, I'm able to add the user who had the destination IP address at the time to the netflow record before storing it on my database and increment the appropriate counters in Mongo.  I'm now working on a front-end (in Meteor) that shows information on the traffic and per user usage in near real-time.

There's a little bit of work now to abstract the tools I've built such that it's easy to use for the wider community.  I'm curious, is this style of IP based user-attribution something that people want/need?  How are others tackling this problem? (I know proxies are popular.)  If there's a demand for it, I'll abstract it, clean it up a bit and put it up on Github but only if it's an area people have found lacking.  Ideas and suggestions welcome :-)

Cheers,
- Scotty O'Brien


_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog