[AusNOG] So who's read an RFC or Internet Draft?

[Mark Smith]

On 12 Oct 2015 9:19 am, "Philip Loenneker" <Philip.Loenneker at tasmanet.com.au>
wrote:
>
> <snip>
> You'll also incur increased collector costs because you have to buy
bigger ones of those too as the volume of collection data increases,
because you can't spread the collection data across a number of collectors.
> </snip>
>
> You can spread the load, you just have to be a little creative :)
>

You've only spread the load across a number of collectors, and you've done
that by adding another element for which the only way to scale is to buy a
bigger one, so you're still pursuing vertical scaling. You've also added a
another point of failure to the whole system, because it is another element
inline (in serial) with the netflow traffic path.

A simple test to work out if you're vertically or horizontally scaling is
can you buy another device of the *same* size/capacity and it adds
processing capacity to the system? If it does, then your scaling model is
horizontal.

> A few weeks ago we successfully got a proof-of-concept up and running
where we have NetFlow feeds going to a load balancer (NetScaler in our
case), which then used source-IP persistence to spread the load over 4
collectors. The collectors run as stateless UDP servers of course, which
makes it difficult to check whether the services are running, so one of our
guys set up an additional service running on TCP that would give a response
code with the status of the collector, so we have health-checks and
auto-failover working. The collectors we are using are nProbe on Linux,
which allows us to dump the raw flows directly into MySQL, so we load
balanced that too and set up a cluster of MySQL servers as the destination.
Tests have been quite promising, and we get redundancy in a system that
wasn't designed to. All of those servers are virtual, which makes scaling
even simpler.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20151012/d1836269/attachment.html>