[AusNOG] endace

[Stuart Wilson]

Hi Carl, Lincoln, Keith.

I'll try to keep the response generic, if you want more detail, feel free to contact me directly at stuart at endace.com

Network General were purchased by NetScout some time back, so the main players in the market are Niksun, Netscout & Endace, along with a bunch of other smaller players.

On to the problem.  The problem isn't so much bandwidth, it's more packet rate.  Capture requires compute and meta-data overhead per packet.

If you're monitoring high speed financial networks, like financial feed networks, then you'll find loads of MOLD-UDP formatted packets, with a minimum packet size of around 60-ish bytes.  On some GbE feeds like OPERA (Options Pricing Reporting USA) you'll get packet rates over 1E6/sec…that's very high.  In general financial networks are heading towards higher rate pipes, and smaller packets because the latency in transmission is reduced that way (and trade latency is crucial to these guys, especially the HFT (High Frequency Traders).

VoIP/Video deployment also decreases the packet size, and increases there packet rate.  Anyone running a large corp network with loads of VoIP will know this is a RPITA.

Packet loss on capture is a problem.  Because you're monitoring, you have no chance to ask for retransmission.  And if you're tracking a complex protocol like GTP-C on a UMTS Gn link, then you'd better not lose track of the protocol conversations which will happen if you lose a packet.

The big banks, Big Telcos, and the not-to-be-mentioned folk know all this, and demand no loss.  For example, the LI laws specify that an intercept must have exactly ALL of the packets, no more, no less.  Holes in the capture traffic means a weak position in court.  That's when it starts to matter.  Or for the HFT guys, they capture ALL trade packets during the day so they can tune their algorithms..missing packets means weaker statistical data, which translates directly to less aggressively tuned trade algorithms – thrust me, those guys care badly.

Disk based capture is a whole load of other problem, which I'll skip for now.  But I'll just say that the problem is not so much proving it can be done, the problem is making a reliable solution that always works.


On to how to perform packet capture
==============================

Std NIC / std OS /  tcpdump
--------------------------------------
It's a way to start for low data rates.
Good for up to GbE on std traffic (depending upon OS & Platform)
For low BW & packet rate it's OK, but expect to lose the occasional packet, even at low rates.
Timestamping can be out by a substantial fraction of a second
The main causes of packet loss are:
  a) OS compute saturation
  b) Random packet loss (packets get lost, TCP usually picks up the pieces, but remember, you're just monitoring here)

Std NIC / PF-RING / tcpdump
----------------------------------------
Better for higher data rates as PF-RING performs interrupt coalescence
Can be pushed to 10GbE
Compute is used, but better than std stack
Timestamping can be out by a substantial fraction of a second

Dedicated capture HW
--------------------------------
Direct DMA into application memory space with OS bypass
Range of interfaces (T1 through 100GBE)
HW based timestamping
Any packet rate
$$

Dedicated capture appliance
----------------------------------------
Same as above but fully managed appliance for large corporations

The last 2 solutions require full control of the solution…HW, Logic, RT SW, and application SW.

Hope that helps.


[Description: Description: Description: endace]
Stuart Wilson (CTO)

Cell +64 2175 1536

stuart at endace.com<mailto:stuart at endace.com>
www.endace.com<http://www.endace.com/>
[Description: Description: Description: power to see all]

This email (including any attachments) is intended to be read by the named recipient(s) only. If the email wasn’t addressed to you, you mustn’t use, distribute or copy any part of it. If you’ve received it in error please delete it (along with any attachments) and inform us of the error. Emails aren’t secure and can’t be guaranteed to be error free as they can be intercepted, amended, lost or destroyed. It’s your responsibility to check this email and any attachments for viruses. These risks are deemed accepted by everyone that communicates with us by email.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120604/e47c4893/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ADAFB065-9C70-4951-BF61-459AB2928FCD[2].png
Type: image/png
Size: 2235 bytes
Desc: ADAFB065-9C70-4951-BF61-459AB2928FCD[2].png
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120604/e47c4893/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 177D3EE8-8011-47E5-9B21-3B2513CC15FB[2].png
Type: image/png
Size: 9935 bytes
Desc: 177D3EE8-8011-47E5-9B21-3B2513CC15FB[2].png
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120604/e47c4893/attachment-0001.png>