<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div>
<div>
<div>Hi Carl, Lincoln, Keith.</div>
<div><br>
</div>
<div>I'll try to keep the response generic, if you want more detail, feel free to contact me directly at stuart@endace.com</div>
<div><br>
</div>
<div>Network General were purchased by NetScout some time back, so the main players in the market are Niksun, Netscout & Endace, along with a bunch of other smaller players.</div>
<div><br>
</div>
<div>On to the problem. The problem isn't so much bandwidth, it's more packet rate. Capture requires compute and meta-data overhead per packet.</div>
<div><br>
</div>
<div>If you're monitoring high speed financial networks, like financial feed networks, then you'll find loads of MOLD-UDP formatted packets, with a minimum packet size of around 60-ish bytes. On some GbE feeds like OPERA (Options Pricing Reporting USA) you'll
get packet rates over 1E6/sec…that's very high. In general financial networks are heading towards higher rate pipes, and smaller packets because the latency in transmission is reduced that way (and trade latency is crucial to these guys, especially the HFT
(High Frequency Traders).</div>
<div><br>
</div>
<div>VoIP/Video deployment also decreases the packet size, and increases there packet rate. Anyone running a large corp network with loads of VoIP will know this is a RPITA.</div>
<div><br>
</div>
<div>Packet loss on capture is a problem. Because you're monitoring, you have no chance to ask for retransmission. And if you're tracking a complex protocol like GTP-C on a UMTS Gn link, then you'd better not lose track of the protocol conversations which
will happen if you lose a packet.</div>
<div><br>
</div>
<div>The big banks, Big Telcos, and the not-to-be-mentioned folk know all this, and demand no loss. For example, the LI laws specify that an intercept must have exactly ALL of the packets, no more, no less. Holes in the capture traffic means a weak position
in court. That's when it starts to matter. Or for the HFT guys, they capture ALL trade packets during the day so they can tune their algorithms..missing packets means weaker statistical data, which translates directly to less aggressively tuned trade algorithms
– thrust me, those guys care badly.</div>
<div><br>
</div>
<div>
<div>Disk based capture is a whole load of other problem, which I'll skip for now. But I'll just say that the problem is not so much proving it can be done, the problem is making a reliable solution that always works.</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>On to how to perform packet capture</div>
<div>==============================</div>
<div><br>
</div>
<div>Std NIC / std OS / tcpdump</div>
<div>--------------------------------------</div>
<div>It's a way to start for low data rates.</div>
<div>Good for up to GbE on std traffic (depending upon OS & Platform)</div>
<div>For low BW & packet rate it's OK, but expect to lose the occasional packet, even at low rates.</div>
<div>Timestamping can be out by a substantial fraction of a second</div>
<div>The main causes of packet loss are:</div>
<div> a) OS compute saturation</div>
<div> b) Random packet loss (packets get lost, TCP usually picks up the pieces, but remember, you're just monitoring here)</div>
<div><br>
</div>
<div>Std NIC / PF-RING / tcpdump</div>
<div>----------------------------------------</div>
<div>Better for higher data rates as PF-RING performs interrupt coalescence</div>
<div>Can be pushed to 10GbE</div>
<div>Compute is used, but better than std stack</div>
<div>
<div>Timestamping can be out by a substantial fraction of a second</div>
</div>
<div><br>
</div>
<div>Dedicated capture HW</div>
<div>--------------------------------</div>
<div>Direct DMA into application memory space with OS bypass</div>
<div>Range of interfaces (T1 through 100GBE)</div>
<div>HW based timestamping</div>
<div>Any packet rate</div>
<div>$$</div>
<div><br>
</div>
<div>Dedicated capture appliance</div>
<div>----------------------------------------</div>
<div>Same as above but fully managed appliance for large corporations</div>
<div>
<div>
<div><br>
</div>
</div>
<div>The last 2 solutions require full control of the solution…HW, Logic, RT SW, and application SW.</div>
<div><br>
</div>
<div>Hope that helps.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<table class="MsoNormalTable" border="0" cellspacing="3" cellpadding="0" width="280" style="width: 210pt; font-family: Calibri; font-size: medium; ">
<tbody>
<tr>
<td style="padding-top: 0.75pt; padding-right: 0.75pt; padding-bottom: 0.75pt; padding-left: 0.75pt; ">
<p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-bottom: 0.0001pt; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif; ">
<span style="font-size: 12pt; color: rgb(31, 73, 125); font-family: 'Times New Roman', serif; "><img width="93" height="68" id="Picture_x0020_1" src="cid:D8A63753-A87D-44F8-82B7-A502248B55FC" alt="Description: Description: Description: endace" type="image/png"></span><span style="font-size: 12pt; color: rgb(31, 73, 125); font-family: 'Times New Roman', serif; "><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-bottom: 0.0001pt; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif; ">
<b><span style="font-size: 10pt; color: rgb(68, 68, 68); font-family: Tahoma, sans-serif; ">Stuart Wilson (CTO)<o:p></o:p></span></b></p>
<p class="MsoNormalCxSpMiddle"><span style="font-size: 10pt; color: rgb(68, 68, 68); font-family: Tahoma, sans-serif; ">Cell +64 2175 1536<o:p></o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin-bottom: 12pt; "><span style="font-size: 10pt; color: rgb(68, 68, 68); font-family: Tahoma, sans-serif; "><a href="mailto:stuart@endace.com" style="color: blue; text-decoration: underline; "><span style="color: blue; ">stuart@endace.com</span></a><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-bottom: 0.0001pt; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif; ">
<span style="font-size: 10pt; color: rgb(68, 68, 68); font-family: Tahoma, sans-serif; "><a href="http://www.endace.com/" style="color: blue; text-decoration: underline; "><span style="color: rgb(68, 68, 68); ">www.endace.com</span></a><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-bottom: 0.0001pt; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif; ">
<span style="font-size: 12pt; color: rgb(31, 73, 125); font-family: 'Times New Roman', serif; "><img border="0" width="151" height="38" id="Picture_x0020_2" src="cid:3F6DD894-0478-49EA-A353-7A1033DAA5A2" alt="Description: Description: Description: power to see all" type="image/png"></span><span style="font-size: 12pt; color: rgb(31, 73, 125); font-family: 'Times New Roman', serif; "><o:p></o:p></span></p>
</td>
</tr>
<tr>
<td style="padding-top: 0.75pt; padding-right: 0.75pt; padding-bottom: 0.75pt; padding-left: 0.75pt; ">
<p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-bottom: 0.0001pt; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif; ">
<span style="font-size: 7pt; color: rgb(68, 68, 68); font-family: Arial, sans-serif; ">This email (including any attachments) is intended to be read by the named recipient(s) only. If the email wasn’t addressed to you, you mustn’t use, distribute or copy any
part of it. If you’ve received it in error please delete it (along with any attachments) and inform us of the error. Emails aren’t secure and can’t be guaranteed to be error free as they can be intercepted, amended, lost or destroyed. It’s your responsibility
to check this email and any attachments for viruses. These risks are deemed accepted by everyone that communicates with us by email.</span></p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</body>
</html>