[AusNOG] Buffers (was Re: Switching Recommendations)

[Paul Gear]

On 07/15/2013 08:53 AM, Lincoln Dale wrote:
> On Sun, Jul 14, 2013 at 8:11 PM, Greg M <gregm at servu.net.au 
> <mailto:gregm at servu.net.au>> wrote:
>
>     Just be aware that if you are pushing the 3560X hard in a server
>     environment
>     you will experience packet drops due to shitty buffers on the
>     2960S/3560X
>     and 3750X series. We ended up choosing the 4948E because of this...
>
>
> Most 1GE switches have anemic buffers which results in 
> less-than-stellar performance if you drive them hard, have bursts or 
> incast traffic.
> Alas, this doesn't even figure in most people's knowledge/requests 
> when it comes it networking.
>
> A good example of the issue you've described is at 
> <http://dev.datasift.com/blog/big-data-bigger-networking> and 
> <http://dev.datasift.com/blog>

Just curious: when/where does one typically draw the line between big 
buffers being required, and big buffers causing latency issues due to 
buffer bloat?  The information i've read suggests that buffer bloat is 
not only caused by large buffers on edge routers, but at many points in 
the network.

Conventional wisdom on the one hand says that for high-volume 
environments (iSCSI storage is a typical example; high-bandwidth 
international links might be another - please correct me if i'm wrong), 
more buffers is better.  On a recent Packet Pushers show where Arista 
were talking about their new switches, they pointed out that their 
buffers seemed overly large, but at the high bandwidths they were 
serving, this was only 250 ms or thereabouts (my memory is a bit hazy, 
but i think it was about 512 MB per 10 Gbps port).  But on the other 
hand, trivially small buffers in CPE/PE routers can result in long 
delays, and even 250 ms is a bit too much for voice traffic (although 
often we can make do - i swear i had about 500 ms this morning between 
my wife's mobile on Vodafone and mine on Telstra).

How does one determine the optimal buffer size (and hence switch 
selection) for a particular environment?  Is there a rule of thumb based 
on the bandwidth of the link and the maximum time one wants a packet to 
queue?  (And how would one even determine what this maximum might be?  I 
would think that it varies depending upon the application.)  I guess 
this paragraph's questions are mostly directed to Greg & Lincoln - in 
the cases you've mentioned, how did you know that small buffers were the 
problem?

If this is something well-covered in the literature, please feel free to 
point me in that direction.

Regards,
Paul

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20130715/57686655/attachment.html>