[AusNOG] Open Networking

Nik Geyer nik at neko.id.au
Wed Jul 6 02:08:19 EST 2016


Broadcom Trident 2 runs 12MB packet buffer, the original Trident was less from memory.

The Broadcom Tomahawk chipset (25/50/100GE cheap switches) run a 7.6MB packet buffer.

The Broadcom Dune (Jericho) chipset uses external DDR4/GDDR5 memory for packet buffering, so you'll see various sizing from different vendors depending on how much they throw at it. For example, the new Arista 7280CR-48 which is based on this chipset has 32GB of packet buffer. The downside to external memory is that it isn't necessarily as fast as on-ASIC/SoC packet buffers, so if you're in a latency sensitive environment it may not be the best option. Weapon of a storage switch though!

Again, it all comes down to due diligence and selecting the right hardware for your requirements. Various features can differ greatly between the chipsets also, e.g. if you want to VXLAN route at line rate without packet recirculation, don't buy a Trident 2 switch (unless it's a Nexus 9300 with it's Merchant+ Insieme ASIC wizardry). Trident 2+ is better though with single pass encap/decap and double the VXLAN performance of the Trident 2. Tomahawk and Jericho are better again, but Tomahawk has truly terrible buffers and arguably some hardware deficiencies vendors are still trying to sort out around arbitration of packets when congested, Jericho switches are expensive.

Sent from my iPhone

On 5 Jul 2016, at 10:40 AM, Andrew Yager <andrew at rwts.com.au<mailto:andrew at rwts.com.au>> wrote:

We've been running it for almost 18 months now in production and have done a fair bit of work with a number of the vendors building OS platforms.

IMO the one to watch in the ISP/"Traditional Network" space is a product called OCNOS by IP Infusion. It's a very Cisco like CLI with a range of "carrier" technologies including MPLS, RSVP, LDP implemented. We have found a couple of limitations with FRR, but still quite acceptable once tested. We've also done a fair bit of interop testing with Juniper and Cisco and are still running it in our labs on some S6000s and have had no major issues. It also has a rudimentary HQOS implementation that we haven't really drilled into yet.

The buffer discussion is something that we keep going back to with Dell. I'm told it's a Broadcom architecture limitation at the moment, but there are new chipsets due out later this year with larger packet buffers.

Thanks,
Andrew




On 4 July 2016 at 17:21, Greg Anderson <ganderson at raywhite.com<mailto:ganderson at raywhite.com>> wrote:
My question would be regarding buffer sizes.  Looking at the Dell S3048-ON BaseT switch, it has only 4MB buffers, the SFP+ S4048-ON has 12MB buffers, but this is about 1/10th to 1/3 (respectively) of what a Nexus 9300 series has (~37-52MB).

Is this really enough to run a converged network environment in a datacentre, especially with iSCSI being one of those protocols?

(Appreciate any advice either on or off list)

Thanks,
Greg.

On Mon, 4 Jul 2016 at 14:46 Matt Smee <m.smee at unsw.edu.au<mailto:m.smee at unsw.edu.au>> wrote:
For what it’s worth, I remember hearing Cumulus now supports PoE+ from 3.0+ though limited hardware so far:

https://docs.cumulusnetworks.com/display/DOCS/Power+over+Ethernet+-+PoE
https://cumulusnetworks.com/support/linux-hardware-compatibility-list/
‘edgecore AS4610-54P<http://www.edge-core.com/ProdDtl.asp?sno=472&AS4610-54P>’

I’ve been impressed with the zero touch deployment part of it but still learning/playing with it at the moment. Though it doesn’t seem quite yet ideal in the enterprise access space, there’s some missing features that I do like such as the various L2 security options but then again it’s definitely more Data Centre focused than enterprise access, though that may change in the future…

I’d also give a +1 to Ben, why pay so much for simple routing/L3 switching? In 3-5 years with some maturity I can’t see how you could ever justify the $bigvendor prices for some deployments or at least some devices within the network. Looking at doing 40/100G and we can see its definitely looking like a good option even now.


From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net<mailto:ausnog-bounces at lists.ausnog.net>] On Behalf Of Simon Attwell
Sent: Monday, 4 July 2016 12:35 PM
To: ausnog at lists.ausnog.net<mailto:ausnog at lists.ausnog.net>
Subject: [AusNOG] Open Networking

Hi All,

Just curious how many of you have deployed / are deploying / Open Networking in production environments.
I'm interested to see if ON is making its way down to the edge (1Gbps PoE/PoE+) or if it's mainly being used at the distribution / core layers or at the service provider level where there's little end device connectivity and it's more about moving the packets around.

Comments on hardware choice / stability / longevity / MTBF / support, are also appreciated.

>From a Cumulus perspective it looks like 1 Gbps - 100Gbps is where things are focused.
Nothing with PoE/PoE+ support so it looks like at the moment we're only talking about datacenter switching.

What I don't see deployed today is a lot of technology mix, especially in switching. Customers have a preference and for support / interop / personal reasons tend to stick with a single vendor for switching.
In the past this has made sense as switches did not always play well with others.

I'm wondering what you all think the 3 - 5 year picture looks like.

I suspect it looks a lot like the current virtualization market. A few major players with custom software built on open source foundations, being hardware agnostic and the holdouts trying to ignore the fact that the industry is fundamentally changing.

- Simon








_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog




--
Andrew Yager, CEO (BCompSc, JNCIS-SP, MACS (Snr) CP)
Real World Technology Solutions - IT People you can trust
Voice | Data | IT Procurement | Managed IT
rwts.com.au<http://rwts.com.au> | 1300 798 718

[http://www.rwts.com.au/email-signature-logo.jpg]
Real World is a Dell Premier Partner

This document should be read only by those persons to whom it is addressed and its content is not intended for use by any other persons. If you have received this message in error, please notify us immediately. Please also destroy and delete the message from your computer. Any unauthorised form of reproduction of this message is strictly prohibited. We are not liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. Please consider the environment before printing this e-mail.
_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20160705/dadd82b4/attachment.html>


More information about the AusNOG mailing list