[AusNOG] RouterBoard

Tue Mar 11 11:52:51 EST 2014

Lol as murphy would have it i saw this about 2 days after I got my devices

Sent from my smart phone

-----Original Message-----
From: Greg McLennan [mclennan at internode.on.net]
Received: Tuesday, 11 Mar 2014, 11:51
To: Alex Samad - Yieldbroker [Alex.Samad at yieldbroker.com]
CC: ausnog at lists.ausnog.net [ausnog at lists.ausnog.net]
Subject: Re: [AusNOG] RouterBoard

If you have time to wait, Mikrotik are releasing a new CCR item later this year. It will be interesting to see througput figures.

CCR1072
(72 core Tilera CPU, 8x 10GBit SFP+, dual PSU and more, coming Q4, 2014),

There are also some new/other CCR's coming to market with more SFP ports :)

As detailed in this recent .PDF  http://mum.mikrotik.com/presentations/IT14/it14.pdf

Cheers
Greg.

----- Original Message -----
From:
"Alex Samad - Yieldbroker" <Alex.Samad at yieldbroker.com>

To:
"Tom Berryman" <tom at connectivityit.com.au>, "David Bomba" <turbo124 at gmail.com>, "Damian Guppy" <the.damo at gmail.com>
Cc:
"ausnog at lists.ausnog.net" <ausnog at lists.ausnog.net>
Sent:
Mon, 10 Mar 2014 03:04:52 +0000
Subject:
Re: [AusNOG] RouterBoard

Hi

Yeah I have read a bit about the single core issues on the CCR, the last time I looked because of this I saw 3 cpu’s floating around 30-60% non-maxed

I started this by looking at VM routers, but I couldn’t get pas the 1Gb/s nic. There is Brocades vyatta, but its just way to expensive compared to routeros

My constraints are more along the lines of, I have core switching already, I wanted to add some core routing
I am happy with the CCR on $$ on CLI
I am not so happy about the current performance, be that limited to my testing via iperf…  I am nearly ready to live with that, on the presumption I can get 8+Gbs with multi stream tcp.

My current risk is support, especially as I have had a hard time working through this CCR performance issue.  I don’t want to roll out 2 of these at each DC and then run into a bug, where the only solution is to throw it away.  I can duplicate about all the functionality of routeros on linux apart from BGP and OSPF. And I am guessing if I looked really hard and spent some time I could get that working as well.

So taking into account their low $$ I can also live with minimal support if I have another hardware solution to match up with it on a similar $$ level.  If they can talk iBGP, OSPF and VRRP, then I am just about set.  :)

So I thought I would dig into the knowledge pool that is AUSNOG and find out what other devices like RouterOS are being used..

Alex

From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Tom Berryman
Sent: Monday, 10 March 2014 1:45 PM
To: David Bomba; Damian Guppy
Cc: ausnog at lists.ausnog.net
Subject: Re: [AusNOG] RouterBoard

David is correct, the Tilera CPU with RouterOS does struggle with single threaded processes – worse than just BGP operating on a single core, all routing (OSPF, RIP and static) processing will happen on the same core. ROS7 is likely to change this (rumours).
But still, the CCR range has forced a lot of people to change how they think about routing (at a relatively small scale) – and has certainly bought the cost down. “Routed” packets per dollar, I don’t think anything in the new hardware market can compete.
Vyatta has other challenges like x86 PCI architecture that will limit your total throughput – however things like processing BGP are drastically improved compared to ROS. Ubiquity has ported the Vyatta/VyOS to MIPS processors, possibly worth a look but I don’t think it has any SFP+.
Given Alex’s application – storage – a layer 3 solution is not likely to be the best.
Alex, have you considered something like the Brocade VDX Ethernet fabric (VDX could enable 40g native interfaces)? Or at least other layer 2 solutions? I noticed that you have tried routing on switches (Dell) perhaps something with some more power with this design would yield better results for you?
Tom

From: AusNOG [mailto:ausnog-bounces at listsausnog.net<mailto:ausnog-bounces at lists.ausnog.net>] On Behalf Of David Bomba
Sent: Monday, 10 March 2014 12:32 PM
To: Damian Guppy
Cc: ausnog at lists.ausnog.net<mailto:ausnog at lists.ausnog.net>
Subject: Re: [AusNOG] RouterBoard

I believe he has the CCR1036-8G-2S+ which has 2x10GB SFP+ ports.

I think the issue he is hitting is the single threaded nature of routerOS for a lot of its functionality.
BGP, for instance spins on a single core. Until ROS becomes multi-core aware/capable a lot of its functionality will be capped at the per core performance.

On 10 March 2014 12:26, Damian Guppy <the.damo at gmail.com<mailto:the.damo at gmail.com>> wrote:
CCR1036 has no 10G ports, only 1G, so im not sure why you would expect to get a single TCP stream past 1G (even with LACP since that is not how LACP works)

--Damian

On Mon, Mar 10, 2014 at 6:58 AM, Alex Samad - Yieldbroker <Alex.Samad at yieldbroker.com<mailto:Alex.Samad at yieldbroker.com>> wrote:
Hi

So I have tested routerOS ... in VM and also bought the ccr1036.

I'm not 100% happy with the ccr1036.  Basically can't push 1 tcp stream past 1Gb/s I can get 8-9Gb/s with multiple streams. I can get UDP up to 9.8Gb/s

I like routerOS interface (have to admit I like the vyatta better from what I saw).

But now I need to find something similar to these devices around the same price and around the same performance, I would like to push it all to a VM but Brocade want my 1st and 2nd child ...

So routerOS support is nowhere close to Cisco and rightly so for the price, so I have some hesitancy in rolling these things out, especially if they are going into the core.

So are there any suggestions from the list ?

Alex

_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog