<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">I think Tony's suggestion of Cumulus +
whitebox switches makes a lot more sense than building your own.
So does VyOS/Vyatta on bare metal x86, although I don't think it
would reach the performance levels of an embedded device (although
don't quote me on that - I haven't tested it).<br>
<br>
Paul<br>
<br>
On 03/10/2014 01:59 PM, Alex Samad - Yieldbroker wrote:<br>
</div>
<blockquote
cite="mid:A3FB5D9FD28C50429DF7692DC31054E60681F8E0@DC1INTADCW8201.yieldbroker.com"
type="cite">
<meta http-equiv="Context-Type" content="text/html;
charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<div class="WordSection1">
<p class="MsoNormal"><span>PFSense … no cli then no.</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>I think as a last resort I might look
at building my own again.</span></p>
<p class="MsoNormal"><span>Alex</span></p>
<p class="MsoNormal"><span> </span></p>
<div>
<div>
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> Nathan Brookfield
[<a class="moz-txt-link-freetext" href="mailto:Nathan.Brookfield@simtronic.com.au">mailto:Nathan.Brookfield@simtronic.com.au</a>]
<br>
<b>Sent:</b> Monday, 10 March 2014 2:57 PM<br>
<b>To:</b> Alex Samad - Yieldbroker; Matt Perkins;
<a class="moz-txt-link-abbreviated" href="mailto:ausnog@lists.ausnog.net">ausnog@lists.ausnog.net</a><br>
<b>Subject:</b> RE: [AusNOG] RouterBoard</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span>Zebra/Quagga has been around for a
very long time and is a very stable set of daemon’s and
the backend to Vyatta so any possible issue you would have
I am sure finding an answer online would be extremely
easy. I think I have had one bug with it in the last 10
years and that was when 4 byte ASN’s came mainstream and
that is long fixed.</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>PFSense is more a Firewall than a
router, it does not have a CLI either from my experience.
I love it as an edge firewall ,t is extremely efficient
and reliable but short of a Gateway I would not use it for
routing at the DC.</span></p>
<p class="MsoNormal"><span> </span></p>
<div>
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> Alex Samad - Yieldbroker [<a
moz-do-not-send="true"
href="mailto:Alex.Samad@yieldbroker.com">mailto:Alex.Samad@yieldbroker.com</a>]
<br>
<b>Sent:</b> Monday, 10 March 2014 2:54 PM<br>
<b>To:</b> Nathan Brookfield; Matt Perkins; <a
moz-do-not-send="true"
href="mailto:ausnog@lists.ausnog.net">
ausnog@lists.ausnog.net</a><br>
<b>Subject:</b> RE: [AusNOG] RouterBoard</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span>Tempting, time ?</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>Had a look at zebra and a very very
quick look at bird.</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>The other issue is support.</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>A few people have suggested
pfsense, it looks interesting, I think I looked at this a
while back, but can’t remember why I didn’t proceed
further.</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>Alex</span></p>
<p class="MsoNormal"><span> </span></p>
<div>
<div>
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> AusNOG [<a moz-do-not-send="true"
href="mailto:ausnog-bounces@lists.ausnog.net">mailto:ausnog-bounces@lists.ausnog.net</a>]
<b>On Behalf Of </b>Nathan Brookfield<br>
<b>Sent:</b> Monday, 10 March 2014 2:48 PM<br>
<b>To:</b> Matt Perkins; <a moz-do-not-send="true"
href="mailto:ausnog@lists.ausnog.net">ausnog@lists.ausnog.net</a><br>
<b>Subject:</b> Re: [AusNOG] RouterBoard</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span>If you’re finding you can do
everything in Linux why not just throw Zebra or Bird
into the mix and solve your issues that way?</span></p>
<p class="MsoNormal"><span> </span></p>
<div>
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> AusNOG [<a moz-do-not-send="true"
href="mailto:ausnog-bounces@lists.ausnog.net">mailto:ausnog-bounces@lists.ausnog.net</a>]
<b>On Behalf Of </b>Matt Perkins<br>
<b>Sent:</b> Monday, 10 March 2014 2:43 PM<br>
<b>To:</b> <a moz-do-not-send="true"
href="mailto:ausnog@lists.ausnog.net">ausnog@lists.ausnog.net</a><br>
<b>Subject:</b> Re: [AusNOG] RouterBoard</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">My opinion and we have been using
CCR's since the first one arrived in Australia is they
are reasonable kit. Overall I find the performance and
price excellent. But there have been just to many
unexplained problems for my liking. Not that we dont
still use them on the edge we do. Im about to roll one
out to quite a far destination over the next week. But
the site has a backup and it is non essential. They are
not ready for the core and they are not ready for a
network that needs 4 9's Perhaps we are at 99.9 now.
Then again if I had to run on a tight budget and I had
the opportunity to trade off reliability. It would be
the number one on my list.
<br>
<br>
Speed<br>
Reliability <br>
Price<br>
<br>
Pick any 3 CCR's fit in to the Speed and Price corner of
the triangle. <br>
<br>
<br>
Matt<br>
<br>
<br>
<br>
On 10/03/14 2:04 PM, Alex Samad - Yieldbroker wrote:</p>
</div>
<blockquote>
<p class="MsoNormal"><span>Hi</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>Yeah I have read a bit about
the single core issues on the CCR, the last time I
looked because of this I saw 3 cpu’s floating around
30-60% non-maxed</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>I started this by looking at VM
routers, but I couldn’t get pas the 1Gb/s nic. There
is Brocades vyatta, but its just way to expensive
compared to routeros</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>My constraints are more along
the lines of, I have core switching already, I wanted
to add some core routing.</span></p>
<p class="MsoNormal"><span>I am happy with the CCR on $$
on CLI</span></p>
<p class="MsoNormal"><span>I am not so happy about the
current performance, be that limited to my testing via
iperf… I am nearly ready to live with that, on the
presumption I can get 8+Gbs with multi stream tcp.</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>My current risk is support,
especially as I have had a hard time working through
this CCR performance issue. I don’t want to roll out
2 of these at each DC and then run into a bug, where
the only solution is to throw it away. I can
duplicate about all the functionality of routeros on
linux apart from BGP and OSPF. And I am guessing if I
looked really hard and spent some time I could get
that working as well.</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>So taking into account their
low $$ I can also live with minimal support if I have
another hardware solution to match up with it on a
similar $$ level. If they can talk iBGP, OSPF and
VRRP, then I am just about set. </span><span>J</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>So I thought I would dig into
the knowledge pool that is AUSNOG and find out what
other devices like RouterOS are being used..</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span>Alex</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span> </span></p>
<div>
<div>
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> AusNOG [<a moz-do-not-send="true"
href="mailto:ausnog-bounces@lists.ausnog.net">mailto:ausnog-bounces@lists.ausnog.net</a>]
<b>On Behalf Of </b>Tom Berryman<br>
<b>Sent:</b> Monday, 10 March 2014 1:45 PM<br>
<b>To:</b> David Bomba; Damian Guppy<br>
<b>Cc:</b> <a moz-do-not-send="true"
href="mailto:ausnog@lists.ausnog.net">ausnog@lists.ausnog.net</a><br>
<b>Subject:</b> Re: [AusNOG] RouterBoard</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><a moz-do-not-send="true"
name="_MailEndCompose"><span>David is correct, the
Tilera CPU with RouterOS does struggle with single
threaded processes – worse than just BGP operating
on a single core, all routing (OSPF, RIP and
static) processing will happen on the same core.
ROS7 is likely to change this (rumours).</span></a></p>
<p class="MsoNormal"><span>But still, the CCR range has
forced a lot of people to change how they think
about routing (at a relatively small scale) – and
has certainly bought the cost down. “Routed” packets
per dollar, I don’t think anything in the new
hardware market can compete.</span></p>
<p class="MsoNormal"><span>Vyatta has other challenges
like x86 PCI architecture that will limit your total
throughput – however things like processing BGP are
drastically improved compared to ROS. Ubiquity has
ported the Vyatta/VyOS to MIPS processors, possibly
worth a look but I don’t think it has any SFP+.</span></p>
<p class="MsoNormal"><span>Given Alex’s application –
storage – a layer 3 solution is not likely to be the
best.
</span></p>
<p class="MsoNormal"><span>Alex, have you considered
something like the Brocade VDX Ethernet fabric (VDX
could enable 40g native interfaces)? Or at least
other layer 2 solutions? I noticed that you have
tried routing on switches (Dell) perhaps something
with some more power with this design would yield
better results for you?</span></p>
<p class="MsoNormal"><span>Tom</span></p>
<p class="MsoNormal"><b><span> </span></b></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> AusNOG [<a moz-do-not-send="true"
href="mailto:ausnog-bounces@lists.ausnog.net">mailto:ausnog-bounces@lists.ausnog.net</a>]
<b>On Behalf Of </b>David Bomba<br>
<b>Sent:</b> Monday, 10 March 2014 12:32 PM<br>
<b>To:</b> Damian Guppy<br>
<b>Cc:</b> <a moz-do-not-send="true"
href="mailto:ausnog@lists.ausnog.net">ausnog@lists.ausnog.net</a><br>
<b>Subject:</b> Re: [AusNOG] RouterBoard</span></p>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal">I believe he has the
CCR1036-8G-2S+ which has 2x10GB SFP+ ports.<br>
<br>
I think the issue he is hitting is the single
threaded nature of routerOS for a lot of its
functionality.</p>
</div>
<p class="MsoNormal">BGP, for instance spins on a
single core. Until ROS becomes multi-core
aware/capable a lot of its functionality will be
capped at the per core performance.</p>
</div>
<div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On 10 March 2014 12:26, Damian
Guppy <<a moz-do-not-send="true"
href="mailto:the.damo@gmail.com" target="_blank">the.damo@gmail.com</a>>
wrote:</p>
<blockquote>
<div>
<div>
<p class="MsoNormal">CCR1036 has no 10G ports,
only 1G, so im not sure why you would expect
to get a single TCP stream past 1G (even
with LACP since that is not how LACP works)</p>
</div>
<div>
<p class="MsoNormal"><span> </span></p>
</div>
<div>
<p class="MsoNormal"><span>--Damian</span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On Mon, Mar 10,
2014 at 6:58 AM, Alex Samad -
Yieldbroker <<a
moz-do-not-send="true"
href="mailto:Alex.Samad@yieldbroker.com"
target="_blank">Alex.Samad@yieldbroker.com</a>>
wrote:</p>
<blockquote>
<p class="MsoNormal">Hi<br>
<br>
<br>
So I have tested routerOS ... in VM
and also bought the ccr1036.<br>
<br>
I'm not 100% happy with the ccr1036.
Basically can't push 1 tcp stream
past 1Gb/s I can get 8-9Gb/s with
multiple streams. I can get UDP up
to 9.8Gb/s<br>
<br>
I like routerOS interface (have to
admit I like the vyatta better from
what I saw).<br>
<br>
But now I need to find something
similar to these devices around the
same price and around the same
performance, I would like to push it
all to a VM but Brocade want my 1st
and 2nd child ...<br>
<br>
So routerOS support is nowhere close
to Cisco and rightly so for the
price, so I have some hesitancy in
rolling these things out, especially
if they are going into the core.<br>
<br>
So are there any suggestions from
the list ?<br>
<br>
Alex<br>
<br>
_______________________________________________<br>
AusNOG mailing list<br>
<a moz-do-not-send="true"
href="mailto:AusNOG@lists.ausnog.net"
target="_blank">AusNOG@lists.ausnog.net</a><br>
<a moz-do-not-send="true"
href="http://lists.ausnog.net/mailman/listinfo/ausnog"
target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></p>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><br>
_______________________________________________<br>
AusNOG mailing list<br>
<a moz-do-not-send="true"
href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a><br>
<a moz-do-not-send="true"
href="http://lists.ausnog.net/mailman/listinfo/ausnog"
target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></p>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
<p class="MsoNormal"> </p>
<pre>_______________________________________________</pre>
<pre>AusNOG mailing list</pre>
<pre><a moz-do-not-send="true" href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a></pre>
<pre><a moz-do-not-send="true" href="http://lists.ausnog.net/mailman/listinfo/ausnog">http://lists.ausnog.net/mailman/listinfo/ausnog</a></pre>
</blockquote>
<p class="MsoNormal"> </p>
<pre>-- </pre>
<pre>/* Matt Perkins </pre>
<pre> Direct 1300 137 379 Spectrum Networks Ptd. Ltd. </pre>
<pre> Office 1300 133 299 <a moz-do-not-send="true" href="mailto:matt@spectrum.com.au">matt@spectrum.com.au</a> </pre>
<pre> Fax 1300 133 255 Level 6, 350 George Street Sydney 2000</pre>
<pre> SIP <a moz-do-not-send="true" href="mailto:1300137379@sip.spectrum.com.au">1300137379@sip.spectrum.com.au</a> </pre>
<pre> PGP/GNUPG Public Key can be found at <a moz-do-not-send="true" href="http://pgp.mit.edu">http://pgp.mit.edu</a> </pre>
<pre>*/</pre>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
AusNOG mailing list
<a class="moz-txt-link-abbreviated" href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a>
<a class="moz-txt-link-freetext" href="http://lists.ausnog.net/mailman/listinfo/ausnog">http://lists.ausnog.net/mailman/listinfo/ausnog</a>
</pre>
</blockquote>
<br>
</body>
</html>