[AusNOG] Juniper vs Cisco vs Brocade - what's best for BGP routing?

[Reuben Farrelly]

On 11/12/2013 4:46 PM, Andrew White wrote:
> Hey guys,
>
> I've recently set up my own AS and I'm looking at broadcasting my own
> BGP. I'm wanting to find some decent hardware at a reasonable price to
> do so.
>
> The same router will also run my servers (about 50 VMs/3 physical boxes)
> and have to deal with multiple upstream providers (two currently, but
> more to come at my DC).
>
> I also want something that can hold a big BGP routing table.

Lets step back for a minute and evaluate things, because what I've seen 
so far are lots of technical solutions and vendor products looking for a 
problem, not problems looking for solutions.

I think you need to begin with some planning and working out the big 
picture requirements.  Where you are now, where you want to go, and how 
you're going to get there.  What capabilities you have now, what 
capabilities you think you'll need later.  How fast is your network 
growing and what happens if that rate of growth slows down?  What 
skillsets you have in house, what you can outsource (as it's sometimes 
better to pay someone else with expertise than skilling up yourself, if 
you rarely need the skills), and what level of technical operation you 
want to get into from a business perspective.

Skeeve is on the money FWIW, and his list is well worth thinking about. 
  Poorly set up BGP will suck your operational time, result in downtime, 
suck up your free time and suck resources away from other tasks that 
will likely give you more business benefit.

If you don't know how to set this sort of thing up, then it may be good 
use of money to hire someone who knows this field to come in and 
evaluate the options for you, and then assist with the planning and 
installation process and give you some training.  Hint: there's FAR more 
to it than throwing a couple of BGP capable devices into a network and 
typing "router bgp 65000".

Some specific BGP things in no particular to think about specifically:

- What do you want BGP for exactly?  Load sharing, optimal outbound path 
selection?  As a general rule, you probably won't need to run full or 
even partial tables if you are not providing transit to other AS's or 
only have one or two uplinks.

- You've mentioned you have your own AS, have you also got PI address 
space as well?

- Is there a good reason why you can't just get upstream providers to 
handle the BGP for you as part of your connectivity?

- If you still want to run BGP, can you get by with a default only feed 
from two upstreams, and advertise your prefixes from there by tweaking 
AS-Paths on your advertisements to them?  Or two links to one upstream 
and tweaking your MEDs?  This gives you lots of the benefits without 
lots of the costs, and you can run your grand total of 4 or 5 prefixes 
and all forwarded in hardware, even on used Cisco 3550s (notwithstanding 
that they're EOL).  Once you head into full BGP table land you're 
talking about a whole different kettle of fish in regards to memory, 
hardware and platforms.

- If you don't have an offsider or access to someone who either knows 
your network and can take over the support of it when you're not there, 
then complicating it by adding BGP will make that a lot more difficult. 
  Not all network engineers (and not very many server guys) have a good 
working knowledge of BGP.  Keeping things simple makes life a lot easier 
when you're away on your honeymoon for example, because there's a much 
greater chance you won't be called to help because your fill-in can't 
work out what's broken because it's beyond their skill level.  Most 
people can work out static routes, physical port down and no comms 
across ethernet, but not as many can troubleshoot why your load balanced 
BGP upstream feeds are no longer receiving traffic on one of the two 
links and your second link is being smashed with traffic.

=> You may be better off saving money instead of setting up big new 
hardware and spending time getting BGP up and running, and instead put 
the money towards a better upstream (or a second upstream) who will 
handle most or all of the routing intelligence function for you and thus 
give you many more nights of sleep and peace.

I have to admit setting up BGP is a great learning exercise, but that's 
not what most ISPs and businesses want to be burning lots of time on 
unless there's a measurable benefit.

Reuben