[AusNOG] Write up - Big ISP, little ISP, local internet exchanges

[Edwin Groothuis]

Hello Alex,

On Fri, Sep 05, 2008 at 12:40:33PM +1000, Campbell, Alex wrote:
> There was a very interesting NANOG presentation on these issues a few years ago:
> http://www.dtdigital.com.au/pdf/scholl-peering-dragnet.pdf
> (it seems to have recently disappeared from the NANOG site so I have temporarily re-posted it)

That is a nice presentation.

> > Take an uplink which is also on the local internet exchange, advertise 
> > your /24s to the local internet exchange and your big /21 to the uplink 
> > provider. Why? Because the uplink provider will advertise your /21 to 
> > the rest of the internet, while it will internally route it via the /24s to the 
> > local internet exchange. Free inbound traffic! And if your port on the 
> > local internet exchange is 100Mbps or 1000Mbps and the link towards 
> > your uplink provider is less than 100Mbps, you will have a nice extra 
> > speed increase with it too.
> 
> Making troubleshooting more difficult and creating unnecessary
> routing asymmetry just to save a few bucks on inbound traffic seems
> like a false economy to me (but perhaps our economic priorities are
> different to others).

False economies are a known issue in the networking world. Think
cable modems which are sold as 1.5Mbps downstream each while the
head-end for all 100 of them them has aggregates them into a single
100Mbps connection. It's people who charge for traffic which came
from the proxy server instead of from the website.

> > Some providers run their accounting systems based on all the IP 
> > traffic going through their edge routers and will bill you for the 
> > traffic even if it doesn't go over the physical wire. Check your terms 
> > and conditions to see what you can do about this Layer 8 behaviour.
> 
> The blog post seems to be suggesting that you could use the terms
> and conditions to legitimise stealing free traffic from your provider.

There is a difference between stealing and getting something) and
getting the stuff thrown in your lap: The transit provider routes
it to the wrong hand-off point. You have an agreement with them for
a certain link, and they don't deliver it to there.

> Providers that do their traffic accounting at their border have
> already solved this problem.  We used to buy transit from Uecomm
> and also peer with them at an MLPA exchange - they didn't care how
> the last-hop traffic got to us but they would charge us for it
> either way (which I didn't have any issues with).

We had (when I was working there) a similar issue with Uecomm about
this, except that it was a spare-backup link (yes, a spare-backup
link, contractual issues) which normally didn't have a BGP session
established. The moment Uecomm joined the PIPE IX in Sydney we got
bills for traffic, because they expected traffic from our IP space
to be send via that link and not to go via the internet exchange.
Three times we send back this bill to our account manager, stating
that there had gone no traffic over the link we had an agreement
with. The fourth month it didn't show up anymore.

If Uecomm was your only or main link provider, then I agree with
you. But knowing how difficult and expensive it is to get a change
into the contracts or into the configuration of your links, I would
make sure that you use all the methods you have to make sure they
only bill what is valid according to the contract.

> As the presentation above suggests, careful logging and/or
> rate-limiting of suspicious IX traffic can detect / prevent most
> of the common bandwidth stealing techniques.

The presentation also states the "before the act" and "after the
act". Before the act is what I would do:

> I can't see many (any) big providers splitting up their network
> and adding a lot of complexity to solve a non-problem.

Network redesigns are not done over a weekend but take longer. The
first one is often from a "wildly grown from the beginning of the
network era" network into a properly designed and predictable
network. The next redesign is because of the growth and experiences
learned from the properly designed and predictable network.

The issue I described, from a transit provider (which I am not)
point of view, is what I would put into the "next redesign" category.

Edwin

-- 
Edwin Groothuis      |            Personal website: http://www.mavetju.org
edwin at mavetju.org    |              Weblog: http://www.mavetju.org/weblog/