[AusNOG] Transparent Caching For Residential ISPs

[Ben]

I think it's mostly about improving users experience now days.  Which is a very difficult
thing to even work out.

It's complicated, because people mostly complain when things don't work, or are unusably slow
and the most frequent web sites people use to test are things like speedtest.net, youtube, google
which don't benefit from http proxy caches.

>From what I understand in general people are seeing 10 to 20% http bandwidth reductions with http
proxy caches depending on their users, the size of their cache, and the algorithams in use.  But if
http traffic is only 20% of total traffic, and you are only seeing savings of 10% then it can be easy
to not see any significance from bandwidth savings.

In order to really benefit from any reduction in bandwidth usage, benefits must be seen when utilisation
is highest, and to web sites that people care about the speed for.  It doesn't really matter how fast
Windows Update goes for instance if it's in the background.

Even without caching there is some benefit to proxying.  Now that Google's initial window size increase
has been in Linux for a significant amount of time, and considering that people upgrade kernels quite
regularly, there are already quite a lof of web sites that use an initial window size of 14600 bytes.  And
Windows still has an inital receive window of 8192 bytes.  Which means a significant number of short HTTP
transactions can be reduced by one round trip time for data retrieved internationally.  As well as this, if
there is any packet loss between the user and the cache, then there can be less reduction in performance,
and changes to utilisation of a limited pipe to the user can be reacted to quicker.  

I think at least now, that most people aren't overloading caches as bad as was done in the past - if there is
overload then highly active sites are just bypassed, as performance is seen as more important than reducing
expense. 

I'm still a fan of transparent proxy caches - but I think they should be non-intrusive, and not guess when
content may be updated based on when they were last modified which was common in the past.  Instead only content
with explicit expiration dates etc should be cached without validation.

I don't know what the situation is ilke in Australia, but in New Zealand quite a few people are using external
DNS servers to watch geo restricted content, nad if proxy has different DNS then this can break these workarounds.

One option is to have opt-out caching, so that if any users think the cache is getting in the way, to allow them
to bypass it to reduce support burden. 

I certainly think the use case has decreased, and I think now days it only really makes sense to use it without
overcommiting bandwidth, in a use case of 3 or more servers so that if one has a problem than there isn't a serious
performance degredation, or on marginal remote links that can't easily be upgraded in bandwidth.  I think the use
case for smaller deployments make the administrative burden prohibital. 

I'm a fan of the trafficserver proxy cache: http://trafficserver.apache.org/  

Ben.

On Wed, Jun 04, 2014 at 06:23:55PM +1000, Michael Marklew wrote:
>    I used transparent caching for many years, but dropped it some 4 years
>    ago. Back in the day it was giving us 30%+ bandwidth savings, but HTTP was
>    most of our traffic. These days it is a much lower percentage with SSL,
>    VPN's & P2P taking up a big share. The additional support load was a pain
>    too with the odd website just not liking the cache and having to be
>    bypassed.
>    With the transit being so much cheaper then it once was I can't see the
>    need anymore. Happy to be corrected and might even put one back in if
>    things have changed?
>    Mikrotik can easily redirect traffic to a proxy & even proxy it itself or
>    use WCCP.
>    Kind Regards,
>    Michael.
>    0249406706
> 
>    On 4 Jun 2014, at 6:05 pm, Jodi Thomson <jodi at team.waspnet.co.nz> wrote:
> 
>      >From: AusNOG [ausnog-bounces at lists.ausnog.net] on behalf of James
>      Mcintosh [james.mcintosh at rocketmail.com]
>      >Sent: Wednesday, 4 June 2014 19:41
>      >To: ausnog at lists.ausnog.net
>      >Subject: [AusNOG] Transparent Caching For Residential ISPs
>      >
>      >
>      >- Worth it?
>      >
>      >- What percentage of your traffic comes off the cache?
>      >
>      >- What solution are you using? Interested in both commercial and open
>      solutions.
>      >
>      >- Any other thoughts or comments
>       
>      Am interested to hear any responses also
>       
>      Cheers
>      JT
>      _______________________________________________
>      AusNOG mailing list
>      AusNOG at lists.ausnog.net
>      http://lists.ausnog.net/mailman/listinfo/ausnog

> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog