[AusNOG] Cloudflare offline

[Mark Tees]

I guess I would hope that when building something that inspects traffic and then generates attack signatures that if I am generating something that uses packet-length rules then I would validate the output with what is a real maximum.

The DOS kiddie definitely won in this aspect and provided a valuable lesson. Props to Cloudflare for full disclosure.

On 04/03/2013, at 10:39 AM, Jeffrey Sims wrote:

> Perhaps that was the hope Mark, the forged headers said 99971 99985 therefore in a valid attempt to block the attack the rule was placed. However the rule has triggered what appears to be (a yet unconfirmed) bug condition resulting in the routers crashing.
> 
> You start an attack, the provider blocks the attack whilst inadvertently crashing themselves, what more could a dos skipt kiddie want than to watch the network fail without them even trying.
> 
> 
> On Mon, Mar 4, 2013 at 10:36 AM, Mark Tees <marktees at gmail.com> wrote:
> 
> The mentioned on their blog that config walk likely wrong as it was typed out manually. The rule should have been 99971-99985 if it was legitimate.
> 
> But that was never legitimate. Any attack signature that had a packet size larger than what is real shouldn't have made it in. A guess would be that maybe their detection system picked up packets with forged headers.
> 
> On 04/03/2013, at 9:40 AM, Jonathan Thorpe wrote:
> 
> > There appears to be something a little odd about the rule they've installed.
> >
> > +    route 173.X.X.X/32-DNS-DROP {
> > +        match {
> > +            destination 173.X.X.X/32;
> > +            port 53;
> > +            packet-length [ 99971 99985 ];
> > +        }
> > +        then discard;
> > +    }
> >
> > Specifically, they mention "In this case, our attack profiler output the fact that the attack packets were between 99,971 and 99,985 bytes long". Unless there's a typo in the report, the packet-length statement above only includes the two listed.
> >
> > The other thing that looks weird is that IPv4 supports a maximum packet length header of 16-bits (65,535), so how are packet lengths above this possible?
> >
> > Kind Regards,
> > Jonathan
> >
> >
> >
> > From: ausnog-bounces at lists.ausnog.net [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Jeffrey Sims
> > Sent: Monday, 4 March 2013 8:55 AM
> > To: Don Gould
> > Cc: ausnog at lists.ausnog.net
> > Subject: Re: [AusNOG] Cloudflare offline
> >
> > +1 for a great approach by Cloudflare!
> >
> > Agreed, now people know there is a potential bug in JunOS that eegits of the internet might try again (ie: mimik the packet size again to trigger the bug conditions to crash the network).
> >
> > Collaboration is the key!
> >
> > On Mon, Mar 4, 2013 at 8:31 AM, Don Gould <don at bowenvale.co.nz> wrote:
> >
> >
> > On 4/03/2013 10:21 a.m., Jethro Carr wrote:
> > On Mon, 2013-03-04 at 04:37 +1100, Emily Ozols wrote:
> > So... Who's getting fired?
> >
> > <bit of snipping...>
> >
> > the key difference
> > between a competent engineer and an incompetent one is whether they
> > followed processes and handled the disaster in a professional manner.
> >
> > I think a key difference is when people just fess up to a mistake so everyone can learn from it! :)
> >
> > Nothing worse than wondering what caused the stuff up and if you could have been part of the cause.
> >
> > Even worse is making a stuff up that you might not have if you'd just known that someone else had done exactly the same thing the week before but didn't share because they were fearful of not being able to feed their kids the next week.
> >
> > +1 for a great approach by Cloudflare!
> >
> > D
> >
> > --
> > Don Gould
> > 31 Acheson Ave
> > Mairehau
> > Christchurch, New Zealand
> > Ph: + 64 3 348 7235
> > Mobile: + 64 21 114 0699
> >
> >
> > _______________________________________________
> > AusNOG mailing list
> > AusNOG at lists.ausnog.net
> > http://lists.ausnog.net/mailman/listinfo/ausnog
> >
> > _______________________________________________
> > AusNOG mailing list
> > AusNOG at lists.ausnog.net
> > http://lists.ausnog.net/mailman/listinfo/ausnog
> 
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>