[AusNOG] DDoS Attacks - Painful and Persistent.

[Roland Dobbins]

On Aug 10, 2009, at 1:14 PM, Nick Brown wrote:

> Our ability to blackhole source addresses has been hindered by the  
> number of source, and the dynamic nature of where the attack traffic  
> is originating from.

S/RTBH plus NetFlow is well-suited to dynamic sources - watch the  
attack traffic, and add/remove blackhole routes at will.  It's a  
highly effective technique for use in such situations, FYI.

> We are working through the data we have got - however for a  
> significant portion of the attack time, our focus has been on  
> restoring services or mitigating the effect the attack has on  
> downstream customers.

Difficult to do this without the ability to classify and ensure one  
has a complete view of the traffic.  Strongly suggest grabbing an open- 
source NetFlow collection/graphing tool and getting NDE going ASAP.   
NetFlow will vastly reduce the volume of data you must collect and  
sort, yet provide you with an accurate picture of the traffic.

> We have managed to mitigate the impact the attack has had by  
> migrating services to alternate locations, however at the same time  
> we have been mindful not to widen our surface area.

The attacker may end up simply following the targets, if he's at all  
persistent.  Good point about not broadening the attack surface!

>  We have looked at commercial third party options, however to date  
> the cost has significantly outweighed simply throwing more capacity  
> at the problem.

It's important to keep in mind that if they're at all serious, the  
attackers are likely to have far more capacity at their disposal than  
you do, unfortunately.

> In the process of this based on a couple recomendations right now. I  
> have also been contacted by both ACMA and AusCERT representatives,  
> and we are only too happy to share information with those who  
> believe it to be relevant in either mitigating the effect on  
> ourselves, or the greater Internet community.

Strongly suggest you reach out to your network infrastructure vendor  
security teams (Cisco PSIRT, Juniper JSIRT, etc.) for their assistance  
both with attack mitigation as well as with providing contacts within  
the large operational security community.

> There is a 3 Minute capture (15MB) available for viewing at http://mirror.as38887.net/Misc/Attack_2009-08-10_202.45.155.46.txt 
>  as captured earlier this morning showing some data pertaining to  
> the type and volume of traffic. Despite dropping the affected  
> prefixes earlier today, bringing the affected prefix back into the  
> global routing table immediately shows the return of the malicious  
> traffic.

Based upon the basic header information provided at the linked URL,  
this may be a fragmented UDP attack (attackers will use large packet  
sizes with out-of-order IP IDs in order to cause additional interrupt  
processing at the end-host).

Does anything live at udp/7575 on this host, or is this just a script- 
kiddie choosing a random port?

> Anyone who wants specific information on how we have and are  
> mitigating this attack so far are welcome to contact me offlist for  
> more info.

Strongly suggest you work with your upstreams/peers and the mitigation  
communities to get this attack pushed back towards the actual bot  
source IPs, that you implement NetFlow export and S/RTBH, and look at  
QPPB, as well, if your platform is capable.

Good luck with this - feel free to contact me 1:1 for more detailed/ 
specific discussions, if you like!

-----------------------------------------------------------------------
Roland Dobbins <rdobbins at arbor.net> // <http://www.arbornetworks.com>

         Unfortunately, inefficiency scales really well.

		   -- Kevin Lawton