[AusNOG] DNS Craziness
Skeeve Stevens
skeeve+ausnog at eintellegonetworks.com
Tue Feb 18 23:53:56 EST 2014
Thanks Josh! Will do in the morning.
...Skeeve
*Skeeve Stevens - *eintellego Networks Pty Ltd
skeeve at eintellegonetworks.com ; www.eintellegonetworks.com
Phone: 1300 239 038; Cell +61 (0)414 753 383 ; skype://skeeve
facebook.com/eintellegonetworks ; <http://twitter.com/networkceoau>
linkedin.com/in/skeeve
twitter.com/theispguy ; blog: www.theispguy.com
The Experts Who The Experts Call
Juniper - Cisco - Cloud - Consulting - IPv4 Brokering
On Tue, Feb 18, 2014 at 11:23 PM, Joshua D'Alton <joshua at railgun.com.au>wrote:
> Actually thinking how its only resolving wrong over MP, and knowing a
> little how AWS is setup in sydney, I'd definitely be making a ticket as I
> just checked route53 and there is nothing I can see that would allow a
> different result in this situation, let alone an IP a week old.
>
>
> On Tue, Feb 18, 2014 at 11:17 PM, Nick @ Deltaband <nick at deltaband.com>wrote:
>
>> For the same reason that if you ask 8.8.8.8 what www.akamai.com is here
>> in Aus, it gives you a different answer to the one it gives you if you do
>> it in the US. And it's not really anything to do with 8.8.8.8, it's akamai
>> answering based on the location of the resolver. It's just that because
>> google uses anycast it looks like you're using the same resolver, which
>> you're not.
>>
>> Assuming route53 uses anycast to, it could just be that particular google
>> resolver is talking to an outdated/incorrect amazon DNS server (assuming
>> it's not a GeoDNS policy/config problem on route53).
>>
>>
>>
>>
>>
>> On 18 February 2014 23:04, Skeeve Stevens <
>> skeeve+ausnog at eintellegonetworks.com> wrote:
>>
>>> Hey Joshua,
>>>
>>> I was thinking the same... but why would 8.8.8.8 report two different
>>> addresses based on the source?
>>>
>>> Yes, the wrong address is an old address.
>>>
>>> The DNS changed over a week ago, so should be fine. I do need to check
>>> the dates... but even so, why would google report differently from
>>> different sources?
>>>
>>>
>>> ...Skeeve
>>>
>>> *Skeeve Stevens - *eintellego Networks Pty Ltd
>>> skeeve at eintellegonetworks.com ; www.eintellegonetworks.com
>>>
>>> Phone: 1300 239 038; Cell +61 (0)414 753 383 ; skype://skeeve
>>>
>>> facebook.com/eintellegonetworks ; <http://twitter.com/networkceoau>
>>> linkedin.com/in/skeeve
>>>
>>> twitter.com/theispguy ; blog: www.theispguy.com
>>>
>>>
>>> The Experts Who The Experts Call
>>> Juniper - Cisco - Cloud - Consulting - IPv4 Brokering
>>>
>>>
>>> On Tue, Feb 18, 2014 at 10:59 PM, Joshua D'Alton <joshua at railgun.com.au>wrote:
>>>
>>>> The fact you see the same behaviour directly from AWS shows it is
>>>> nothing to do with google, they are merely doing the recursive lookup for
>>>> you.
>>>>
>>>> So I'd be checking the AWS route53 settings. Also, is the wrong IP a
>>>> previous IP, if so was it changed recently and therefore maybe some
>>>> TTL/caching issue, or if it isn't the correct IP, does the IP
>>>> resolve/display some sort of domain squatting site, if so I'd be checking
>>>> nameservers and your domain panel incase you've been hijacked (deliberately
>>>> or not).
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 10:53 PM, Skeeve Stevens <
>>>> skeeve+ausnog at eintellegonetworks.com> wrote:
>>>>
>>>>> That would have to be some seriously specific GeoDNS as they are both
>>>>> hitting Google Sydney (I assume).
>>>>>
>>>>> Respond with blah over MP and blah of iPrimus... I wouldn't think so?
>>>>>
>>>>>
>>>>> ...Skeeve
>>>>>
>>>>> *Skeeve Stevens - *eintellego Networks Pty Ltd
>>>>> skeeve at eintellegonetworks.com ; www.eintellegonetworks.com
>>>>>
>>>>> Phone: 1300 239 038; Cell +61 (0)414 753 383 ; skype://skeeve
>>>>>
>>>>> facebook.com/eintellegonetworks ; <http://twitter.com/networkceoau>
>>>>> linkedin.com/in/skeeve
>>>>>
>>>>> twitter.com/theispguy ; blog: www.theispguy.com
>>>>>
>>>>>
>>>>> The Experts Who The Experts Call
>>>>> Juniper - Cisco - Cloud - Consulting - IPv4 Brokering
>>>>>
>>>>>
>>>>> On Tue, Feb 18, 2014 at 10:49 PM, Nick @ Deltaband <nick at deltaband.com
>>>>> > wrote:
>>>>>
>>>>>> Without seeing the hosts it's hard to know, but Route 53 supports a
>>>>>> GeoDNS type service that can provide different answers based on a bunch of
>>>>>> different policies inc latency. Could it be that?
>>>>>>
>>>>>>
>>>>>> On 18 February 2014 22:41, Skeeve Stevens <
>>>>>> skeeve+ausnog at eintellegonetworks.com> wrote:
>>>>>>
>>>>>>> Hey all,
>>>>>>>
>>>>>>> I hope to hell I haven't missed something here, but I have spent a
>>>>>>> lot of hours trying to figure out this problem and it is proving a little
>>>>>>> hard to nail down.
>>>>>>>
>>>>>>> Simply, googles public DNS 8.8.8.8 is giving me two different
>>>>>>> responses from different peering connections.
>>>>>>>
>>>>>>> A response from 8.8.8.8 over Megaport MLPA is giving me a different
>>>>>>> response to 8.8.8.8 over direct iPrimus -> Google peering.
>>>>>>>
>>>>>>> The DNS is being sourced from AWS Route53 - so there shouldn't be
>>>>>>> any conflicting information.
>>>>>>>
>>>>>>> There is no round-robin involved because both sides are consistently
>>>>>>> returning the same information.
>>>>>>>
>>>>>>> The correct answer is actually via the iPrimus connection.
>>>>>>>
>>>>>>> The mind blowing thing is that I get the same 'different' response
>>>>>>> if I query the AWS DNS server directly via each connection (as below).
>>>>>>>
>>>>>>> I hope this makes sense... and that I haven't missed something
>>>>>>> obvious. It almost feels like there is some split-view DNS going on here,
>>>>>>> but 8.8.8.8 wouldn't do that.
>>>>>>>
>>>>>>> Alternatively, one of the connections has some sort of transparent
>>>>>>> DNS... i.e. the Megaport one.. but it is just going through a Juniper
>>>>>>> SRX550 cluster doing NAT into a Juniper MX border going into Megaport -
>>>>>>> none of which is doing DNS magic.
>>>>>>>
>>>>>>> The iPrimus connection is going into Checkpoints as the edge
>>>>>>> straight into iPrimus... but they are reporting the correct answer.
>>>>>>>
>>>>>>> My head hurts :(
>>>>>>>
>>>>>>>
>>>>>>> *VIA MEGAPORT*
>>>>>>>
>>>>>>> [root at auth01n ~]# traceroute 8.8.8.8
>>>>>>> traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
>>>>>>> 1 10.x.0.3 (10.x.0.3) 0.382 ms 0.353 ms 0.256 ms
>>>>>>> 2 10.x.0.62 (10.x.0.62) 3.370 ms 3.325 ms 3.320 ms
>>>>>>> 3 10.x.0.25 (10.x.0.25) 3.314 ms 3.297 ms 3.269 ms
>>>>>>> *NAT HERE*
>>>>>>> 4 as15169.sydney.megaport.com (103.26.68.56) 3.180 ms 3.142 ms
>>>>>>> 3.142 ms
>>>>>>> 5 72.14.237.21 (72.14.237.21) 3.125 ms 3.070 ms 3.077 ms
>>>>>>> 6 google-public-dns-a.google.com (8.8.8.8) 3.042 ms 2.331 ms
>>>>>>> 2.236 ms
>>>>>>>
>>>>>>>
>>>>>>> [root at auth01n ~]# dig a vpn.x.com @8.8.8.8
>>>>>>>
>>>>>>> ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> a vpn.x.com @
>>>>>>> 8.8.8.8
>>>>>>> ;; global options: +cmd
>>>>>>> ;; Got answer:
>>>>>>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24261
>>>>>>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
>>>>>>>
>>>>>>> ;; QUESTION SECTION:
>>>>>>> ;vpn.x.com. IN A
>>>>>>>
>>>>>>> ;; ANSWER SECTION:
>>>>>>> vpn.x.com. 299 IN A *x.x.171.124*
>>>>>>>
>>>>>>> ;; Query time: 166 msec
>>>>>>> ;; SERVER: 8.8.8.8#53(8.8.8.8)
>>>>>>> ;; WHEN: Tue Feb 18 22:28:24 2014
>>>>>>> ;; MSG SIZE rcvd: 48
>>>>>>>
>>>>>>> =============
>>>>>>>
>>>>>>> *VIA IPRIMUS/EQUINIX*
>>>>>>>
>>>>>>> [root at auth01 ~]# traceroute 8.8.8.8
>>>>>>> traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
>>>>>>> 1 10.x.65.2 (10.x.65.2) 0.476 ms 1.003 ms 1.227 ms
>>>>>>> 2 10.x.126.5 (10.x.126.5) 0.471 ms 0.608 ms 0.410 ms
>>>>>>> *NAT HERE*
>>>>>>> 3 x.x.static.syd.iprimus.net.au (203.134.x.x) 2.253 ms 2.064 ms
>>>>>>> 2.075 ms
>>>>>>> 4 atm3-2.ac02.syd.iprimus.net.au (203.134.2.249) 1.149 ms 0.768
>>>>>>> ms at-1-0-2.ic02.pth.iprimus.net.au (203.134.2.225) 1.107 ms
>>>>>>> 5 xe-0-3-0.bsr01.equ.iprimus.net.au (203.134.2.234) 1.313 ms
>>>>>>> 0.646 ms xe-1-0-0.bsr01.equ.iprimus.net.au (203.134.2.254) 1.249
>>>>>>> ms
>>>>>>> 6 google-gw.syd.iprimus.net.au (203.134.2.98) 1.406 ms 1.260 ms
>>>>>>> 1.209 ms
>>>>>>> 7 72.14.237.21 (72.14.237.21) 1.602 ms 1.871 ms 2.017 ms
>>>>>>> 8 google-public-dns-a.google.com (8.8.8.8) 1.333 ms 1.393 ms
>>>>>>> 1.403 ms
>>>>>>>
>>>>>>>
>>>>>>> [root at auth01 ~]# dig a vpn.x.com @8.8.8.8
>>>>>>>
>>>>>>> ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> a vpn.x.com @
>>>>>>> 8.8.8.8
>>>>>>> ;; global options: +cmd
>>>>>>> ;; Got answer:
>>>>>>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43084
>>>>>>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
>>>>>>>
>>>>>>> ;; QUESTION SECTION:
>>>>>>> ;vpn.x.com. IN A
>>>>>>>
>>>>>>> ;; ANSWER SECTION:
>>>>>>> vpn.x.com. 174 IN A *x.x.102.124*
>>>>>>>
>>>>>>> ;; Query time: 143 msec
>>>>>>> ;; SERVER: 8.8.8.8#53(8.8.8.8)
>>>>>>> ;; WHEN: Tue Feb 18 22:30:29 2014
>>>>>>> ;; MSG SIZE rcvd: 48
>>>>>>>
>>>>>>>
>>>>>>> ...Skeeve
>>>>>>>
>>>>>>> *Skeeve Stevens - *eintellego Networks Pty Ltd
>>>>>>> skeeve at eintellegonetworks.com ; www.eintellegonetworks.com
>>>>>>>
>>>>>>> Phone: 1300 239 038; Cell +61 (0)414 753 383 ; skype://skeeve
>>>>>>>
>>>>>>> facebook.com/eintellegonetworks ; <http://twitter.com/networkceoau>
>>>>>>> linkedin.com/in/skeeve
>>>>>>>
>>>>>>> twitter.com/theispguy ; blog: www.theispguy.com
>>>>>>>
>>>>>>>
>>>>>>> The Experts Who The Experts Call
>>>>>>> Juniper - Cisco - Cloud - Consulting - IPv4 Brokering
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> AusNOG mailing list
>>>>>>> AusNOG at lists.ausnog.net
>>>>>>> http://lists.ausnog.net/mailman/listinfo/ausnog
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> AusNOG mailing list
>>>>> AusNOG at lists.ausnog.net
>>>>> http://lists.ausnog.net/mailman/listinfo/ausnog
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20140218/ea87fa71/attachment.html>
More information about the AusNOG
mailing list