[AusNOG] DNS Craziness
Joshua D'Alton
joshua at railgun.com.au
Tue Feb 18 23:23:12 EST 2014
Actually thinking how its only resolving wrong over MP, and knowing a
little how AWS is setup in sydney, I'd definitely be making a ticket as I
just checked route53 and there is nothing I can see that would allow a
different result in this situation, let alone an IP a week old.
On Tue, Feb 18, 2014 at 11:17 PM, Nick @ Deltaband <nick at deltaband.com>wrote:
> For the same reason that if you ask 8.8.8.8 what www.akamai.com is here
> in Aus, it gives you a different answer to the one it gives you if you do
> it in the US. And it's not really anything to do with 8.8.8.8, it's akamai
> answering based on the location of the resolver. It's just that because
> google uses anycast it looks like you're using the same resolver, which
> you're not.
>
> Assuming route53 uses anycast to, it could just be that particular google
> resolver is talking to an outdated/incorrect amazon DNS server (assuming
> it's not a GeoDNS policy/config problem on route53).
>
>
>
>
>
> On 18 February 2014 23:04, Skeeve Stevens <
> skeeve+ausnog at eintellegonetworks.com> wrote:
>
>> Hey Joshua,
>>
>> I was thinking the same... but why would 8.8.8.8 report two different
>> addresses based on the source?
>>
>> Yes, the wrong address is an old address.
>>
>> The DNS changed over a week ago, so should be fine. I do need to check
>> the dates... but even so, why would google report differently from
>> different sources?
>>
>>
>> ...Skeeve
>>
>> *Skeeve Stevens - *eintellego Networks Pty Ltd
>> skeeve at eintellegonetworks.com ; www.eintellegonetworks.com
>>
>> Phone: 1300 239 038; Cell +61 (0)414 753 383 ; skype://skeeve
>>
>> facebook.com/eintellegonetworks ; <http://twitter.com/networkceoau>
>> linkedin.com/in/skeeve
>>
>> twitter.com/theispguy ; blog: www.theispguy.com
>>
>>
>> The Experts Who The Experts Call
>> Juniper - Cisco - Cloud - Consulting - IPv4 Brokering
>>
>>
>> On Tue, Feb 18, 2014 at 10:59 PM, Joshua D'Alton <joshua at railgun.com.au>wrote:
>>
>>> The fact you see the same behaviour directly from AWS shows it is
>>> nothing to do with google, they are merely doing the recursive lookup for
>>> you.
>>>
>>> So I'd be checking the AWS route53 settings. Also, is the wrong IP a
>>> previous IP, if so was it changed recently and therefore maybe some
>>> TTL/caching issue, or if it isn't the correct IP, does the IP
>>> resolve/display some sort of domain squatting site, if so I'd be checking
>>> nameservers and your domain panel incase you've been hijacked (deliberately
>>> or not).
>>>
>>>
>>> On Tue, Feb 18, 2014 at 10:53 PM, Skeeve Stevens <
>>> skeeve+ausnog at eintellegonetworks.com> wrote:
>>>
>>>> That would have to be some seriously specific GeoDNS as they are both
>>>> hitting Google Sydney (I assume).
>>>>
>>>> Respond with blah over MP and blah of iPrimus... I wouldn't think so?
>>>>
>>>>
>>>> ...Skeeve
>>>>
>>>> *Skeeve Stevens - *eintellego Networks Pty Ltd
>>>> skeeve at eintellegonetworks.com ; www.eintellegonetworks.com
>>>>
>>>> Phone: 1300 239 038; Cell +61 (0)414 753 383 ; skype://skeeve
>>>>
>>>> facebook.com/eintellegonetworks ; <http://twitter.com/networkceoau>
>>>> linkedin.com/in/skeeve
>>>>
>>>> twitter.com/theispguy ; blog: www.theispguy.com
>>>>
>>>>
>>>> The Experts Who The Experts Call
>>>> Juniper - Cisco - Cloud - Consulting - IPv4 Brokering
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 10:49 PM, Nick @ Deltaband <nick at deltaband.com>wrote:
>>>>
>>>>> Without seeing the hosts it's hard to know, but Route 53 supports a
>>>>> GeoDNS type service that can provide different answers based on a bunch of
>>>>> different policies inc latency. Could it be that?
>>>>>
>>>>>
>>>>> On 18 February 2014 22:41, Skeeve Stevens <
>>>>> skeeve+ausnog at eintellegonetworks.com> wrote:
>>>>>
>>>>>> Hey all,
>>>>>>
>>>>>> I hope to hell I haven't missed something here, but I have spent a
>>>>>> lot of hours trying to figure out this problem and it is proving a little
>>>>>> hard to nail down.
>>>>>>
>>>>>> Simply, googles public DNS 8.8.8.8 is giving me two different
>>>>>> responses from different peering connections.
>>>>>>
>>>>>> A response from 8.8.8.8 over Megaport MLPA is giving me a different
>>>>>> response to 8.8.8.8 over direct iPrimus -> Google peering.
>>>>>>
>>>>>> The DNS is being sourced from AWS Route53 - so there shouldn't be any
>>>>>> conflicting information.
>>>>>>
>>>>>> There is no round-robin involved because both sides are consistently
>>>>>> returning the same information.
>>>>>>
>>>>>> The correct answer is actually via the iPrimus connection.
>>>>>>
>>>>>> The mind blowing thing is that I get the same 'different' response if
>>>>>> I query the AWS DNS server directly via each connection (as below).
>>>>>>
>>>>>> I hope this makes sense... and that I haven't missed something
>>>>>> obvious. It almost feels like there is some split-view DNS going on here,
>>>>>> but 8.8.8.8 wouldn't do that.
>>>>>>
>>>>>> Alternatively, one of the connections has some sort of transparent
>>>>>> DNS... i.e. the Megaport one.. but it is just going through a Juniper
>>>>>> SRX550 cluster doing NAT into a Juniper MX border going into Megaport -
>>>>>> none of which is doing DNS magic.
>>>>>>
>>>>>> The iPrimus connection is going into Checkpoints as the edge straight
>>>>>> into iPrimus... but they are reporting the correct answer.
>>>>>>
>>>>>> My head hurts :(
>>>>>>
>>>>>>
>>>>>> *VIA MEGAPORT*
>>>>>>
>>>>>> [root at auth01n ~]# traceroute 8.8.8.8
>>>>>> traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
>>>>>> 1 10.x.0.3 (10.x.0.3) 0.382 ms 0.353 ms 0.256 ms
>>>>>> 2 10.x.0.62 (10.x.0.62) 3.370 ms 3.325 ms 3.320 ms
>>>>>> 3 10.x.0.25 (10.x.0.25) 3.314 ms 3.297 ms 3.269 ms
>>>>>> *NAT HERE*
>>>>>> 4 as15169.sydney.megaport.com (103.26.68.56) 3.180 ms 3.142 ms
>>>>>> 3.142 ms
>>>>>> 5 72.14.237.21 (72.14.237.21) 3.125 ms 3.070 ms 3.077 ms
>>>>>> 6 google-public-dns-a.google.com (8.8.8.8) 3.042 ms 2.331 ms
>>>>>> 2.236 ms
>>>>>>
>>>>>>
>>>>>> [root at auth01n ~]# dig a vpn.x.com @8.8.8.8
>>>>>>
>>>>>> ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> a vpn.x.com @
>>>>>> 8.8.8.8
>>>>>> ;; global options: +cmd
>>>>>> ;; Got answer:
>>>>>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24261
>>>>>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
>>>>>>
>>>>>> ;; QUESTION SECTION:
>>>>>> ;vpn.x.com. IN A
>>>>>>
>>>>>> ;; ANSWER SECTION:
>>>>>> vpn.x.com. 299 IN A *x.x.171.124*
>>>>>>
>>>>>> ;; Query time: 166 msec
>>>>>> ;; SERVER: 8.8.8.8#53(8.8.8.8)
>>>>>> ;; WHEN: Tue Feb 18 22:28:24 2014
>>>>>> ;; MSG SIZE rcvd: 48
>>>>>>
>>>>>> =============
>>>>>>
>>>>>> *VIA IPRIMUS/EQUINIX*
>>>>>>
>>>>>> [root at auth01 ~]# traceroute 8.8.8.8
>>>>>> traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
>>>>>> 1 10.x.65.2 (10.x.65.2) 0.476 ms 1.003 ms 1.227 ms
>>>>>> 2 10.x.126.5 (10.x.126.5) 0.471 ms 0.608 ms 0.410 ms
>>>>>> *NAT HERE*
>>>>>> 3 x.x.static.syd.iprimus.net.au (203.134.x.x) 2.253 ms 2.064 ms
>>>>>> 2.075 ms
>>>>>> 4 atm3-2.ac02.syd.iprimus.net.au (203.134.2.249) 1.149 ms 0.768
>>>>>> ms at-1-0-2.ic02.pth.iprimus.net.au (203.134.2.225) 1.107 ms
>>>>>> 5 xe-0-3-0.bsr01.equ.iprimus.net.au (203.134.2.234) 1.313 ms
>>>>>> 0.646 ms xe-1-0-0.bsr01.equ.iprimus.net.au (203.134.2.254) 1.249 ms
>>>>>> 6 google-gw.syd.iprimus.net.au (203.134.2.98) 1.406 ms 1.260 ms
>>>>>> 1.209 ms
>>>>>> 7 72.14.237.21 (72.14.237.21) 1.602 ms 1.871 ms 2.017 ms
>>>>>> 8 google-public-dns-a.google.com (8.8.8.8) 1.333 ms 1.393 ms
>>>>>> 1.403 ms
>>>>>>
>>>>>>
>>>>>> [root at auth01 ~]# dig a vpn.x.com @8.8.8.8
>>>>>>
>>>>>> ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> a vpn.x.com @
>>>>>> 8.8.8.8
>>>>>> ;; global options: +cmd
>>>>>> ;; Got answer:
>>>>>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43084
>>>>>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
>>>>>>
>>>>>> ;; QUESTION SECTION:
>>>>>> ;vpn.x.com. IN A
>>>>>>
>>>>>> ;; ANSWER SECTION:
>>>>>> vpn.x.com. 174 IN A *x.x.102.124*
>>>>>>
>>>>>> ;; Query time: 143 msec
>>>>>> ;; SERVER: 8.8.8.8#53(8.8.8.8)
>>>>>> ;; WHEN: Tue Feb 18 22:30:29 2014
>>>>>> ;; MSG SIZE rcvd: 48
>>>>>>
>>>>>>
>>>>>> ...Skeeve
>>>>>>
>>>>>> *Skeeve Stevens - *eintellego Networks Pty Ltd
>>>>>> skeeve at eintellegonetworks.com ; www.eintellegonetworks.com
>>>>>>
>>>>>> Phone: 1300 239 038; Cell +61 (0)414 753 383 ; skype://skeeve
>>>>>>
>>>>>> facebook.com/eintellegonetworks ; <http://twitter.com/networkceoau>
>>>>>> linkedin.com/in/skeeve
>>>>>>
>>>>>> twitter.com/theispguy ; blog: www.theispguy.com
>>>>>>
>>>>>>
>>>>>> The Experts Who The Experts Call
>>>>>> Juniper - Cisco - Cloud - Consulting - IPv4 Brokering
>>>>>>
>>>>>> _______________________________________________
>>>>>> AusNOG mailing list
>>>>>> AusNOG at lists.ausnog.net
>>>>>> http://lists.ausnog.net/mailman/listinfo/ausnog
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> AusNOG mailing list
>>>> AusNOG at lists.ausnog.net
>>>> http://lists.ausnog.net/mailman/listinfo/ausnog
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20140218/b416218e/attachment.html>
More information about the AusNOG
mailing list