<div dir="ltr">Thanks Josh! Will do in the morning.</div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><div><br>...Skeeve</div><div><br></div><div><div><b style="font-size:13px;font-family:Calibri">Skeeve Stevens - </b><span style="font-size:13px;font-family:Calibri">eintellego Networks Pty Ltd</span></div>
<div><div><span style="font-family:Calibri;font-size:13px"><a href="mailto:skeeve@eintellegonetworks.com" target="_blank">skeeve@eintellegonetworks.com</a> ; <a href="http://www.eintellegonetworks.com/" target="_blank">www.eintellegonetworks.com</a></span><font><p style="font-family:Calibri;font-size:13px;margin:0px">
Phone: 1300 239 038; Cell +61 (0)414 753 383 ; <a>skype://skeeve</a></p><p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://facebook.com/eintellegonetworks" target="_blank">facebook.com/eintellegonetworks</a> ; <a href="http://twitter.com/networkceoau" target="_blank"></a><a href="http://linkedin.com/in/skeeve" target="_blank">linkedin.com/in/skeeve</a> </p>
<p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://twitter.com/theispguy" target="_blank">twitter.com/theispguy</a><span style="color:rgb(0,0,0)"> ; blog: </span><a href="http://www.theispguy.com/" target="_blank">www.theispguy.com</a><br>
</p><p style="font-family:Calibri;font-size:13px;margin:0px"><img src="http://eintellegonetworks.com/logos/ein09.png"><br></p><p style="margin:0px"><span style="color:rgb(127,0,127);font-family:Calibri,sans-serif;font-size:13px">The Experts Who The Experts Call</span></p>
</font></div><div style="font-family:Calibri,sans-serif;font-size:14px;color:rgb(127,0,127)"><span style="color:rgb(0,32,96);font-size:13px">Juniper - Cisco </span><span style="color:rgb(0,32,96);font-size:13px">- Cloud</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- Consulting</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- IPv4 Brokering</span></div>
</div></div></div></div>
<br><br><div class="gmail_quote">On Tue, Feb 18, 2014 at 11:23 PM, Joshua D'Alton <span dir="ltr"><<a href="mailto:joshua@railgun.com.au" target="_blank">joshua@railgun.com.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Actually thinking how its only resolving wrong over MP, and knowing a little how AWS is setup in sydney, I'd definitely be making a ticket as I just checked route53 and there is nothing I can see that would allow a different result in this situation, let alone an IP a week old.</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Feb 18, 2014 at 11:17 PM, Nick @ Deltaband <span dir="ltr"><<a href="mailto:nick@deltaband.com" target="_blank">nick@deltaband.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>For the same reason that if you ask 8.8.8.8 what <a href="http://www.akamai.com" target="_blank">www.akamai.com</a> is here in Aus, it gives you a different answer to the one it gives you if you do it in the US. And it's not really anything to do with 8.8.8.8, it's akamai answering based on the location of the resolver. It's just that because google uses anycast it looks like you're using the same resolver, which you're not. <br>
</div><div><br></div><div>Assuming route53 uses anycast to, it could just be that particular google resolver is talking to an outdated/incorrect amazon DNS server (assuming it's not a GeoDNS policy/config problem on route53).</div>
<div><br></div><div><br></div><div><div><br></div></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On 18 February 2014 23:04, Skeeve Stevens <span dir="ltr"><<a href="mailto:skeeve+ausnog@eintellegonetworks.com" target="_blank">skeeve+ausnog@eintellegonetworks.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hey Joshua,<div><br></div><div>I was thinking the same... but why would 8.8.8.8 report two different addresses based on the source?</div>
<div><br></div><div>Yes, the wrong address is an old address.</div><div>
<br></div><div>The DNS changed over a week ago, so should be fine. I do need to check the dates... but even so, why would google report differently from different sources?</div></div><div class="gmail_extra"><div>
<br clear="all">
<div><div dir="ltr"><div><br>...Skeeve</div><div><br></div><div><div><b style="font-size:13px;font-family:Calibri">Skeeve Stevens - </b><span style="font-size:13px;font-family:Calibri">eintellego Networks Pty Ltd</span></div>
<div><div><span style="font-family:Calibri;font-size:13px"><a href="mailto:skeeve@eintellegonetworks.com" target="_blank">skeeve@eintellegonetworks.com</a> ; <a href="http://www.eintellegonetworks.com/" target="_blank">www.eintellegonetworks.com</a></span><font><p style="font-family:Calibri;font-size:13px;margin:0px">
Phone: <a href="tel:1300%20239%20038" value="+611300239038" target="_blank">1300 239 038</a>; Cell <a href="tel:%2B61%20%280%29414%20753%20383" value="+61414753383" target="_blank">+61 (0)414 753 383</a> ; <a>skype://skeeve</a></p>
<p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://facebook.com/eintellegonetworks" target="_blank">facebook.com/eintellegonetworks</a> ; <a href="http://twitter.com/networkceoau" target="_blank"></a><a href="http://linkedin.com/in/skeeve" target="_blank">linkedin.com/in/skeeve</a> </p>
<p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://twitter.com/theispguy" target="_blank">twitter.com/theispguy</a><span> ; blog: </span><a href="http://www.theispguy.com/" target="_blank">www.theispguy.com</a><br>
</p><p style="font-family:Calibri;font-size:13px;margin:0px"><img src="http://eintellegonetworks.com/logos/ein09.png"><br></p><p style="margin:0px"><span style="color:rgb(127,0,127);font-family:Calibri,sans-serif;font-size:13px">The Experts Who The Experts Call</span></p>
</font></div><div style="font-family:Calibri,sans-serif;font-size:14px;color:rgb(127,0,127)"><span style="color:rgb(0,32,96);font-size:13px">Juniper - Cisco </span><span style="color:rgb(0,32,96);font-size:13px">- Cloud</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- Consulting</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- IPv4 Brokering</span></div>
</div></div></div></div>
<br><br></div><div><div><div class="gmail_quote">On Tue, Feb 18, 2014 at 10:59 PM, Joshua D'Alton <span dir="ltr"><<a href="mailto:joshua@railgun.com.au" target="_blank">joshua@railgun.com.au</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">The fact you see the same behaviour directly from AWS shows it is nothing to do with google, they are merely doing the recursive lookup for you.<div><br></div><div>So I'd be checking the AWS route53 settings. Also, is the wrong IP a previous IP, if so was it changed recently and therefore maybe some TTL/caching issue, or if it isn't the correct IP, does the IP resolve/display some sort of domain squatting site, if so I'd be checking nameservers and your domain panel incase you've been hijacked (deliberately or not).</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Feb 18, 2014 at 10:53 PM, Skeeve Stevens <span dir="ltr"><<a href="mailto:skeeve+ausnog@eintellegonetworks.com" target="_blank">skeeve+ausnog@eintellegonetworks.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">That would have to be some seriously specific GeoDNS as they are both hitting Google Sydney (I assume).<div>
<br></div><div>Respond with blah over MP and blah of iPrimus... I wouldn't think so?</div></div>
<div class="gmail_extra"><div><br clear="all"><div><div dir="ltr"><div><br>...Skeeve</div><div><br></div><div><div><b style="font-size:13px;font-family:Calibri">Skeeve Stevens - </b><span style="font-size:13px;font-family:Calibri">eintellego Networks Pty Ltd</span></div>
<div><div><span style="font-family:Calibri;font-size:13px"><a href="mailto:skeeve@eintellegonetworks.com" target="_blank">skeeve@eintellegonetworks.com</a> ; <a href="http://www.eintellegonetworks.com/" target="_blank">www.eintellegonetworks.com</a></span><font><p style="font-family:Calibri;font-size:13px;margin:0px">
Phone: <a href="tel:1300%20239%20038" value="+611300239038" target="_blank">1300 239 038</a>; Cell <a href="tel:%2B61%20%280%29414%20753%20383" value="+61414753383" target="_blank">+61 (0)414 753 383</a> ; <a>skype://skeeve</a></p>
<p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://facebook.com/eintellegonetworks" target="_blank">facebook.com/eintellegonetworks</a> ; <a href="http://twitter.com/networkceoau" target="_blank"></a><a href="http://linkedin.com/in/skeeve" target="_blank">linkedin.com/in/skeeve</a> </p>
<p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://twitter.com/theispguy" target="_blank">twitter.com/theispguy</a><span> ; blog: </span><a href="http://www.theispguy.com/" target="_blank">www.theispguy.com</a><br>
</p><p style="font-family:Calibri;font-size:13px;margin:0px"><img src="http://eintellegonetworks.com/logos/ein09.png"><br></p><p style="margin:0px"><span style="color:rgb(127,0,127);font-family:Calibri,sans-serif;font-size:13px">The Experts Who The Experts Call</span></p>
</font></div><div style="font-family:Calibri,sans-serif;font-size:14px;color:rgb(127,0,127)"><span style="color:rgb(0,32,96);font-size:13px">Juniper - Cisco </span><span style="color:rgb(0,32,96);font-size:13px">- Cloud</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- Consulting</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- IPv4 Brokering</span></div>
</div></div></div></div>
<br><br></div><div><div><div class="gmail_quote">On Tue, Feb 18, 2014 at 10:49 PM, Nick @ Deltaband <span dir="ltr"><<a href="mailto:nick@deltaband.com" target="_blank">nick@deltaband.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Without seeing the hosts it's hard to know, but Route 53 supports a GeoDNS type service that can provide different answers based on a bunch of different policies inc latency. Could it be that? </div><div class="gmail_extra">
<br><br><div class="gmail_quote">On 18 February 2014 22:41, Skeeve Stevens <span dir="ltr"><<a href="mailto:skeeve+ausnog@eintellegonetworks.com" target="_blank">skeeve+ausnog@eintellegonetworks.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hey all,<div><br></div><div>I hope to hell I haven't missed something here, but I have spent a lot of hours trying to figure out this problem and it is proving a little hard to nail down.</div>
<div><br></div>
<div>Simply, googles public DNS 8.8.8.8 is giving me two different responses from different peering connections.</div><div><br></div><div>A response from 8.8.8.8 over Megaport MLPA is giving me a different response to 8.8.8.8 over direct iPrimus -> Google peering.</div>
<div><br></div><div>The DNS is being sourced from AWS Route53 - so there shouldn't be any conflicting information.</div><div><br></div><div>There is no round-robin involved because both sides are consistently returning the same information.</div>
<div><br></div><div>The correct answer is actually via the iPrimus connection.</div><div><br></div><div>The mind blowing thing is that I get the same 'different' response if I query the AWS DNS server directly via each connection (as below).</div>
<div><br></div><div>I hope this makes sense... and that I haven't missed something obvious. It almost feels like there is some split-view DNS going on here, but 8.8.8.8 wouldn't do that.</div><div><br></div><div>
Alternatively, one of the connections has some sort of transparent DNS... i.e. the Megaport one.. but it is just going through a Juniper SRX550 cluster doing NAT into a Juniper MX border going into Megaport - none of which is doing DNS magic.</div>
<div><br></div><div>The iPrimus connection is going into Checkpoints as the edge straight into iPrimus... but they are reporting the correct answer.</div><div><br></div><div>My head hurts :(</div><div><br></div><div><br>
</div>
<div><b>VIA MEGAPORT</b></div><div><br></div><div><div>[root@auth01n ~]# traceroute 8.8.8.8</div><div>traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets</div><div> 1 10.x.0.3 (10.x.0.3) 0.382 ms 0.353 ms 0.256 ms</div>
<div> 2 10.x.0.62 (10.x.0.62) 3.370 ms 3.325 ms 3.320 ms</div><div> 3 10.x.0.25 (10.x.0.25) 3.314 ms 3.297 ms 3.269 ms</div><div><b>NAT HERE</b></div><div> 4 <a href="http://as15169.sydney.megaport.com" target="_blank">as15169.sydney.megaport.com</a> (103.26.68.56) 3.180 ms 3.142 ms 3.142 ms</div>
<div> 5 72.14.237.21 (72.14.237.21) 3.125 ms 3.070 ms 3.077 ms</div><div> 6 <a href="http://google-public-dns-a.google.com" target="_blank">google-public-dns-a.google.com</a> (8.8.8.8) 3.042 ms 2.331 ms 2.236 ms</div>
<div><br></div>
<div><br></div><div>[root@auth01n ~]# dig a <a href="http://vpn.x.com" target="_blank">vpn.x.com</a> @<a href="http://8.8.8.8" target="_blank">8.8.8.8</a> </div><div><br></div><div>; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> a <a href="http://vpn.x.com" target="_blank">vpn.x.com</a> @<a href="http://8.8.8.8" target="_blank">8.8.8.8</a></div>
<div>;; global options: +cmd</div><div>;; Got answer:</div><div>;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24261</div><div>;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0</div><div>
<br></div><div>;; QUESTION SECTION:</div><div>;<a href="http://vpn.x.com" target="_blank">vpn.x.com</a>. IN A</div><div><br></div><div>;; ANSWER SECTION:</div><div><a href="http://vpn.x.com" target="_blank">vpn.x.com</a>. 299 IN A <b>x.x.171.124</b></div>
<div><br></div><div>;; Query time: 166 msec</div><div>;; SERVER: 8.8.8.8#53(8.8.8.8)</div><div>;; WHEN: Tue Feb 18 22:28:24 2014</div><div>;; MSG SIZE rcvd: 48</div></div><div><br></div><div>=============</div><div><br>
</div>
<div><b>VIA IPRIMUS/EQUINIX</b></div><div><br></div><div><div>[root@auth01 ~]# traceroute 8.8.8.8</div><div>traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets</div><div> 1 10.x.65.2 (10.x.65.2) 0.476 ms 1.003 ms 1.227 ms</div>
<div> 2 10.x.126.5 (10.x.126.5) 0.471 ms 0.608 ms 0.410 ms</div><div><b>NAT HERE</b></div><div> 3 <a href="http://x.x.static.syd.iprimus.net.au" target="_blank">x.x.static.syd.iprimus.net.au</a> (203.134.x.x) 2.253 ms 2.064 ms 2.075 ms</div>
<div> 4 <a href="http://atm3-2.ac02.syd.iprimus.net.au" target="_blank">atm3-2.ac02.syd.iprimus.net.au</a> (203.134.2.249) 1.149 ms 0.768 ms <a href="http://at-1-0-2.ic02.pth.iprimus.net.au" target="_blank">at-1-0-2.ic02.pth.iprimus.net.au</a> (203.134.2.225) 1.107 ms</div>
<div> 5 <a href="http://xe-0-3-0.bsr01.equ.iprimus.net.au" target="_blank">xe-0-3-0.bsr01.equ.iprimus.net.au</a> (203.134.2.234) 1.313 ms 0.646 ms <a href="http://xe-1-0-0.bsr01.equ.iprimus.net.au" target="_blank">xe-1-0-0.bsr01.equ.iprimus.net.au</a> (203.134.2.254) 1.249 ms</div>
<div> 6 <a href="http://google-gw.syd.iprimus.net.au" target="_blank">google-gw.syd.iprimus.net.au</a> (203.134.2.98) 1.406 ms 1.260 ms 1.209 ms</div><div> 7 72.14.237.21 (72.14.237.21) 1.602 ms 1.871 ms 2.017 ms</div>
<div> 8 <a href="http://google-public-dns-a.google.com" target="_blank">google-public-dns-a.google.com</a> (8.8.8.8) 1.333 ms 1.393 ms 1.403 ms</div>
<div><br></div><div><br></div><div>[root@auth01 ~]# dig a <a href="http://vpn.x.com" target="_blank">vpn.x.com</a> @<a href="http://8.8.8.8" target="_blank">8.8.8.8</a> </div><div><br></div><div>; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> a <a href="http://vpn.x.com" target="_blank">vpn.x.com</a> @<a href="http://8.8.8.8" target="_blank">8.8.8.8</a></div>
<div>;; global options: +cmd</div><div>;; Got answer:</div><div>;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43084</div><div>;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0</div><div>
<br></div><div>;; QUESTION SECTION:</div><div>;<a href="http://vpn.x.com" target="_blank">vpn.x.com</a>. IN A</div><div><br></div><div>;; ANSWER SECTION:</div><div><a href="http://vpn.x.com" target="_blank">vpn.x.com</a>. 174 IN A <b>x.x.102.124</b></div>
<div><br></div><div>;; Query time: 143 msec</div><div>;; SERVER: 8.8.8.8#53(8.8.8.8)</div><div>;; WHEN: Tue Feb 18 22:30:29 2014</div><div>;; MSG SIZE rcvd: 48</div></div><div><br clear="all"><div><div dir="ltr"><div><br>
...Skeeve</div><div><br></div><div><div><b style="font-size:13px;font-family:Calibri">Skeeve Stevens - </b><span style="font-size:13px;font-family:Calibri">eintellego Networks Pty Ltd</span></div><div><div><span style="font-family:Calibri;font-size:13px"><a href="mailto:skeeve@eintellegonetworks.com" target="_blank">skeeve@eintellegonetworks.com</a> ; <a href="http://www.eintellegonetworks.com/" target="_blank">www.eintellegonetworks.com</a></span><font><p style="font-family:Calibri;font-size:13px;margin:0px">
Phone: <a href="tel:1300%20239%20038" value="+611300239038" target="_blank">1300 239 038</a>; Cell <a href="tel:%2B61%20%280%29414%20753%20383" value="+61414753383" target="_blank">+61 (0)414 753 383</a> ; <a>skype://skeeve</a></p>
<p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://facebook.com/eintellegonetworks" target="_blank">facebook.com/eintellegonetworks</a> ; <a href="http://twitter.com/networkceoau" target="_blank"></a><a href="http://linkedin.com/in/skeeve" target="_blank">linkedin.com/in/skeeve</a> </p>
<p style="font-family:Calibri;font-size:13px;margin:0px"><a href="http://twitter.com/theispguy" target="_blank">twitter.com/theispguy</a><span> ; blog: </span><a href="http://www.theispguy.com/" target="_blank">www.theispguy.com</a><br>
</p><p style="font-family:Calibri;font-size:13px;margin:0px"><img src="http://eintellegonetworks.com/logos/ein09.png"><br></p><p style="margin:0px"><span style="color:rgb(127,0,127);font-family:Calibri,sans-serif;font-size:13px">The Experts Who The Experts Call</span></p>
</font></div><div style="font-family:Calibri,sans-serif;font-size:14px;color:rgb(127,0,127)"><span style="color:rgb(0,32,96);font-size:13px">Juniper - Cisco </span><span style="color:rgb(0,32,96);font-size:13px">- Cloud</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- Consulting</span><span style="color:rgb(0,32,96);font-size:13px"> </span><span style="color:rgb(0,32,96);font-size:13px">- IPv4 Brokering</span></div>
</div></div></div></div>
</div></div>
<br>_______________________________________________<br>
AusNOG mailing list<br>
<a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
<a href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div></div></div>
<br>_______________________________________________<br>
AusNOG mailing list<br>
<a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
<a href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</blockquote></div><br></div>