<div dir="ltr">Shouldn't be anything in the data path - goes directly from US->AU tunnel router, into core router, and then to the various cpanel and plesk servers - which all seem to be seeing the same problem from *.<a href="http://amazonses.com">amazonses.com</a> servers.<div><br></div><div>Lots of log entries in exim (and similar from mailenable on the plesk boxes) that look like this:</div><div><br></div><div><div>2015-06-03 08:51:59 SMTP connection from [54.240.9.143]:51571 (TCP/IP connection count = 4)</div><div>2015-06-03 08:52:59 SMTP connection from [54.240.27.130]:50131 (TCP/IP connection count = 4)</div><div>2015-06-03 08:54:48 SMTP data timeout (message abandoned) on connection from <a href="http://a9-143.smtp-out.amazonses.com">a9-143.smtp-out.amazonses.com</a> [54.240.9.143]:51571 F=<<a href="mailto:0000014db3aa6e4b-eac45098-5237-4f1e-a8c9-7063fc355f4a-000000@amazonses.com">0000014db3aa6e4b-eac45098-5237-4f1e-a8c9-7063fc355f4a-000000@amazonses.com</a>></div><div>2015-06-03 08:55:49 SMTP data timeout (message abandoned) on connection from <a href="http://a27-130.smtp-out.us-west-2.amazonses.com">a27-130.smtp-out.us-west-2.amazonses.com</a> [54.240.27.130]:50131 F=<<a href="mailto:0000014db6492aa4-3cdf28c6-732e-4fad-aa3d-45b48f026257-000000@us-west-2.amazonses.com">0000014db6492aa4-3cdf28c6-732e-4fad-aa3d-45b48f026257-000000@us-west-2.amazonses.com</a>></div></div><div><br></div><div>Good point on the tcpdump, have started a dump on 54.240/16, will see exactly how far they're getting, and what size packets they're sending (or not sending, as the case may be). Might also do the same from the US router.</div><div><br></div><div>pings and mtr's of various sizes all look as expected. Tracepath was a nice suggestion from one person, haven't seen it before, but it also shows the frag needed packets working correctly. </div><div><br></div><div>Will keep plugging away at it :)</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 3 June 2015 at 10:26, Seamus Ryan <span dir="ltr"><<a href="mailto:s.ryan@uber.com.au" target="_blank">s.ryan@uber.com.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-AU" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Don’t suppose your client has a spam filtering appliance/device/product that is performing packet/protocol inspection on smtp?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">If so, try turning it off temporarily (the packet inspection, not the whole service). Ive seen some nasty issues which sound very similar
to your description caused by smtp protocol inspection. Seems to do more harm than good.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Seamus<u></u><u></u></span></p><span class="">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> AusNOG [mailto:<a href="mailto:ausnog-bounces@lists.ausnog.net" target="_blank">ausnog-bounces@lists.ausnog.net</a>]
<b>On Behalf Of </b>Damien Gardner Jnr<br>
<b>Sent:</b> Wednesday, 3 June 2015 8:44 AM<br>
<b>To:</b> <a href="mailto:ausnog@lists.ausnog.net" target="_blank">ausnog@lists.ausnog.net</a><br>
<b>Subject:</b> [AusNOG] MTU debugging? (Or possibly just a fault with Amazon SES?)<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</span><div>
<p class="MsoNormal">Hi Folks,<u></u><u></u></p><div><div class="h5">
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This one is doing my head in somewhat. I have a customer who needs to receive emails from a body who use Amazon SES in the US to send emails. I can see the connections coming into the customer mailserver, however they then timeout with
no data after connecting.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">If I send various sized pings from amazon and linode instances in the US, they work perfectly up until the point where they hit the MTU of our US->AU tunnel, and then get back a Frag-Needed packet, so that's all working perfectly as expected.
e.g.:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">ubuntu@ip-172-31-4-204:~$ ping -M do -s 1425 <a href="http://plesk03.rendrag.net.au" target="_blank">
plesk03.rendrag.net.au</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">PING <a href="http://plesk03.rendrag.net.au" target="_blank">plesk03.rendrag.net.au</a> (103.235.52.251) 1425(1453) bytes of data.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">From <a href="http://rtr01-e0.lax01.ca.rendrag.net.au" target="_blank">rtr01-e0.lax01.ca.rendrag.net.au</a> (174.136.108.50) icmp_seq=1 Frag needed and DF set (mtu = 1452)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">ping: local error: Message too long, mtu=1452<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">ping: local error: Message too long, mtu=1452<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">ubuntu@ip-172-31-4-204:~$ ping -M do -s 1424 <a href="http://plesk03.rendrag.net.au" target="_blank">
plesk03.rendrag.net.au</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">PING <a href="http://plesk03.rendrag.net.au" target="_blank">plesk03.rendrag.net.au</a> (103.235.52.251) 1424(1452) bytes of data.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">1432 bytes from <a href="http://plesk03.rendrag.net.au" target="_blank">plesk03.rendrag.net.au</a> (103.235.52.251): icmp_seq=1 ttl=111 time=167 ms<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">1432 bytes from <a href="http://plesk03.rendrag.net.au" target="_blank">plesk03.rendrag.net.au</a> (103.235.52.251): icmp_seq=2 ttl=111 time=170 ms<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">As as far as I can see, things are working as they should. However the body using Amazon SES has contacted Amazon support and received a 'This usually signifies an MTU misconfiguration on the remote end, we cannot help with this' reply.
Which leaves me at something of a stalemate..<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Are there any other tests I can run to make sure it's not my issue?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I can pull down files no problems at all with http, torrents, etc. Although one interesting exception is that speedtest does not work - https requests to
<a href="http://c.speedtest.net" target="_blank">c.speedtest.net</a> just block after the initial request with no response until the connection is brought down by RST. (Although that happens in multiple regions in my upstreams' network as well, so I've been assuming it was
a problem with speedtest for the last 6 months..)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Any ideas?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<p>Damien Gardner Jnr<br>
VK2TDG. Dip EE. GradIEAust<br>
<a href="mailto:rendrag@rendrag.net" target="_blank">rendrag@rendrag.net</a> - <a href="http://www.rendrag.net/" target="_blank">http://www.rendrag.net/</a><u><br>
</u>--<br>
We rode on the winds of the rising storm,<br>
We ran to the sounds of thunder.<br>
We danced among the lightning bolts,<br>
and tore the world asunder<u></u><u></u></p>
</div>
</div>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">
<p>Damien Gardner Jnr<br>VK2TDG. Dip EE. GradIEAust<br><a href="mailto:rendrag@rendrag.net" target="_blank">rendrag@rendrag.net</a> - <span><a href="http://www.rendrag.net/" target="_blank">http://www.rendrag.net/</a><u><br></u></span>--<br>We rode on the winds of the rising storm,<br> We ran to the sounds of thunder.<br>We danced among the lightning bolts,<br> and tore the world asunder</p></div></div>
</div>