[AusNOG] Sensible MTU For Core Routers

Sun Apr 19 18:00:39 EST 2015

Determine your core's maximum MTU based on hardware, and enforce that at
the distribution layer. Enforcing MTU at the edge is a crap shoot at best,
and counter productive at worst.

Paul Wilkins

On 19 April 2015 at 13:15, Mark ZZZ Smith <markzzzsmith at yahoo.com.au> wrote:

> Another one that comes to mind,
>
> - Don't assuming that a inter-router or PE-CE protocol validates or tests
> that MTUs match or are sufficient. EIGRP doesn't check MTUs when it comes
> up, and if I recall correctly it isn't required of IS-IS either.
>
> I don't know if the default has changed, however a while back TCP
> connections from a Cisco router used the minimum maximum TCP segment size
> of 576 octets and didn't use the TCP MSS option. That included for BGP TCP
> connections. This is a safe choice, however it means that BGP TCP
> connections didn't take advantage of much larger interface MTUs to reduce
> BGP establishment times, and also wouldn't uncover mismatched MTUs on both
> ends of the link. So BGP coming up wasn't as much of an absolute indication
> of that the link was fully and properly configured. If I recall correctly,
> the command to enable full utilisation of link MTU for BGP and other TCP
> connections is the global 'ip tcp path-mtu-discovery' command.
>
>   ------------------------------
>  *From:* Mark ZZZ Smith <markzzzsmith at yahoo.com.au>
> *To:* James Mcintosh <james.mcintosh at rocketmail.com>; "
> ausnog at lists.ausnog.net" <ausnog at lists.ausnog.net>
> *Sent:* Sunday, 19 April 2015, 12:41
> *Subject:* Re: [AusNOG] Sensible MTU For Core Routers
>
> Hmm, you've opened the MTU can of worms.
>
> It's wise to set your internal interface MTUs to be both consistent and
> large enough for the outset to support any sort of tunnelling you might
> possibly perform across your network internally at any time in the future
> e.g., GRE, L2TP, IPsec, MPLS. Having a too small MTU is can be a much worse
> problem to have than a too big MTUs, so in the past I've just set it to the
> maximum supported by the device. Changing an MTU at a later date can either
> be very disruptive to the traffic flowing across the interface (because it
> may have to redo memory buffer allocations, or renegotiate it with the
> other end e.g., PPP) or may even require a reboot of a device (so service
> outage, out of hours work etc. and I like to say it takes 10 times the
> effort to work out of hours, so you avoid it everywhere you can.)
>
> A few other rules about MTUs:
>
> - know what the MTU value actually is describing. Is it a payload MTU
> (e.g. 1500 octets for Ethernet), a "whole of frame" MTU, which includes
> overhead fields of the frame or packet (e.g., for Ethernet, 12 octets Dst
> Addr. & Src. Addr. + 2 octet type/length + 1500 octet payload + 4 octet FCS
> (checksum) = 1518 octets. (Not that you'll likely to be exposed to it when
> setting an MTU, however there is also 8 octets of per-frame preamble in
> Ethernet, as well as an interframe gap that is also measured in bits, if
> you happen to want to calculate the maximum frames per second a layer 1
> link speed of e.g., 10Mbps supports. ))
>
> - VLAN tags or other tunnelling headers/fields may or may not be included
> in the MTU that you set. For example, if you're setting the "whole of frame
> MTU" on an interface, the vendor might be limiting it to allow 1 or more
> VLAN tags to be added. IOW, you're not actually setting the "whole of frame
> MTU".
>
> - Layer 2 payload MTUs are layer 3 whole of packet MTUs. This becomes
> important to know because if you use the term "IPv4 MTU" are you describing
> the IPv4 whole of packet MTU, or the IPv4 payload MTU? The IPv4 payload can
> vary because of IPv4 options (they're rare, however the one you're likely
> to see is the Router Alert option), as can the IPv6 payload because of
> Extension Headers.
>
> - We're now in a multiprotocol world, so it may be possible to set
> different payload and/or whole of frame MTUs for different protocols.
> Typically you'll be setting the "Layer 2 MTU", either the payload or whole
> of frame MTU. However, on routers it is likely to be possible to set
> different MTUs for different layer 3 protocols, and they'll be the "whole
> of frame MTU" for the layer 3 protocol. This can be useful to know in a few
> scenarios, but also explains why there might be multiple per-interface MTU
> parameters you can set on a router.
>
> - Some links can support different "MTUs" in different directions e.g.,
> PPP. PPP supports setting Max Receive Unit and Maximum Transmit Unit
> values, and they're announced to each other (or negotiatied, I'm getting a
> bit rusty)
>
> - in general make try to make sure all devices attached to the same link
> have the same MTU, otherwise one of the devices can generate a frame that
> is too big for other devices to receive, and it will be silently dropped.
> You can be a bit loose with this rule as long as you ensure that all frames
> that will ever appear on the link will be no larger than the smallest MTU
> that any of the attached devices (routers, switches, hosts.) support. This
> is a bit in conflict with my practice of setting it to as large as the
> device will support. I've figured that a bit of effort to ensure too large
> frames are originated or forwarded is a lesser effort than the possible
> effort (out of hours reboot etc.) involved in changing an MTU. I would
> strictly follow this rule when hosts are attached to the link that I don't
> administer.
>
> - avoid "dumbell MTU" scenarios i.e., large/small/large MTU values.
> They'll likely trigger packet fragmentation or path MTU discovery.
> Fragmentation can't or is costly to do in hardware, and PMTUD can be
> unreliable either because the ICMP Packet Too Big messages are dropped by
> overly zealous security people or are rate limited. In the later case, the
> rate limits can be increased, however they're still generated by the router
> control plane, which taxes the general purpose control plane CPU. Examples
> of dumbell MTU scenarios are residential PPPoE access without large PPPoE
> MTU support (i.e., customer LAN MTU:1500, PPPoE MTU:1492, ISP MTU:1500) or
> VPN scenarios (e.g., branch LAN MTU:1500, IPsec/L2TP tunnel MTU (much <
> 1500), central LAN MTU:1500). Lowering the outer MTUs to suit the middle of
> the dumbell MTU is better, and in IPv4 can be achieved with the DHCP MTU
> option (assuming the hosts support it) or the IPv6 RA MTU option. (The
> other option to 'fix' this is the TCP MSS hack, however that only works for
> TCP, not UDP, IPsec, GRE etc., etc., etc., and is ugly when you look at
> what it is actually having to do to pull it off (because the TCP MSS option
> doesn't have to be in a fixed location in the TCP segment.))
>
> - IPv4 minimum whole of packet MTU is 68 octets, IPv6 minimum whole of
> packet MTU is 1280 octets.
>
>
> With all those worms, it's probably time to go fishing.
>
>
>
>
>
>
>
> ------------------------------
>  *From:* James Mcintosh <james.mcintosh at rocketmail.com>
> *To:* "ausnog at lists.ausnog.net" <ausnog at lists.ausnog.net>
> *Sent:* Sunday, 19 April 2015, 10:53
> *Subject:* [AusNOG] Sensible MTU For Core Routers
>
> Hey Noggers,
>
> With router MTU's being configurable up to a whopping 9216 what are people
> setting for their interface MTU's on internet connected core and PE routers?
>
> Is it is still just the default of 1500?
>
> What about core switches? What are you setting there to get get
> interoperability with various ethernet providers?
>
>
> -James
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20150419/ee75b404/attachment.html>