[AusNOG] the state of bufferbloat awareness and remediation in australia?
Dave Taht
dave.taht at gmail.com
Sun Jan 26 13:18:41 EST 2020
The following info comes courtesy of sebastian as to what is appearing
in germany on several PPOE messages as for link speed and framing.
On Sat, Jan 25, 2020 at 5:57 PM Damien Gardner Jnr <rendrag at rendrag.net> wrote:
>
> Had started replying the other day, but seems I forgot to hit send, so i’ll reply tothis one instead :)
>
> We see them as PPPOE headers, on our AAPT wholesale services with NBN and AAPT direct DSL services - but not on Telstra services. It’s very handy, as AAPT at least rate limit how often you can do a line state diagnosis to pull sync rates from NBNco (it’s about six or seven per 30 mins maximum), so if you have a tech at the customer’s premises messing with cabling, being able to see sync rates on every reconnection saves a lot of hair pulling once you hit the rate limit for LSD’s.
>
> On Sun, 26 Jan 2020 at 12:52 pm, Ryan Mounce <ryan at mounce.com.au> wrote:
>>
>> NBN Co do inject options into DHCP and PPPoE requests with the down/up sync rate for their wholesale xDSL services so that ISPs (RSPs in NBN lingo) can shape individual services correctly. I have no sense how widely ISPs are actually taking advantage of this.
>>
>> -Ryan
>>
>> On Fri, 17 Jan 2020 at 11:24, Dave Taht <dave.taht at gmail.com> wrote:
>>>
>>> Hi, all. I'm here at linux.conf.au having just given a talk about how
>>> tcp works in the bufferbloated age[1].
>>>
>>> On my way here I stopped in for a few days with geoff huston and
>>> george michaelson who filled my ears with the chaos of the NBN rollout
>>> and other issues in the australian network infrastructure... and gave
>>> me a shot at some data they had on bloat, and ecn usage in the wild
>>> which I hope to write up over the next month or so.
>>>
>>> I was wondering about a few things:
>>>
>>> How y'all doing on eliminating bufferbloat from your networks? Using
>>> things like fq_codel, sch_fq + bbr, etc?
>>>
>>> Do you have any awareness from your regulatorium?
>>>
>>> I see, from sites like whirlpool, that some consumer hardware here,
>>> like the fritzbox, have fq_codel now, but it's not clear if ISPs are
>>> actively configuring it (or what we call "sqm") yet. (theres a PPPoe
>>> message now in use in parts of germany for up/down and frame rate
>>> seeing increasing deployment).
>>>
>>> Lastly:
>>>
>>> Anyone need a wayward network researcher/theorist for a few weeks or
>>> months to help address their bufferbloat issues in their stacks and
>>> hw? - maybe not this trip but on some other occasion?
>>> (I rather like hanging in australia)
>>>
>>> [1] blatant plug http://youtu.be/ZeCIbCzGY6k
>>>
>>> --
>>> Make Music, Not War
>>>
>>> Dave Täht
>>> CTO, TekLibre, LLC
>>> http://www.teklibre.com
>>> Tel: 1-831-435-0729
>>> _______________________________________________
>>> AusNOG mailing list
>>> AusNOG at lists.ausnog.net
>>> http://lists.ausnog.net/mailman/listinfo/ausnog
>>
>> --
>> Regards,
>> Ryan Mounce
>>
>> ryan at mounce.com.au
>> 0415 799 929
>>
>> Sent from mobile
>> _______________________________________________
>> AusNOG mailing list
>> AusNOG at lists.ausnog.net
>> http://lists.ausnog.net/mailman/listinfo/ausnog
>
> --
>
> Damien Gardner Jnr
> VK2TDG. Dip EE. GradIEAust
> rendrag at rendrag.net - http://www.rendrag.net/
>From sebastian:
So, I can try to describe the situation here in Germany, as I
see it currently. The incumbent, Deutsche Telekom started to transmit
a peculiar piece of information as part of the PPPoE connection
establishment. I assume that they are using the defined RFC 1994
message field of the success-message passed from the upstream
PPPoE-termination point to the modem (RFC 1994 Sektion 4.2):
4.2. Success and Failure
Description
If the Value received in a Response is equal to the expected
value, then the implementation MUST transmit a CHAP packet with
the Code field set to 3 (Success).
If the Value received in a Response is not equal to the expected
value, then the implementation MUST transmit a CHAP packet with
the Code field set to 4 (Failure), and SHOULD take action to
terminate the link.
A summary of the Success and Failure packet format is shown below.
The fields are transmitted from left to right.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Code | Identifier | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Message ...
+-+-+-+-+-+-+-+-+-+-+-+-+-
Code
3 for Success;
4 for Failure.
Identifier
The Identifier field is one octet and aids in matching requests
and replies. The Identifier field MUST be copied from the
Identifier field of the Response which caused this reply.
Message
The Message field is zero or more octets, and its contents are
implementation dependent. It is intended to be human readable,
and MUST NOT affect operation of the protocol. It is recommended
that the message contain displayable ASCII characters 32 through
126 decimal. Mechanisms for extension to other character sets are
the topic of future research. The size is determined from the
Length field.
Here an example of what is actually send by DT (confirmed for both DSL
and GPON-FTTH links), or rather what OpenWrt's pppd displays in the
log:
Thu May 2 18:56:32 2019 daemon.debug pppd[21113]: rcvd [PAP AuthAck
id=0x1 "SRU=9461#SRD=48387#"]
The values transmitted behind the SRU and SRD keys appear to be
approximate PTM/VLAN/PPPoE/IPv4/TCP-payload rates for Upload and
Download respectively in Kbps.
Message='SRU=yyyy#SRD=xxxx#'
xxxx=Download rate in KBit/s
yyyy=Upload rate in KBit/s
Another German ISP (1&1, united internet) also uses the same mechanism
to convey similar information:
Message='[UI-SBR:xxxx,yyyy;UI-LINEID:zzzz;]'
xxxx=Download rate in KBit/s
yyyy=Upload rate in KBit/s
zzzz=LineID
Except here fr 1&1 the transmitted rates are supposedly gross rates
(corrected for PTM's64/65 encoding) while DT sends estimated net
rates.
The largest German national maker/brand of home routers AVM started to
automatically extrcat this information and display a processed version
of these values in its log (as "information of the ISP about the real
rates", or something with similar meaning) and they also use these
values to indirectly configure their own ingress and egress traffic
shapers (indirectly as they do not use these values verbatim, which
makes sense, as a shaper requires gross rates to configure so DT's net
rates need to be used to estimate the required gross rates). All in
AVM seems to still struggle to get this working correctly.
As far as I can see there is nothing in the actual standards
standardizing this, but as shown above it certainly does not violate
RFC1994... Personally I wish that the IETF would standardize something
like that (and that they would do it properly by requiring the full
required tuple: Upstream: gross ISP shaper setting, minimal required
per-packet-overhead; and the same 2 values for the downstream).
OpenWrt and sqm-sripts currently do not extract this information
automatically (well, IIRC this message makes it into OpenWrt's log,
but sqm-sxripts currently does nothing with this). This is partly
because it is unclear how many users would profit from this (DT's and
1&1' german customers are only a few dozen million in total, with only
a tiny fraction running OpenWrt), and my own ISP playing cute and
sending a non-sensical:
Sat Jan 11 03:47:29 2020 daemon.debug pppd[14593]: rcvd [PAP
AuthAck id=0x1 "Pedo mellon a minno : ######DEU.DTAG.A70X1# -
<unknown>"]
instead... (TANGENT: Now, I do not hate the lord of the rings, but
just having read the Hobbit to my kids, I do think that his first book
is much nicer with its considerably smaller scope)
Since I will not be able to test this, I have refrained from trying to
do anything automatic with it (we would need to put some logic into
the hotplug script and a config option to tell SQM an interface is
PPPoE based, so that on up of an interface sqm will try to quizz the
logfile for this information, which is a pain given the different
formats and a lack of proper documentation what these values are
supposed to mean from each ISP)
> help? Is it a standard?
As far as I know it is not. Then again it is not in violation
of the applicable RFC.
> does it work in stock openwrt?
Yes, in that OpenWrt's pppd seems to pass the message to the
logfile even if not configured in verbose mode. No, in that neither
OpenWrt nor sqm-scripts do anything with this information.
> dsl only?
No, Deutsvhe Telekom seems to be using it for both DSL as well
as GPON customers (the big question I have is how long will they keep
carrying the PPPoE encapsulation along).
> what isps?
So far this is confirmed for Deutsche Telekom and 1&1/united
internet, the first being Germany's incumbent (former monopoly) telco,
the second mostly a Telekom reseller that started to build its own
backbone and now mostly rents local loops/bitstream from the incumbent
but uses its own back-end.
I have zero information about other ISPs and I also do not
know about whether the European daughters of DT use the ame scheme or
not (DT intends to harmonize its complete European network, but they
are not halfway done with this and I have no clue whether they intend
to do this for SRU/SRD).
> --
> We rode on the winds of the rising storm,
> We ran to the sounds of thunder.
> We danced among the lightning bolts,
> and tore the world asunder
--
Make Music, Not War
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729
More information about the AusNOG
mailing list