[AusNOG] Spreading the load of ISP customers at Layer2

[Damian Ivereigh]

I would love to use fq_codel, but right now we just use Mikrotik's red. 
The biggest hassle is that NBN shape down at 10ms which is pretty hard 
on the shaping software!

Does make me wonder if we should ditch the Mikrotiks and use an open 
source solution.

Damian

On 9/13/21 10:33 PM, Dave Taht wrote:
> Wow. You live in such a different world than I. I would really like to
> better understand problems such as these, but where
> you are worried about arp at this low level, I worry about good queue
> and subscriber bandwidth management like that in this:
>
> https://github.com/rchac/LibreQoS
>
> (leveraging sch_cake in some releases)
>
> so, and I know it's kind of off topic from the problem you have... how
> the heck do you do bandwidth and queue management
> in either scenario below?
>
> On Mon, Sep 13, 2021 at 2:24 AM Damian Ivereigh <damo at launtel.net.au> wrote:
>> Hi guys,
>>
>> We have built all our ISP infrastructure based on the NBN style doubled
>> tagging of services - in other words each subscriber circuit comes
>> through on it's own ctag. This makes separating everything really easy
>> because we pipe each vlan through to different BNG's. However we are now
>> presented with a wholesaler who does not separate each circuit, but
>> instead just bridges them all together into a single circuit. We can
>> distinguish each circuit only by inspecting the DHCP Option82 so that we
>> can allocate the right IP address, which is fine, but it is hard to
>> allocate them to use a particular BNG to send and receive traffic.
>>
>> By the way I am not talking dynamic load balancing just having multiple
>> BNG with a subsection of the customers on each one - load sharing?
>>
>> Until now with double tagging, we can reuse the same gateway IP address
>> (i.e. the side facing the customer) on all the BNG and because each BNG
>> only sees it's circuits, it will only respond to arps that it should do
>> on the vlans assigned to it. However with all the customers on the same
>> circuit it is impossible for multiple BNG to have the same IP address
>> without creating all sorts of duplicate arps etc. We could turn off arp
>> on all but one of the BNG and then put up with the asymmetric routing
>> (makes reverse path filtering impossible) - i.e. send all upload traffic
>> through a single BNG, but download comes from different ones (according
>> to what BNG they are allocated to).
>>
>> I have come up with another hack by using essentially using arp spoofing
>> where we get a separate box to respond to the arp requests based on what
>> the source IP is, but I can't help wondering how others have handled
>> this. The wholesaler tells me there are other ISPs with 5000+ services
>> on the single circuit (feels like a recipe for a broadcast storm to me).
>>
>> Oh and no we don't want to use PPPoE :-)
>>
>> Ideas anyone?
>>
>> Damian
>>
>> --
>> Launtel - We're at your call
>> Tel: 1800LAUNTEL (1800528683)
>> Mob: 0418217582
>> Fax: 1300784109
>> http://www.launtel.net.au
>>
>> _______________________________________________
>> AusNOG mailing list
>> AusNOG at lists.ausnog.net
>> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
-- 
Launtel - We're at your call
Tel: 1800LAUNTEL (1800528683)
Mob: 0418217582
Fax: 1300784109
http://www.launtel.net.au