[AusNOG] Spreading the load of ISP customers at Layer2

[Alastair Johnson]

5K subscribers on a single port/handover VLAN for any modern BNG should 
not be a problem...

Assuming you're sticking with IPoE/DHCP and you really want to split 
subscribers over multiple BNGs, I'd do something like this:

1. Present the backhaul VLAN to your N BNGs. You will need to use unique 
IP addresses on the subscriber-interfaces (or equivalent depending on 
your vendor).

2. Configure the BNGs for DHCP relay or DHCP-to-RADIUS proxying (or 
equivalent).

3. Since DHCP will be broadcast, all BNGs should receive the 
DHCPDISCOVER from the client and relay/proxy it to your DHCP/RADIUS servers.

4. Based on Option-82 information, have your DHCP/RADIUS servers return 
a DHCPOFFER/RADIUS Access-Accept only for the BNG you *want* to serve 
the client. That should bring up subscriber management on the specific 
BNG and ensure you have symmetric traffic flows for that subscriber.

You might have to deal with the other BNGs generating logs for the 
session failing to establish on them (no DHCPOFFER or RADIUS response, 
or maybe you could send an Access-Reject or something).

Depending on how the N:1 VLAN is configured, you may need to be cautious 
with local-proxy-arp to ensure that only the active BNG responds for 
subscriber ARPs.

There should be no problems with broadcast storms - assuming the 
wholesaler is using N:1 split-horizon/forced-forwarding to prevent 
sub-to-sub communication at L2.

There are other ways to solve for this, but I'm not sure why you'd want 
to make this much more complex :-).

AJ
<I haven't done BNG stuff for 7 or 8 years, but I'm fairly confident 
this will work - at least on the platforms I'm most familiar with>


On 9/13/21 2:23 AM, Damian Ivereigh wrote:
> Hi guys,
> 
> We have built all our ISP infrastructure based on the NBN style doubled 
> tagging of services - in other words each subscriber circuit comes 
> through on it's own ctag. This makes separating everything really easy 
> because we pipe each vlan through to different BNG's. However we are now 
> presented with a wholesaler who does not separate each circuit, but 
> instead just bridges them all together into a single circuit. We can 
> distinguish each circuit only by inspecting the DHCP Option82 so that we 
> can allocate the right IP address, which is fine, but it is hard to 
> allocate them to use a particular BNG to send and receive traffic.
> 
> By the way I am not talking dynamic load balancing just having multiple 
> BNG with a subsection of the customers on each one - load sharing?
> 
> Until now with double tagging, we can reuse the same gateway IP address 
> (i.e. the side facing the customer) on all the BNG and because each BNG 
> only sees it's circuits, it will only respond to arps that it should do 
> on the vlans assigned to it. However with all the customers on the same 
> circuit it is impossible for multiple BNG to have the same IP address 
> without creating all sorts of duplicate arps etc. We could turn off arp 
> on all but one of the BNG and then put up with the asymmetric routing 
> (makes reverse path filtering impossible) - i.e. send all upload traffic 
> through a single BNG, but download comes from different ones (according 
> to what BNG they are allocated to).
> 
> I have come up with another hack by using essentially using arp spoofing 
> where we get a separate box to respond to the arp requests based on what 
> the source IP is, but I can't help wondering how others have handled 
> this. The wholesaler tells me there are other ISPs with 5000+ services 
> on the single circuit (feels like a recipe for a broadcast storm to me).
> 
> Oh and no we don't want to use PPPoE :-)
> 
> Ideas anyone?
> 
> Damian
>