Which VPN to use?

[defaultuserfoo]

Using a different network on the remote site with its own DHCP server probably won't work, at least not easily.  I'm pretty sure that the access point controller uses DHCP options to tell the access points where to find itself so that they can establish a tunnel to the controller.  Maybe that won't work anyway when MSS settings interfere ...

Never said use a different network. What I said was split the network between the two sites just to keep things clear. You'll still use the same /24 (or whatever the local is) but set the remote to specific IP's. This suggestion is basically just for "record keeping" and not a necessity at all.

Right --- it just seems like it's a different network because it's a different place, would have it's own DHCP server and it's own range of addresses.

There will be about 20 access points on the remote site (and there are about 20 locally). [...]

With these access points, I probably can't give them static addresses because that would involve to configure them manually.  I think there's some option to tell them where the access point controller is.  That would also require to pre-configure them all manually first, and I don't know if that would work.

20 AP's means a lot of clients I would think. Both of my sites have a symmetrical gig for internet. I have no problems whatsoever with bandwidth so don't believe the warnings you hear. That said I only have 2 servers and 10 cameras at the remote site. Cameras are nothing when it comes to bandwidth but the servers do use a bit, as I said, no problems though.

Well, try to get a 1GB internet connection in this country ...  If you're lucky, you have to pay for 50Mbit and actually get between 2 and 20 --- and then your VPN connection doesn't work because you're being forced to use a router you have to rent or buy from the ISP (because there is no alternative due to the technology they're using) which is so crappy that it blocks the connection ...  Or you don't get internet at all, that can easily happen.  It's a nightmare, and I guess it's too late to matter because this country has made sure to be left behind, and was left behind a long time ago, not only in that regard.

Anyway, it's a big area to cover and not so many clients.  The area is so big, there will be more access points than clients, at least for a while.

Is it not possible to just extend the VLAN as if there was a network between the two sites?  What's the problem with DHCP?  If I need to, I can set a long lease time, and the APs tend to keep their IPs even when they can't reach the controller and continue to work once they are configured.

Vlans are layer 2. Can't route them so I'm not sure what you mean by "a network in between".
DHCP, not really a problem but I found you can't assign a DHCP range to just the remote site and addresses were being handed out to the local site as well. If you don't care what address goes where, then just leave one DHCP pool and let it go.

If opnsense can do this correctly, you'll find that it's exactly what you wanted.

Oh, sorry. I meant "... as if there was a network CABLE between ...".

I doesn't matter at which site the addresses are being used.  I'll have to try it out once all sites are migrated from Ubiquity EdgeRouters to OPNsense.

[defaultuserfoo]

What do you mean by having to register the Unifi devices mandatorily? I used those devices for years and never had to do anything like that.

I have been reading that you can not deploy their so-called dream machine without creating an account with them and registering it with them.  Apparently you can turn off the connection to that account later on but should you ever want to sell your hardware, the next buyer won't be able to use it because of that account.

And how else would you configure their access points?

Since you chose to use a professional solution by HP, what makes you think you need to try to extend a broadcast domain in order to have the APs configured? I would assume even HP has seen some kind of routed enterprise network over the years.

If you look at the HP manuals (https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c02704528), they describe discovery procedures over routed networks, like a means for DHCP- or DNS-based discovery (see page 6-6 onwards).

I didn't check the documentation.  Finding anything on HPs web sites is a nightmare and they keep changing everything all the time, so I couldn't be bothered to check.  And that access point controller requires a very weird configuration that doesn't make any sense.  It wants you to somehow use what they call an internet port --- as if anyone would connect that thing to the internet --- and I'm not sure what it's supposed to be for.  The other port it has, they call a LAN port, and apparently the APs can only be managed through the LAN port.  So the most reasonable way to configure it was to use the LAN port as a LAN port in that it connects to the VLAN the access points are connected to so that they can be managed while the internet port has the VLANs on it that carry the data that goes over the APs to the clients.  However, the internet port isn't really needed for that because the ports on the switches the APs are connected to carry these VLANs, so the acceess point controller doesn't create a bottleneck.  IIRC I tried it with the internet port disconnected because it's not needed, but I left it plugged in because it didn't work when disconnected ...  I guess they intend the controller to be a bottleneck, but I don't know, it just doesn't make sense and I don't want a bottleneck like that.

So the easiest way seems to just extend the VLAN the APs are being managed through to the remote site.  With all the fine VPN technology we nowadays have, that can't be a problem, can it? :)

But I'll take a look at the documentation.  Maybe it shows a better way.

[defaultuserfoo]

But I'll take a look at the documentation.  Maybe it shows a better way.

Ok, yes ... Did I mention that HP has outstanding documentation for their stuff?

How do I add an option space and all the things mentioned in appendix E7 in that documentation to the DHCP server in OPNsense?

I can try that with the local DHCP server first and if I can get it to work, I can do it at the remote site and just have the management traffic go through the wireguard connection which will be used for everything else anyway.

(I did try with the DHCP server in OPNsense but it didn't work because I didn't set up right, so switched back to the DHCP server in the controller ...  I'd rather use the DHCP server in OPNsense anyway ...)

[Patrick M. Hausen]

You don't need to register your devices mandatorily. But then I never used their "dream machine". You just deploy their free controller software via Docker, in a Linux VM, in a FreeBSD jail, whatever suits your environment best.
If you have multiple locations, tell the APs the address of the controller via DHCP.

And stretched layer 2 is a problem regardless of the VPN technology. You can't cheat with physics (bandwidth x delay).

[defaultuserfoo]

I was told you can't use a VM anymore and that the USB-thing they used to have only works for some of the APs.  Ubiquity has never been forthcoming with information of any kind.

And docker?  That's a container, isn't it?  I don't like containers and rather use VMs.  (Try to get fail2ban to work in a container ...)

Are you saying that stuff happening on layer 2 is designed and/or required to work with the bandwidths and delays only layer 1 provides and that when you introduce bandwidth limitations and delays by tunneling the connection over some kind of VPN, the ghosts residing in the wires hit the fan and it all comes apart?


PS: https://docs.opnsense.org/manual/dhcp.html :

"To configure options that are not available in the GUI one can add custom configuration files on the firewall itself. Files can be added in /usr/local/etc/dhcpd.opnsense.d/ for IPv4"

So I'll try that ...

[Patrick M. Hausen]

I was told you can't use a VM anymore.

Well, the Unifi controller software is available as a FreeBSD package and via a custom apt repo for Debian or Ubuntu yo you jet set up an environment of your choice - hardware, VM, jail - and use "pkg install" or "apt-get install" and you have a Unifi environment. I still run one @work ;)

But you decided not to use Unifi, so that's rather irrelevant to you, I guess. I know how bad their documentation is and I am not happy with their attitude to support, either.

Are you saying that stuff happening on layer 2 is designed and/or required to work with the bandwidths and delays only layer 1 provides and that when you introduce bandwidth limitations and delays by tunneling the connection over some kind of VPN, the ghosts residing in the wires hit the fan and it all comes apart?

Spot on. Any ARP request, IPv6 neighbor discovery packet - which are mechanisms designed for a local link - must be sent over the VPN link, too, if you want layer 2 connectivity. Layer 2 relies on these mechanisms. That's why among network professionals "stretched layer 2" is generally frowned upon. Lookup Ivan Pepelnjak oder Greg Ferro, of you don't trust Patrick M. Hausen  :)

No with a gigabit link and a limited number of stations on both sides it will probably work but then again it might fail in unexpected ways and is hard to debug. Layer 3 (IP) was invented for "Internetworking", i.e. connecting networks.

PS: https://docs.opnsense.org/manual/dhcp.html :

"To configure options that are not available in the GUI one can add custom configuration files on the firewall itself. Files can be added in /usr/local/etc/dhcpd.opnsense.d/ for IPv4"

Custom DHCP options can be configured right in the UI. The mechanism you cited is for unsupported (by the UI) dhcpd configuration options. Not necessary for e.g. Unifi controller discovery. See my screenshot above.

HTH,
Patrick

[defaultuserfoo]

It may have been an option to switch to access points from Ubiquity some day.  Maybe it's still an option, especially when it's easy to run their controller software.  Apparently they're making good APs --- and their EdgeMax hardware is/was good, too, especially for the price.

It's not like I don't believe you.  I haven't tried it yet and you're probably right.

I put a config file, and it's being nicely appended to /var/dhcpd/etc/dhcpd.conf .  However, it doesn't work.  It seems like the DHCP server doesn't receive DHPC requests on that VLAN interface.

Won't it listen when it's not enabled in the GUI for that interface?  I can't enable it in the GUI because my configuration file won't work when it is.

I disabled the DHCP server in the AP controller andI disabled an AP by disabling a port on a switch the AP is connected to, and the controller shows one less AP.  When I re-enable the switch port, the AP apparently retains its IP address, talks to the controller and synchronizes and continues to work as usual.  There are no leases showing up in OPNsense, not in the GUI and not in /var/dhcpd/var/db/dhcpd.leases.  There are no corresponding entries in the log file of the DHCP server, either.  I added firewall rules to the VLAN interface like the automatically created ones on the interfaces for which the DHPC server is enabled.

Is it not possible to get the required configuration to work?

[Patrick M. Hausen]

Won't it listen when it's not enabled in the GUI for that interface?  I can't enable it in the GUI because my configuration file won't work when it is.

As far as I know it doesn't. Why do you need an extra configuration file? What precisely do you need to set in the DHCP server that you cannot achieve through advanced options in the UI?

[Vilhonator]

Docker or any container allowes you to install software natively regardless which OS you use (for example Steam uses container to install games to Linux and SteamOS), so ease of use is down to which OS you run the container in.

What comes to Ubiqity, you can either buy cloud key (physical device with Ubiqitys own OS installed on it) and connect that to Ubiqity switch.

If you don't have a switch from Ubiqity, then you can install the free software and run it on either physical machine or VM image of windows, Linux or Mac OS (just read the instructions)

Yes, you have to register a local user for Ubiqity (without it Web GUI won't work), but you don't have to register online.

VLANs share their bandwidth and delay on physical ports (for example if you have 200 computers on VLAN 20 and 100 computers on VLAN 30 and both VLANs are signed to 1 single 10Gb port, overall bandwidth of both VLANs is 20Gb/s, then it can go below 10Gb/s due to physical limitation of the NIC, you can't exceed physical hardware limitations with VLANs.)

Delay depends on the length and type of the cable and hardware your firewall has. Now you can improve delay and bandwidth within internal networks by adding switch, then firewall only needs to take care of internet side of things.

VLANs are generally used by schools and corporates to separate networks (1 example would be 1 VLAN to manage your network including having SSH and Webgui access to switch and firewall, 2nd which has access to only internet and 3rd which has access to intranet stuff like IP phones and internet)

You certainly can setup VLAN for VPN, but unless you have speciffic requirement (for example you would have to allow internet and / or intranet access, but blog web and SSH access to firewall etc.), it's pretty pointless.

If you are trying to achieve something on home network and instructions won't make any sense to you, then just leave it there, read more documents about things in question, so that you have general knowledge of things and try again.

VPN doesn't require much knowledge, but VLANs require NIC that supports 802.1q Tagging (IEE 802.1q) and switches with VLAN tagg support and quite a bit knowledge to get them up and runnign well.

[Demusman]

Just an FYI about Ubiquiti AP's, you don't NEED to use a controller at all.
You can use the mobile app and configure them as stand-alone AP's.
They call it "limited functionality" or something like that but all that means is without their control over them that their controller provides.
I have quite a few of them out there without a controller and they work just like any other AP would.

Just something to keep in mind.

[defaultuserfoo]

What is a "mobile app"?  Don't they have a cli built in or at least a WEB GUI?

Even if they do, I don't want to configure more than 3--5 APs individually.  More than 2 is already bad enough ...

[defaultuserfoo]

Won't it listen when it's not enabled in the GUI for that interface?  I can't enable it in the GUI because my configuration file won't work when it is.

As far as I know it doesn't. Why do you need an extra configuration file? What precisely do you need to set in the DHCP server that you cannot achieve through advanced options in the UI?

I would need to do what they describe in Appendix E7 of this documentation: https://support.hpe.com/hpesc/public/api/document/c02704528?docVersion=1#pdfjs.action=download

(And I seem to remember that I tried it with an ISC DHCP server on Linux when I first tried to configure the AP controller and didn't work back then.  I'll have to look into the name server option, maybe that works.  But it seems to require to pre-configure the APs, which would suck.)

[defaultuserfoo]

Docker or any container allowes you to install software natively regardless which OS you use (for example Steam uses container to install games to Linux and SteamOS), so ease of use is down to which OS you run the container in.

So it's still a container?  If that is what steam uses, how come that they keep steam processes running even after you closed their programs, like they are a virus, and how come that I can kill these processes just like that?  Is it for monitoring you and stealing your data?  A container shouldn't allow you to kill processes running in it from the outside just like that.  And whatever runs in the container shouldn't be allowed to use my screen, for example.  It doesn't matter if you call it "Docker" instead of "container" --- or isn't it a container?

What comes to Ubiqity, you can either buy cloud key (physical device with Ubiqitys own OS installed on it) and connect that to Ubiqity switch.

I was told you can't really use the USB thing anymore because it's limited to some models of APs only.  There was some other reason why you couldn't use the software that runs on Linux.  Maybe the information is false, but when the European distributor for Ubiquity (who's trying to sell me their APs which I can't buy when I can't configure them) tells me this, what am I supposed to believe?

Besides, the MSM422 costs about EUR 15 each off ebay, including the mounting plate.  It's not their fault, but Ubiquity can't beat that.  And wich of the APs Ubiquity makes have metal housings?

If you don't have a switch from Ubiqity, then you can install the free software and run it on either physical machine or VM image of windows, Linux or Mac OS (just read the instructions)

There are instructions?  Don't get me started about documentation when it comes to Ubiquity.  If you need documentation or instructions, don't touch anything from Ubiquity.

Yes, you have to register a local user for Ubiqity (without it Web GUI won't work), but you don't have to register online.

That's not what I read on their forum.  People there clearly said you have to make an account with them and register your hardware (i. e. their so-called dream machine) or you will not be able to even configure it.

I guess Ubiquity dreams that someone would do that.  And when a bunch of people have done it, they will just discontinue the accounts and the machines become worthless after they got their money and their data.  Or they will tell you which hosts you can connect to and what you can do and not do with your hardware.  Keep dreaming ...

Maybe ask on their forum what kind of SLA you get when you buy one of their DMs and what guarantees they have that they will fulfill it :)

VLANs share their bandwidth and delay on physical ports (for example if you have 200 computers on VLAN 20 and 100 computers on VLAN 30 and both VLANs are signed to 1 single 10Gb port, overall bandwidth of both VLANs is 20Gb/s, then it can go below 10Gb/s due to physical limitation of the NIC, you can't exceed physical hardware limitations with VLANs.)

10GB switches are freaking expensive.  I wish I had one, but they aren't affordable yet.

Delay depends on the length and type of the cable and hardware your firewall has. Now you can improve delay and bandwidth within internal networks by adding switch, then firewall only needs to take care of internet side of things.

Are you sure that the length of the cable makes a noticable difference in latency?  How would that happen?

Routing between VLANs with switches isn't such a great idea when you use the VLANs to keep different networks apart instead of using separate cabling.

VLANs are generally used by schools and corporates to separate networks (1 example would be 1 VLAN to manage your network including having SSH and Webgui access to switch and firewall, 2nd which has access to only internet and 3rd which has access to intranet stuff like IP phones and internet)

You certainly can setup VLAN for VPN, but unless you have speciffic requirement (for example you would have to allow internet and / or intranet access, but blog web and SSH access to firewall etc.), it's pretty pointless.

Why would you set up your networks without VLANs?

If you are trying to achieve something on home network and instructions won't make any sense to you, then just leave it there, read more documents about things in question, so that you have general knowledge of things and try again.

VPN doesn't require much knowledge, but VLANs require NIC that supports 802.1q Tagging (IEE 802.1q) and switches with VLAN tagg support and quite a bit knowledge to get them up and runnign well.

VPN is one of the most complicated things you can try to set up.  It's easy in OPNsense now, but you should have tried it 20 years ago ...  VLANs are easy, unless you're using switches from Ubiquity.  VLANs on HP switches are a blaze and fun to configure (And HP treats them different than Ubiquity and Cisco do, but where is that documented with switches from Ubiquity?).

[Demusman]

What is a "mobile app"?  Don't they have a cli built in or at least a WEB GUI?

Even if they do, I don't want to configure more than 3--5 APs individually.  More than 2 is already bad enough ...

Android or IOS apps.
Actually not sure if they have an IOS app but I use the Android app and it's very simple to setup AP's.

[Demusman]

VLANs are generally used by schools and corporates to separate networks (1 example would be 1 VLAN to manage your network including having SSH and Webgui access to switch and firewall, 2nd which has access to only internet and 3rd which has access to intranet stuff like IP phones and internet)

You certainly can setup VLAN for VPN, but unless you have speciffic requirement (for example you would have to allow internet and / or intranet access, but blog web and SSH access to firewall etc.), it's pretty pointless.

If you are trying to achieve something on home network and instructions won't make any sense to you, then just leave it there, read more documents about things in question, so that you have general knowledge of things and try again.

VPN doesn't require much knowledge, but VLANs require NIC that supports 802.1q Tagging (IEE 802.1q) and switches with VLAN tagg support and quite a bit knowledge to get them up and runnign well.

I don't think you understand vlan's very well. You do realize they are just Virtual LAN's, right??
If you have a switch on any network, you're using a vlan. Not just schools and corporations, any network with a switch. Now you can segment a switch with multiple vlans and this is the equivalent of adding another switch, it becomes 2 broadcast domains instead of 1. Doesn't really take a lot of knowledge compared to VPN's.