Messages

Is it possible for OPNsense to do policy-based routing? If so, is there a limit to the number of tunnels/VPNs that can be on simultaneously?

This is what I want to achieve. Instead of forcing an entire device to use the VPN, I want devices to be routed to the appropriate VPN based on the domains that it's using. For example, all queries to Chase will be routed to a US VPN. All queries to the BBC would be routed to a UK VPN. Any queries to domains not on a list would be routed directly to the Internet and not use a VPN.

Is it easy to set up this policy-based routing? If so, how do I set it up? Is there a guide out there?

Thank you!

Even if 64 GByte did actually work with an N1x0, you still only have 4 threads and 1 memory channel, which will give quite low performance for more than 2-3 VMs.

Also, the N100 only has 9 PCIe 3.x lanes. Every of the 4 NICs takes one, you usually need 4 for an NVME drive, at least one for the USB controller for keyboard and mouse and that is it. When those devices have a SATA controller at all, they have to pull tricks like using less PCIe lanes for the NVME drive. I know that because my box also has two SFP+ slots, which each takes up another PCIe lane.

You may think you can get away with using USB storage, but that is not recommendable for Proxmox or any other VM host, for that matter. ANd if you move the storage to a NAS, you have gained nothing, because that means another physical machine.

As I said: An N1x0 is just fine for OpnSense bare-metal, probably for a Proxmox host running OpnSense and a Unifi controller, but not for a real home lab server that runs >4 VMs. You can try, but you will outgrow that solution very fast.

Thanks for the info. Please excuse the dumb questions. I still have a lot to learn.

Is it worth virtualizing everything? I get that it saves money since I don't need to buy another machine, but I also want to do it right. These N100 mini PCs are relatively power-efficient and I think I can combine that with a UPS.

So I am under the impression that the OPNsense router should be a PC all to itself. Is the Unifi controller placed in a VM then? I'm assuming this isn't doing any routing, but to control Ubiquiti's networking gear like switches. If I utilize Adguard Home with extensive blocklists, say over 1 million rules count, and set OPNsense to connecting to 7-10 VPNs simultaneously, is 16GB of RAM enough or should I be looking at 32GB? I think OPNsense, potentially Ubiquiti controller, Adguard Home, and multiple VPNs running simultaneously will be the use case with PC #1.

What you suggested earlier, I'll move it to PC#2. Good idea?

fw - firewall
.1q - IEEE 802.1q, the standard that defines how to transport several VLANs over a single physical link between two devices, like e.g. an OPNsense firewall and a managed switch

Thank you, Patrick. I still have a lot to learn. I would give you a like if there were a button.

Its not clear to me how the lab part is related to the fw part.
Is the lab just some hosts on the lan site, perhaps multiple network subnets on the LAN side?
It's why it's good to use managed switch where you can configure vlan ID's and use a .1q connection to fw LAN port, this way you build your lab nets and then fw controls access between those LAN nets, and, controls access from all LAN nets to the WAN side.

With just two copper ifaces on the fw device (WAN/LAN), if you want segmentation on the LAN side for "lab" stuff, using .1q is the way to do it (if you want opnsense fw control, etc).

Hello, what do you mean by fw and .1q?

As for the OpnSense box: Yes, I prefer dedicated. The N1x0 boxes will do just fine and they are limited RAM-wise to 16 GByte anyway, AFAIK, so no, you cannot put in 64 GByte - the N1x0 are also limited to just one channel of memory, so your "spare" memory slot is normally the "only" one (unless you have another CPU).

Usually, you can get away with 8 GByte, at least if OpnSense is running bare-metal, so such passively-cooled boxes will suffice.

The Proxmox server (which is not itself a VM, but the host machine where all VMs and LXCs run on) can physically be a mini PC, too, but you should get something more along the lines of an Intel i5 CPU, which is less limited RAM-wise and can handle the load of multiple VMs better with more than 4 threads. You will likely need active cooling for this. For that reason and because eventually, you will need HDD storage for backup as well, I would suggest getting a small NAS device, like the Ugreen DXP 4800 Pro (or one of the larger models)  - not the non-Pro one, because it also is limited with RAM, whereas the 4800 Pro can handle up to 96 GBytes and up to 2 NVME and 4 HDD disks, of which you first should use 2 as a ZFS mirror to be able to expand later. I already stated how much NVME storage you will need, namely 1-2 TByte.
For backups, plan to have ~6 TByte and add your media pool size to that for HDD storage. Take that times 2 for a mirror or distribute it over N HDDs and add 1 for RAID-Z1. or 2 for RAID-Z2.

If you already have a NAS, use that for independent backup of your Proxmox server. Most older NASes are unable to run Proxmox, many because they do not even have x64 architecture and / or not enough RAM.

The Unifi Network Controller ist a piece of software that runs on top of Linux and controls most Unifi devices (switches and access points), so you do not need a Unifi router with an integrated controller. There now is a replacement for that, called UOS, which is in beta, but works just fine.

None of these are able to control their Protect line of products, though. Ubiquiti has ceased to support a pure software solution for those.


By "internal" and "external" Docker, I mean separating your internal (LAN) services and external (WAN) services on different networks and thus, VMs, so you need two Docker instances. You do not want someone using your external services break in and then sniff around in your LAN...

As for Home Assistant (OS): https://www.youtube.com/watch?v=Z4gvkmJ8q48

I do not use Adguard Home or Technitium, but I know you can install Adguard into OpnSense or as a separate VM under Proxmox (although that obviously introduces another single point of failure).

Too bad there isn't a "like" button on this forum, but thank you for the info!

According to Reddit, the Intel N100/N150 does work with 32, 48, and 64GB RAM modules. It's unofficial as Intel only tested and validated up to 16GB, but it works if you trust people from Reddit. I haven't had a chance to verify it myself, but I'm tempted now.

While you can actually run OpnSense virtualized under Proxmox, there are some pitfalls and also, your 16 GB mini PC will probably not fit the bill because of RAM shortcomings. If you only want a Unifi controller and OpnSense, then fine, but any other "big" VM after that and you will likely have too little RAM.

I like to keep OpnSense on a seperate mini PC, for this, security and practicality reasons (i.e. when you work on your Proxmox box, you will still want internet access).

As for the Proxmox machine: You will need at least 16 GByte, then add between 4 and 16 GByte for each VM. Windows VMs will take more RAM, LCXs can take as little as 1 GB. YOur avergae Linux VM will take 4 GB, a big Docker host with many containers may take 16 GByte. Do not underestimate the number of VMs you will eventually end up with. Think of:

- Proxmox Host itself (depending on file storage size 8-16 GByte)

- Home Assistant (8 GByte)
- Windows box for different purposes (16 GByte)
- Internal Docker Host (16 GByte)
- External Docker Host or Webserver (8-16 GByte)
- Proxmox Backup Server (4 GByte), also takes a lot of file storage
- Unifi Network Controller or UOS (4 GByte)
- Media Server Jellyfin or Plex (8-16 GByte)
- Centralized Log Server (e.g. Graylog) (4 GByte)
- OpnSense (8-16 GByte with no viable option of ballooning)

Add that up and you will find that 64 GByte is a good starting point, 128 GByte is better. 16 GByte will get you as far as 2-3 VM max.

Also, do not buy a VM SSD storage that is too small (i.e. less than 1-2 TByte) or too cheap (with ZFS, you will definitely need a "Pro" type of SSD) and plan for additional HDD storage capacity for backups and media in the multi-TByte range.

I'm so glad you posted this because I am also considering Ubiquiti for the home network. You mentioned the Unifi Network Controller. What is that and does that mean I won't have to buy a Ubiquiti router?

I am very new to this so please bear with me. Are you saying I should have at least two mini PCs with OPNsense being its own dedicated machine? In this case, how should I spec it in terms of RAM and SSD storage? I won't mind buying more RAM if the need is there.

Thanks for the ideas. I have zero experience with virtual machines. You call it a Proxmox machine. Are you saying I should install Proxmox directly to the mini PC and then use that to host other VMs like Windows? Unfortunately, the mini PC only has one spare SODIMM slot so the max I can use is 64GB, which apparently works despite Intel saying 16GB is the maximum. To be honest, I don't even know what Docker is and you mentioned internal and external. I don't know what Home Assistant is. How much storage do you think I need for the SSD with the various use cases that you mentioned?

Where should I install Adguard Home or Technitium?

N150 16GB ram and 256GB disk, just get the fastest stuff your device can use.
i226-v works a-ok with freeBSD.
Is it a WAN/LAN setup, or just LAN/LAN ?
You can have vlan tagging to support many vlan's on one interface, as long as your switch can do that too.

Build it and then see if you run into issues.

I intend to use it as a WAN/LAN setup.

I guess I'm familiar with the concept of a homelab, but I didn't know there was a term for it.

I'm also in the middle of researching and building a brand new home network. The most experience I have is with a GL.iNet router. I do have a sense of the kind of features that I want.

I have zero experience with OPNsense and so I'm not sure what kind of specs I'll need to have a good functional router. I already have an existing mini PC (N150 with 16GB of RAM and dual i-226 NICs) that I'm using with Windows, but will likely repurpose it as a router). I also want to get another barebone mini PC or maybe more in order to have a good homelab. I find these mini PCs to be incredibly powerful and power-efficient.

So I don't know all that I would want in a router. I know I value privacy and security so the OPNsense router needs to be as secure as possible. I'll be connecting them to an access point, which is likely to be an Asus RT-AC86U flashed with the Merlin firmware. I want to connect to multiple VPN servers simultaneously and then route traffic based on the device/app/domain. I also want to run either Adguard Home, which I have experience with, or even try out Technitium. Jellyfin or Plex is also what I want to run. What I'm not familiar with is Promox or Docker and how this will fit into my setup. From what I have read so far, it sounds like I'll need or want to use Promox and Docker.

Given my use case, is 16GB of RAM enough, or should I go for 32GB or more? Since this isn't a file server, I don't want more capacity on the SSD than I need to run these apps. What size do you think will be sufficient?

I'm reading that some people are virtualizing OPNsense? Is this recommended or should I just install it directly since this will be a dedicated router? Should I have a separate machine just for AdGuard Home and Jellyfin?

I've been using NextDNS for a while, but I'm thinking I might be better off hosting this myself. Is Adguard Home capable of being self-hosted and then when I'm out and using a cellular connection, I'll direct all DNS queries back to Adguard Home in my home network?

Looking forward to reading your responses. Thank you!