Messages

Dear all,

Recently, I had the opportunity to go to fiber Internet connectivity. Speed is great, I get almost 1GB/s symetrical on a PHYSICAL computer connected on the box. ;D

What a disapointment when I got (after tests and optimizations) only the half behind my firewall VM.
The whole virtualization is probably to blame with all added latency and overhead (driver, hypervisor, etc...), so I don't know how to solve that.

But as a solution, I have an unused physical machine that was a previous hypervisor with 32GB of RAM and a core I5 and this should be just fine for that job (please comment if you think the opposite)

My idea is to go from a single VM which offers now with backups and snapshots a lot of flexibility and reliability to an HA cluster between the new physical and the VM machines.
Performance loss in case of problem/maintenance is 100% acceptable but of course, not the config/connectivity loss because I've also a VPN tunnel to another location and family is connecting to my infra (yes it's a home lab! ;))

VM network config:
- 2 virtual NICs (VMXNET3, one for WAN, one for LANs
- the LANs interface is configured with multiple VLANs/subnets
- All default GWs are with IPs finishing with .1

I'd like to keep the .1 as default GWs, so this has to be moved to the virtual IPs. .2 and .3 are all reserved for that project on all LAN subnets.

I know already there is difficulties with drivers and stuff (doc speaks about the necessity to use LAGG) to do such a mixed setup, but knowing the above, where do I start?

Thanks in advance for you great help :)


Réfs:
https://docs.opnsense.org/manual/hacarp.html
https://docs.opnsense.org/manual/how-tos/carp.html

I have one install of 20.7 where I deliver the opnsense on the the other end of a tunnel as an additional DNS server (both have unbound with DNSSEC and DNS-over-TLS (port 853) configured) via DHCP, as the DNS is unreliable on this box since 20.7.

I assume the ISP for this box interfers with DNSSEC/DNS-over-TLS...

We might have something here... I've 2 OPNsense with an IPSEC tunnel between them. Both has their local Internet/DNS breakout, but I have a feeling that sometimes, something is not working properly in DNS resolution.

I hate putting something else as a workaround, but this might be my excuse to try out Pi-Hole, but I'm not sure yet if it can forward DNS resolution in a secure way... ;D

Console output during move/crash would be a good start. :)

Well that is the thing: there is no "crash", only network connectivity loss to WAN interface I presume.
When it happened, I could still access the console web page but WAN_GW was not reachable (can't recall the amount of loss, but it was high)

The VM is configured as follow :
2 NICs: one WAN, one LAN with multiple VLANs/interfaces

I'll try to reproduce the issue tonight when wife and myself are not working! ::)

Any advice for logging that properly?

Some more info:
- vSphere 7.0U1
- VMXNET3 cards on VDS
- multiple VLANs & interfaces
- no IPS activated

Hi there,

Don't know where to start with, but since I've upgraded from 20.1 to 207, it seems I have more and more strange issues (I got some unbound issues too, I had to disable DNSSEC temporarily)

Usually, my FW is on a "should run on this host" in order that my SIEM can capture the traffic. Today, I had to maintain some ESXi host and I vMotioned the FW a few times... it crashed twice! ??? None of the other VM suffered a network connectivity.

I've to mention this NEVER happened before with previous versions, I could vMotion dozen of times.

I couldn't see some issues because I didn't had to move the VM since I've upgraded.

Where do we start to troubleshoot that? ;)

To be a little fair the change log opening sentence seems a bit ambiguous.  I don't think it was intended that way but the first time I read it I thought it was saying there were other changes that weren't noted in the log.

Yep, me this morning: "yeah, finally a fix", snapshot, upgrade... and Meeeeeeh, revert snapshot! ::)

I might have read to fast, my bad! ;)

Just accept plugin situation at the top of the table.

Thanks as well, one issue less! ;)
(I knew about package, but I didn't know how to remove that red...)

Same for me (it's a VM, no VLAN tagging, interface firectly on Internet with public address)

I let the IPS without blocking mode for now. At the second you activate blocking mode, it crashs ;D

Monitoring on 20.1.5 was ok, now since 20.1.6 it is not working again ! :o

(So portal OK with manual entry, but not popping up automatically)

Well, for now, update to 20.1.5 did include one Unbound change (I can't figure out if there was a version change), but this fixed the issue.

I'll monitor it and do some further testing. thanks everybody so far! ;)

[EDIT:]

Hi there,

So I've some new info but still no solution! :P
What I can confirm: it is definitivelly a DNS resolution issue.

What I've tried:
- One rule, full access, no portal but FW as DNS server in DHCP: doesn't work
- Portal standard FW rules, portal activated but Google DNS in DHCP: work fine

Some more info about my setup sbout DNS:
- FW general option DNS servers are the ones from the LAN
- LAN DNS servers are forwarding to the FW. All clients/servers are using them as they are the domain controllers (via DHCP or fixed settings)
- I've Unbound activated on LAN and guest interfaces.
- Unbound settings are the following: DNSSEC actvated, transparent, no forward, transparent local zone, standard port 53 and those extra settings:

server:
forward-zone:
name: "."
forward-ssl-upstream: yes
forward-addr: 8.8.8.8@853
forward-addr: 8.8.4.4@853

I've crosscheck to see if whenever I try Internet on guest if something would be blocked, I see nothing on firewall logs.

I could use "direct" google DNS servers, but I wanted to have Unbound used for all interfaces.
Any clue ? Would that be a bug ? :o

EDIT: the portal only comes with google DNS ONLY IF I browse a web site that has been resolved before portal activation (so IP already in local DNS cache). If unknown, like affter a restart or on a new device, portal doesn't come.

First, thanks a lot for your time, I really appreciate. It took me some time to test everything, but here we are! ;)

ok i missed some of your comments..
you mean it was working before?
now it aint? but disabling captive portal makes your connection work?

Yes it was working at some point. And no, I can't make it work by disabling the portal. :-\

Like tong said it's important to double-check your firewall rules (make sure the allow rules takes precedence aka above the deny rules), You can troubleshoot Firewall rules by going to Firewall> log files> Live view and type in the filter 8001 (or whatever port you want to filter) and check whether it's being blocked or denied (red) or allowed (green), you can also use ".*" for advanced filter ex: 192.168.xxx.1.*8001 to see all the rules associated with IP 192.168.xxx.1 on port 8001.

Well, I've crosschecked again, except one LAN more I'm blocking, the rest sticks to tutorial.

my captive portal is accessible at http://192.168.XXX.1:8001 (not 8000 as I saw almost everywhere)
this is probably due to captive portal zone number ( 0->8000, 1->8001, etc) you can check which zone your captive portal has by clicking edit and check the zone number (maybe after deleting the test CP it wasn't removed from the cache).

try adding your DNS server in the allowed address in captive portal configuration, I believe some people reported that the CP worked after doing so

After removing the DB and also recreated the portal from scratch, the new portal comes on 8000 (zone 0)
I've tried to add the DNS server in the allowed addresses  it didn't change the result.

When I look at FW logs, I see an allowed incoming for DHCP address request, an outgoing ICMP to the client and some allowed DNS queries at first connection but nothing is blocked even if everything points out to a non-working DNS resolution.
I've checked that as well, Unbound is linked to all the necessary interfaces (my 2 LANs and the guest), both LAN work fine. So it is still weird, I'm missing something there...

Any clue? Might not be the portal itself after all! ::)

Cool :)

Thanks to all, that answers my question !

Anyone ? :P

Hi there,

In order to be able to use DNS over TLS, I've looked arround and found those advanced options for Unbound :

server:
forward-zone:
name: "."
forward-ssl-upstream: yes
forward-addr: 8.8.8.8@853
forward-addr: 8.8.4.4@853

The only question is : whenever the advanced option field will be removed, how can we achieve the same result ?  ;)