Messages

I am having a strange issue. I was running 25.7.4 without any issues. I applied 25.7.5 on 10th of October. On 11th morning at around 5:23 AM, all network went offline. I tried to connect to OPNsense but could not. Since I run OPNsense as a VM in Proxmox, I noticed that the OPNsense VM was suddenly showing 90%+ CPU. I could not even connect to console. I tried to open serial console, but it was also non-responsive. I had to force start the VM.

Same thing happened on 12th and 13th October. On 13th October, I reverted to 25.7.4 snapshot and observed. No issues with CPU on 14th, 15th and 16th.

On 16th, I reapplied 25.7.5. It worked on 17th but today, on 18th morning at 5:40 AM, OPNsense VM again suddenly went to 90% CPU and became unresponsive. The VM had to be force restarted.

After applying 25.7.4 snapshot on 13th, I had created a script to write out CPU usage to a file every 2 minutes between 4:00 AM and 6:00 AM. Scheduled it through cron in OPNsense. This morning, after I recovered the VM from non-responsive state, when I check the CPU usage log, there was no entry after 5:40 AM. OPNsense was so badly hung that it did not even run the cron job. The CPU usage entries started at 5:50 AM after reboot. To summarize:

25.7.5 applied on 10th.
11th : CPU 90% and OPNsense unresponsive at 5:23 AM
12th : CPU 90% and OPNsense unresponsive at 5:00 AM
13th : CPU 90% and OPNsense unresponsive at 5:00 AM

25.7.4 snapshot restored on 13th.
14th : no issue
15th : no issue
16th : no issue

25.7.5 re-applied on 16th.
17th : no issue
18th : CPU 90% and OPNsense unresponsive at 5:40 AM

The high CPU always happens between 5:00 AM and 6:00 AM. I did not make any change to OPNsense configuration for this test. This rules out everything except 25.7.5 as the source of problem. What can I look at to find out what is causing this behavior?

Thanks

The subject says it all. I want to thank the OPNsense developers for coming out with a very stable update. After updating to 25.7.2, OPNsense is running very stable. I am a home user with very simple network. But I have come to rely upon OPNsense in more ways than I could imagine as I explore new features. It has become an essential part of my technological life. So, thank you, thank you, thank you...!!

Thanks you for this theme. I have tried several dark themes for OPNsense but they all made one page or the other unreadable. This theme is very good. All the pages are readable.

I was running OPNsense 25.7.1 without any issues. Then I saw that a hotfix 25.7.1_1 was released.

Before applying the hotfix, I took a snapshot. After applying hotfix, the CPU immediately went to 100% and I found that Caddy Domains widget failed to load. Slowly other widgets also showed "Widget failed to load". I watched overnight but same behavior. I reverted back the snapshot which should have taken the system back to 25.7.1, but for some reason, dashboard is showing 25.7. However, CPU is back to normal and all widgets are loading. I tested applying the 25.7.1_1 hotfix twice and both time, CPU went to 100% and stayed there for extended time with widgets failed to load error on Caddy Domains and few other widgets.

Without applying 25.7.1_1 hotfix, the Dynamic DNS is showing Current IP and Updated columns blank. These columns were showing correct values prior to 25.7 update. So 25.7 broke Dynamic DNS client. 25.7.1_1 fixed it but broke lot of other things.

Just mentioning it in case someone else has seen the same issue.

Honestly, I do not understand how you cannot see the timestamps of your configuration backups. For me, the backups are named like "config-OPNsense.xxx.yy-20250718013839.xml", so I see both the timestamp and the hostname of the device I backup.

root@hostname:~ # cd /conf/backup/
root@hostname:/conf/backup # ls -l
total 290
-rw-r-----  1 wwwonly wheel 191855 Jul 17 12:38 config-1752770332.5005.xml
-rw-r-----  1 root    wheel 192305 Jul 17 12:56 config-1752771392.6376.xml
-rw-r-----  1 root    wheel 192324 Jul 17 19:30 config-1752795049.2193.xml
root@hostname:/conf/backup #

In Google drive backup, the files are named as:

hostname-1751692538.xml
hostname-1751780029.xml
hostname-1751953779.xml

No date/time in the file name...

In case you run Opnsense on ZFS you could simply create a snapshot of the system before upgrading. Then you should be able to revert to the previous config easily.
I do that and additionally since my Opnsense runs on top of a Proxmox machine I use the backup feature of Proxmox to backup and recover.

Since the configuration file itself got corrupted, most likely because of the picture, snapshot would not have helped. It was NOT because of update to 25.1.11. If only I had rebooted the firewall after uploading picture file, I would have known about it right away.

Here is the sequence of events:
a) I uploaded picture file into OPNsense. Most likely, this caused configuration file to get corrupted. I did not restart the firewall as there was no need to.
b) Three days later, I updated to 25.1.11 which caused firewall restart and it failed to read the configuration file. If I had taken a snapshot prior to 25.1.11 update, (which I forgot to take), it would have restored the corrupted configuration file resulting in the same issue.

I use the Nextcloud backup plugin for automatic backups and OPNsense uses the hostname and the current date and time to create the filename. We backup all our OPNsense firewalls (8 in total) to the same Nextcloud account and the files are easily manageable.

I should not have to use a separate software to backup. This morning, when OPNsense switched to a default configuration, most devices on my local network also became inaccessible. OPNsense was down for like 4-5 hours. During this time, the devices which acquire their reserved DHCP IP addresses from OPNsense, all acquired some random IP and went off the network. No LAN access, no WAN access. Is it possible to restore from NextCloud without LAN/WAN access?

OPNsense is the hub of networking, at least for home user like me. When it fails, lot of things break. I have created my own procedures for restoring from backup without LAN/WAN access and I test them multiple times a year. I run OPNsense on a dedicated fanless hardware. In case that fails, I also have a VM as a backup which I can bring up just by switching cables and restoring the most recent backup. All tested and documented for my own use.

The problem is that I find it very difficult to identify the backup date/time just by looking at file names. This is a critical omission in OPNsense.

Also, I was not expecting that three backups would be corrupted because I uploaded a picture in OPNsense.

> This is something OPNsense developers should look at. I used a built-in feature which should not have caused any issues.

Fair, just need steps to reproduce please.


Cheers,
Franco

Steps to reproduce are simple:

Go to System:Settings:General and there is a Picture option. Choose a large picture file on the PC and upload. Then reboot the firewall. Most likely, it will fail to reboot as config.xml could not be read.

The issue is resolved. I was able to restore a slightly older backup. But here is what I noticed:

The size of backup file, both local and uploaded to Google drive had suddenly increased from 4MB to 15MB over last couple of days.
When I tried to restore the 15MB file, it could not be restored, either locally from backup or from Google drive.
When I tried to select the 15MB backup file from webGUI, it threw an error saying that file could not be parsed or file could not be uploaded or file could not be restored.

So, I restored the 4MB file from local backup and it worked without any issue.

I was racking my brain to see what caused increase in file size.
Then I remembered that as I was going through some pages in webGUI, I noticed a Picture option under System:Settings:General. Just to test what it does, I uploaded a picture. That is what caused the configuration to go bad. Since OPNSense could not load the configuration, it switched to some default configuration. The backups also became unrestorable.

To test, I removed the picture sections from the backup xml file and the file size was reduced to 4MB.

This is something OPNsense developers should look at. I used a built-in feature which should not have caused any issues.

Also, I find the OPNsense backups system to be pretty crappy. It does not append date and timestamp to the backup files. So, there is no way to tell which file was created when. There is no way to verify a backup file unless I do a restore.

OPNsense is adding features but backup and restore of configuration reliably should be topmost.

I happened to login to OPNsense this morning and noticed that there was a pending update. I applied the update and everything broke. My interface names changed to some some different names (bge0, bge1 etc.), total loss of internet connectivity. I was able to SSH into OPNsense and rename the interfaces to igc0 and igc1. This restored internet connectivity, but all other configuration is totally lost. It is like the firewall has been factory reset.

I am trying to restore from backups but the config.xml files seems to revert to default. Everything is down. I don't know what to do. Is there a way to revert back to an old version and restore backup?

Hi,

I have setup Caddy exactly as described in OPNsense documentation.
I changed the OPNsense https port to 8443, checked the Disable web GUI redirect rule.
I have created the WAN and LAN rules for This firewall destination.
Enabled Caddy, supplied e-mail, Auto HTTPS is ON.
Domain is https://test.duckdns.org. Certificate is Auto HTTPS.
Handler: Domain is https://test.duckdns.org, Directive: reverse_proxy
Upstream:
Protocol: http://
Upstream domain: 192.168.250.162
Upstream port: 2283
Access list private_ipv4 created as per documentation and added to the domain access list

OPNsense runs on a dedicated hardware box, WAN and LAN interfaces only. Dynamic DNS is configured in OPNsense to use DuckDNS. I can do nslookup test.duckdns.org and it resolves to my WAN IP address, no problem. If I access https://test.duckdns.org from within my local network, it can access the server on 192.168.250.162:2283 without any issues. But when I access from internet, I get the error:

net::ERR_HTTP2_PROTOCOL_ERROR

The Caddy file is:

# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


# caddy_user=root

# Global Options
{
   log {
      output net unixgram//var/run/caddy/log.sock {
      }
      format json {
         time_format rfc3339
      }
   }

   servers {
      protocols h1 h2
   }

   email xyz@gmail.com
   grace_period 10s
   import /usr/local/etc/caddy/caddy.d/*.global
}

# Reverse Proxy Configuration


test.duckdns.org {
   @c1ac6e21-c93d-461e-8546-246789993f33_testduckdnsorg {
      not client_ip 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8
   }
   handle @c1ac6e21-c93d-461e-8546-246789993f33_testduckdnsorg {
      abort
   }

   handle {
      reverse_proxy 192.168.250.162:2283 {
      }
   }
}

import /usr/local/etc/caddy/caddy.d/*.conf

I have done Google search but nothing. I have read all 21 pages of this thread but no matching issue. Caddy is getting certificate from Lets Encrypt. Any help would be appreciated.

OPNsense 25.1.9_2-amd647
os-caddy 2.0.1

Thanks,
Arun

Update: Issue resolved. The documentation is not good. It did not say when to stop creating reverse proxy. It went to configure access list for local access only. I did that and it blocked access from internet. I had to remove the access list from the domain and then reverse proxy started to work from internet. The documentation of Caddy needs to be revised as some options do not exist in OPNsense. It also needs to be made more descriptive to describe so many other options available.

My network is dead simple. No VLAN, no guest network, just one LAN interface /24 subnet, no IPv6, one WAN interface. I have switched to Dnsmasq and Unbound combination as per the documentation example.

https://docs.opnsense.org/manual/dnsmasq.html#configuration-examples

See what happens...

Thanks for all the wonderful replies and clarification !!!

Yes.

Both.

So do I need to configure both kea and Dnsmasq just to replace ISC DHCP?

Never mind, I found the answer in docs. Dnsmasq is recommended for small networks and is the most recent Opnsense offering. I will switch to using it as it seems to be a long term option.

Is ISC DHCP going away? What is the replacement? kea or Dnsmasq?

Thanks

The IPv6 Configuration Type for LAN is the problem. These are the options offered:

Static IPv6 <- I do not have a static IPv6 for LAN interface. I tried assigning some random IPv6 IP address and Matter over Thread devices stopped working.

DHCPv6 <- I do not have DHCPv6 setup for LAN interface, the IPv6 address is provided by the Thread Border Router. If I turn on DHCPv6 on LAN, the Matter over Thread devices stop working.

SLAAC <- I don't even know what this is
PPPoEv6 <- Don't have PPPoE, I have Verizon FIoS
6rd Tunnel <- No idea what this is
6to4 Tunnel <- No idea

Track Interface <- This is what I have setup currently. LAN interface tracks the WAN interface for IPv6. IPv6 is enabled on WAN interface even though not supported by ISP. If I disable IPv6 on WAN interface, this option gets disabled and Matter over Thread devices stop working. But enabling this option floods the logs with the error messages that I posted.

IPv6 is very confusing and I have spent lot of time trying to understand it, but no luck. So, I really need help.

Thanks...