Messages

Thanks!

Yes I run redis, monit, flowd and normally ntopng (I don't use vlan, lag).

Over wifi I am not seeing the issue, so I am suspecting something between the realtek nic .251, opensense .254, my dock mac .3. I noticed that I had my mac set as static IP instead of DHCP. I changed this to DHCP to avoid any conflict.

I also changed the cable, and now things are good. So I will put back things like redis etc....

Hello Team,

I have deactivate Unbond, Netflow, Ntopng to reduce the load. But I still have the issue but without any idea of what to look for.

Attached is my VM conf. CPU is AMD Ryzen 7 9700X.

What log could I be looking into OPNSense host to see interrupts or local freezes on the guest?

Maybe an ARP/ IP conflict:
2025-01-08T17:19:45    Error    dhcpd    uid lease 192.168.30.197 for client 6c:7e:67:c5:5f:c1 is duplicate on 192.168.30.0/24
My laptop has Zscaler not sure if this could bring some strange behaviours but normal this mac has a static IP and should not be duplicated ...


Merci

Any idea of what could be the issue? Maybe a driver issue on proxmox of r8126 on 6.11 kernel?

FYI a capture from my laptop to Proxmox host through wire. I wonder if this has something to do with OPNSense but my girlfriend does have the same issue on Wifi.

Code Select Expand

dmesg | grep -i r8169
[    0.890543] r8169 0000:0a:00.0: enabling device (0000 -> 0003)
[    0.901128] r8169 0000:0a:00.0 eth0: RTL8126A, 34:5a:60:03:c4:ad, XID 64a, IRQ 58
[    0.901132] r8169 0000:0a:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[    3.160413] r8169 0000:0a:00.0 enp10s0: renamed from eth0
[   21.473988] r8169 0000:0a:00.0 enp10s0: entered allmulticast mode
[   21.474024] r8169 0000:0a:00.0 enp10s0: entered promiscuous mode
[   21.500368] RTL8251B 5Gbps PHY r8169-0-a00:00: attached PHY driver (mii_bus:phy_addr=r8169-0-a00:00, irq=MAC)
[   21.940489] r8169 0000:0a:00.0 enp10s0: Link is Down
[   24.815676] r8169 0000:0a:00.0 enp10s0: Link is Up - 2.5Gbps/Full - flow control off
root@Proxmox ~#

Hello All,

I have been using OPNsense 24.7.11_2 over Proxmox 8.3 6.11.0-2-pve and before pfsense from a while. I am facing an unstable issue which I can't find any log to help really to troubleshoot the issue. On calls, from time to time it hangs for like 2 to 3s and then keeps going.

Some logs, but what else can I be checking?

If I restart the services, I get:

Code Select Expand

Enter an option: 11

Writing firmware settings: FreeBSD OPNsense
Writing trust files...done.
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
certctl: No changes to trust store were made.
Writing trust bundles...done.
Configuring login behaviour...done.
Configuring CRON...done.
Setting timezone: Europe/Paris
Setting hostname: OPNsense.localdomain
Generating /etc/resolv.conf...done.
Generating /etc/hosts...done.
Configuring loopback interface...done.
Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring CAM interface...done.
Configuring Download interface...done.
Configuring LAN interface...done.
Configuring POP interface...done.
Configuring WAN interface...done.
Configuring WIFI interface...done.
Setting up routes...done.
Setting up gateway monitor...done.
Configuring firewall.......done.
Starting DHCPv4 service...done.
Starting DHCPv6 service...done.
Starting router advertisement service...done.
Starting NTP service...done.
Configuring OpenSSH...done.
Starting web GUI...done.
Syncing OpenVPN settings...done.
Stopping ntopng.
Waiting for PIDS: 54790.
Stopping redis.
Waiting for PIDS: 45839.
Stopping node_exporter.
Stopping acme_http_challenge.
Waiting for PIDS: 31589.
Stopping flowd.
Stopping mdns_repeater.
Waiting for PIDS: 19673.
Stopping qemu_guest_agent.
Waiting for PIDS: 15465.
Stopping monit.
Waiting for PIDS: 89582.
Stopping flowd_aggregate...done
setup vtnet1
setup vtnet0 [egress only]
setup vtnet2
Starting flowd_aggregate.
Starting monit.
Starting Monit 5.34.3 daemon with http interface at /var/run/monit.sock
kldload: can't load virtio_console: module already loaded or in kernel
Starting qemu_guest_agent.
Starting mdns_repeater.
Starting flowd.
rmdir: /var/etc/acme-client/home/deploy: Not a directory
rmdir: /var/etc/acme-client/home/dnsapi: Not a directory
rmdir: /var/etc/acme-client/home/notify: Not a directory
Starting acme_http_challenge.
Starting node_exporter.
Starting redis.
Certificates generated /usr/local/share/ntopng/httpdocs/ssl/ntopng-cert.pem
Starting ntopng.
md5sum: invalid option -- q
usage: md5sum [-bctwz] [files ...]
usage: grep [-abcDEFGHhIiLlmnOopqRSsUVvwxz] [-A num] [-B num] [-C num]
        [-e pattern] [-f file] [--binary-files=value] [--color=when]
        [--context=num] [--directories=action] [--label] [--line-buffered]
        [--null] [pattern] [file ...]
06/Jan/2025 15:02:22 [Ntop.cpp:4052] WARNING: Unable to find timezone: using UTC
06/Jan/2025 15:02:22 [Redis.cpp:171] Successfully connected to redis 127.0.0.1@0
06/Jan/2025 15:02:22 [Redis.cpp:171] Successfully connected to redis 127.0.0.1@0
06/Jan/2025 15:02:22 [Ntop.cpp:2642] Parent process is exiting (this is normal)

The client has disconnected from the server.  Reason:
Invalid packet header.  This probably indicates a problem with key exchange or encryption.
What I noticed, is that my client gets disconnected from the host when the issue appears is:

Code Select Expand

root@Proxmox ~# ping 1.1.1.1
64 bytes from 1.1.1.1: icmp_seq=858 ttl=57 time=9.94 ms
64 bytes from 1.1.1.1: icmp_seq=859 ttl=57 time=10.1 ms

The client has disconnected from the server.  Reason:
Invalid packet header.  This probably indicates a problem with key exchange or encryption.

Could this be an issue on Proxmox versus on OPNsense? is there any other log that could make sense to check on OPNSense before checking on Proxmox side?

Is it a key change happening on OPNsense all the time? something to do with the certificate?

Merci
XabiX

I used them when I need to know what is being blocked. I have kept the rules but removed all loging.

Merci

Is there a way to disable IPv6 on my interfaces outside of the POP? Before i never saw these assigned maybe this is linked to an improvement of the Interface Overview :)

I do have: IPv6 Configuration Type to None on those interfaces.

BTW I did remove: Log packets matched from the default block rules

but I still see those msg. I assume these are bc they are captured by my own deny all IPv6 rule?

Thanks Patrick!

My bad as I forgot that I have an internal interface which operates with IPv6 which is the media setup box of my ISP for TV and services like replay/netflix etc...

Therefore, if this is acceptable, should I allow this traffic just towards this LAN interface?

Is your comment also valid for the traffic towards udp 3702?

Merci

Hello Experts,

I don't understand why I am seeing this traffic and if I should either allow it or put a non verbose rule entry to stop it to fill the logs.

Besides I was trying WS and I see udp 3702 too blocked. My setup is IPv4 so not sure if I need those too.

Code Select Expand

[ndp -a
Neighbor                             Linklayer Address  Netif Expire    1s 5s
2a01:e0a:3ba:cb90::2                 92:f5:ca:c9:f3:92 vtnet0 permanent R
fe80::90f5:caff:fec9:f392%vtnet0     92:f5:ca:c9:f3:92 vtnet0 permanent R
fe80::9c90:88ff:fe48:d45b%vtnet1     9e:90:88:48:d4:5b vtnet1 permanent R
fe80::449f:54ff:fe80:6bf1%vtnet2     46:9f:54:80:6b:f1 vtnet2 permanent R
fe80::bc00:eeff:fe5d:31e3%vtnet3     be:00:ee:5d:31:e3 vtnet3 permanent R
2a01:e0a:3ba:cb91::1                 da:dc:fd:fa:f7:7c vtnet4 permanent R
fe80::b9a8:d032:e210:1c2a%vtnet4     dc:00:b0:44:74:64 vtnet4 23h56m0s  S
fe80::d8dc:fdff:fefa:f77c%vtnet4     da:dc:fd:fa:f7:7c vtnet4 permanent R
2a01:e0a:3ba:cb91:61da:fc7d:3083:ed4f dc:00:b0:44:74:64 vtnet4 23h56m0s  S
fe80::8db:32ff:feb9:b45c%vtnet6      0a:db:32:b9:b4:5c vtnet6 permanent R code]

[code]pfctl -s rules | grep "from fe80::/10"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type echoreq keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type echoreq keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routersol keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routersol keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routeradv keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routeradv keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbrsol keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbrsol keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbradv keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbradv keep state label "d147534c4012c8dd65eda59292c0ab90"
pass in quick on vtnet4 inet6 proto udp from fe80::/10 to fe80::/10 port = dhcpv6-client keep state label "202cde82e72bc8757ce87db904864c07"
pass in quick on vtnet4 inet6 proto udp from fe80::/10 to ff02::/16 port = dhcpv6-client keep state label "202cde82e72bc8757ce87db904864c07"
pass in quick on vtnet4 inet6 proto udp from fe80::/10 to ff02::/16 port = dhcpv6-server keep state label "fcfc7f20b012cb13daa2953a063f4f4e"
pass in quick on vtnet4 inet6 proto udp from fe80::/10 to (self) port = dhcpv6-client keep state label "a329a5ad6317f1c72757431e7a8232aa"
pass in quick on vtnet0 inet6 proto udp from fe80::/10 to fe80::/10 port = dhcpv6-client keep state label "4408d4bb3e3b231599822fa8f4546f8d"
pass in quick on vtnet0 inet6 proto udp from fe80::/10 to ff02::/16 port = dhcpv6-client keep state label "4408d4bb3e3b231599822fa8f4546f8d"
pass in quick on vtnet0 inet6 proto udp from fe80::/10 to ff02::/16 port = dhcpv6-server keep state label "3e5fbb29b91da43363e550aead699e16"
pass in quick on vtnet0 inet6 proto udp from fe80::/10 to (self) port = dhcpv6-client keep state label "43f521ff1b149fea894c4f31417849bb"
pass in quick on vtnet4 inet6 from fe80::/10 to ! (vtnet1:network) flags S/SA keep state allow-opts label "178c7c3c8c26cb8456b49510389dd6e3"/code]

Any help is more than welcome.

Merci

So it s not an ERROR to consider!

Noted and the ticket can be closed. My DNS issue was linked to the GW not available with dpinger but solved.

This thread can be closed. THANKS Franco !!!

Hello

After upgrade to OPNsense 23.7.1_3-amd64 I have internet issues and not able to get answers to my DNS queries. Not sure yet why as I have the floating rules there as I had in the past.

I did see this error, is there anything that I should do to solved it? I didn't had the opty to check with my TV and setupbox if it works.

Code Select Expand

2023-08-09T16:08:20 Error opnsense /diag_logs_settings.php: The command '/usr/sbin/daemon -f -p '/var/run/dhcpleases6.pid' '/usr/local/opnsense/scripts/dhcp/prefixes.sh'' returned exit code '3', the output was 'daemon: process already running, pid: 37599'
2023-08-09T16:08:20 Error opnsense /diag_logs_settings.php: The command '/usr/local/sbin/dhcpd -6 -user dhcpd -group dhcpd -chroot /var/dhcpd -cf /etc/dhcpdv6.conf -pf /var/run/dhcpdv6.pid vtnet4' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.4.3-P1 Copyright 2004-2022 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Config file: /etc/dhcpdv6.conf Database file: /var/db/dhcpd6.leases PID file: /var/run/dhcpdv6.pid There's already a DHCP server running. If you think you have received this message due to a bug rather than a configuration issue please read the section on submitting bugs on either our web page at www.isc.org or in the README file before submitting a bug. These pages explain the proper process and the information we find helpful for debugging. exiting.'
2023-08-09T16:08:17 Error opnsense /diag_logs_settings.php: The command '/bin/kill -'TERM' '17962''(pid:/var/dhcpd/var/run/dhcpdv6.pid) returned exit code '1', the output was 'kill: 17962: No such process'

Merci

I was able to solve the issue by changing from HTTP to HTTPs and select a certificate.

Nice and easy re-installation. Great work Franco and the team!

Hello

I upgrade from 23.1.11_1 (or whatever was the last release before 23.7) and I decided to export my config, reinstall and import my saved config.

OPNsense does work so I have internet etc... btu I can't login in the GUI anymore on the LAN (it does work on the WIFI interface thought). What log file could help? Any insight? could it be that Acme did not re issued the cert automatically?

Code Select Expand

<27>1 2023-07-31T16:48:19+02:00 OPNsense.localdomain lighttpd 18271 - [meta sequenceId="2"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.2308) server stopped by UID = 0 PID = 71338
<27>1 2023-07-31T16:48:19+02:00 OPNsense.localdomain lighttpd 71600 - [meta sequenceId="3"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1909) server started (lighttpd/1.4.71)
<27>1 2023-07-31T16:49:23+02:00 OPNsense.localdomain lighttpd 71600 - [meta sequenceId="1"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.2308) server stopped by UID = 0 PID = 78885
<27>1 2023-07-31T16:49:54+02:00 OPNsense.localdomain lighttpd 38726 - [meta sequenceId="1"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1909) server started (lighttpd/1.4.71)
<27>1 2023-07-31T16:49:56+02:00 OPNsense.localdomain lighttpd 55593 - [meta sequenceId="2"] (/usr/obj/usr/ports/www/lighttpd/work/lighttpd-1.4.71/src/server.c.1909) server started (lighttpd/1.4.71)

Code Select Expand

2023-07-31 16:48:24 [root:groupadd] acme(169)
2023-07-31 16:48:24 [root:useradd] acme(169):acme(169):ACME protocol client:/var/db/acme:/bin/sh
2023-07-31 16:48:31 [root:groupadd] git_daemon(964)
2023-07-31 16:48:31 [root:useradd] git_daemon(964):git_daemon(964):git daemon:/nonexistent:/usr/sbin/nologin
2023-07-31 16:48:44 [root:groupadd] _lldpd(949)
2023-07-31 16:48:44 [root:useradd] _lldpd(949):_lldpd(949):lldpd user:/nonexistent:/usr/sbin/nologin
2023-07-31 16:49:52 [unknown:groupmod] admins(1999)

Thanks
XabiX

Thanks all, I had the same issue and disabled IPS for now.

i would be happy to share my remote connexion ;)

Hello all,

I am dissappointed as I am facing also issue since the upgrade on IPv6. Not sure why but can t even get to ping ipv6.google.com.

root@OPNsense:~ # ping6 -I vtnet4 2a00:1450:400a:804::2004
PING6(56=40+8+8 bytes) 2a01:e0a:3ba:cb90::2 --> 2a00:1450:400a:804::2004

vtnet4: flags=8a63<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: POP
        options=800a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
        ether da:dc:fd:fa:f7:7c
        inet 192.168.0.254 netmask 0xffffff00 broadcast 192.168.0.255
        inet6 2a01:e0a:3ba:cb90::2 prefixlen 64
        inet6 fe80::d8dc:fdff:fefa:f77c%vtnet4 prefixlen 64 scopeid 0x5

Routing tables
Internet6:
Destination                       Gateway                       Flags   Nhop#    Mtu    Netif Expire
default                           fe80::72fc:8fff:fe6a:95d%vtnet4 UGS       6   1500   vtnet4
::1                               link#7                        UHS         1  16384      lo0
2000::/3                          fe80::72fc:8fff:fe6a:95d%vtnet4 UGS       7   1500   vtnet4
2a01:e0a:3ba:cb90::/64            link#5                        U           5   1500   vtnet4
2a01:e0a:3ba:cb90::2              link#5                        UHS         4  16384      lo0
fe80::%vtnet4/64                  link#5                        U           5   1500   vtnet4
fe80::d8dc:fdff:fefa:f77c%vtnet4  link#5                        UHS         4  16384      lo0
fe80::%lo0/64                     link#7                        U           3  16384      lo0
fe80::1%lo0                       link#7                        UHS         2  16384      lo0

traceroute6 to 2a00:1450:400a:804::2004 (2a00:1450:400a:804::2004) from 2a01:e0a:3ba:cb90::2, 64 hops max, 28 byte packets
1  2a01:e0a:3ba:cb90::2  3048.035 ms !A  3014.750 ms !A  2999.995 ms !A

2022-01-30T21:14:52   Error   opnsense   /system_gateways.php: ROUTING: setting IPv6 default route to fe80::72fc:8fff:fe6a:95d   
2022-01-30T21:14:52   Error   opnsense   /system_gateways.php: ROUTING: IPv6 default gateway set to opt3

interface FW allow all ipv4 and ipv6 to go out.

Any idea?

Merci