Messages

So far I appear to have always been able to get things working by randomly changing the settings, but today I have taken my first steps in Wireshark :).

But first I have made significant changes to simplify my setup as much as possible:
* I removed all custom routes and gateways (at least temporary)
* I updated to OPNsense 24.1
* I migrated dhcpd, unbound and ntpsec on my Rpi to Kea, unbound and chrony on OPNsense
* I have disabled all multicast optimizations in my Unifi Network application (WiFi and overall network) as well as in my central Cisco switch.

So now all network services are running in OPNsense and I have disabled the Rpi (temporarily).

Wireshark on my laptop (wlan0, same WiFi as my android phone) picks up these 6 "IPv6 Multicast" messages every 300-500 seconds:

Code Select Expand

No. Time Source Destination Protocol Length Info
43 1862.997666324 fe80::20d:b9ff:fe45:cc09 ff02::1 ICMPv6 190 Router Advertisement from 00:0d:b9:45:cc:09
44 1862.998941542 fe80::3354:e5e9:84ba:9992 ff02::1 ICMPv6 86 Neighbor Advertisement 2a02:xxxx:xxxx:xxxx:2e1f:1791:743d:a738 (ovr) is at 5c:e4:2a:d0:a5:2f
45 1863.011271674 fe80::3354:e5e9:84ba:9992 ff02::16 ICMPv6 110 Multicast Listener Report Message v2
46 1863.914649666 fe80::3354:e5e9:84ba:9992 ff02::16 ICMPv6 110 Multicast Listener Report Message v2
47 1864.000173244 fe80::3354:e5e9:84ba:9992 ff02::1 ICMPv6 86 Neighbor Advertisement 2a02:xxxx:xxxx:xxxx:2e1f:1791:743d:a738 (ovr) is at 5c:e4:2a:d0:a5:2f
48 1865.001346469 fe80::3354:e5e9:84ba:9992 ff02::1 ICMPv6 86 Neighbor Advertisement 2a02:xxxx:xxxx:xxxx:2e1f:1791:743d:a738 (ovr) is at 5c:e4:2a:d0:a5:2f

The Router Advertisement packet contains:

Code Select Expand

Frame 43: 190 bytes on wire (1520 bits), 190 bytes captured (1520 bits) on interface wlan0, id 0
Ethernet II, Src: PCEngines_45:cc:09 (00:0d:b9:45:cc:09), Dst: IPv6mcast_01 (33:33:00:00:00:01)
Internet Protocol Version 6, Src: fe80::20d:b9ff:fe45:cc09, Dst: ff02::1
Internet Control Message Protocol v6
    Type: Router Advertisement (134)
    Code: 0
    Checksum: 0x69bf [correct]
    [Checksum Status: Good]
    Cur hop limit: 64
    Flags: 0x00, Prf (Default Router Preference): Medium
        0... .... = Managed address configuration: Not set
        .0.. .... = Other configuration: Not set
        ..0. .... = Home Agent: Not set
        ...0 0... = Prf (Default Router Preference): Medium (0)
        .... .0.. = ND Proxy: Not set
        .... ..00 = Reserved: 0
    Router lifetime (s): 600
    Reachable time (ms): 0
    Retrans timer (ms): 0
    ICMPv6 Option (Prefix information : 2a02:xxxx:xxxx:xxxx::/64)
    ICMPv6 Option (Route Information : Medium 2a02:xxxx:xxxx:xxxx::/64)
    ICMPv6 Option (Recursive DNS Server 2a02:xxxx:xxxx:xxxx::1)
    ICMPv6 Option (DNS Search List Option <domain>)
    ICMPv6 Option (MTU : 1500)
    ICMPv6 Option (Source link-layer address : 00:0d:b9:45:cc:09)

Well, yes - I know that I'm failing the "be specific" rule, but was hoping to get some high level tips what to test next. I'm afraid we're getting very close to setting up wireshark, which I am not at all familiar with, but if that's what it takes, then that's what I'll do.

Maybe you do see something off in this?

Code Select Expand

LAN:
Status up
MAC address 00:0d:b9:..:..:.. - PC Engines GmbH
MTU                 1500
IPv4 address 192.168.2.1/24
IPv4 gateway auto-detected: 192.168.2.14
IPv6 link-local fe80::20d:b9ff:fe45:cc09/64
IPv6 address 2a02:....:....:902::1/64                           <-- virtual IP
                        2a02:....:....:902:20d:b9ff:fe45:cc09/64
IPv6 gateway auto-detected: 2a02:....:....:902::14
Media         1000baseT <full-duplex>
(The gateway for both IPv4 and IPv6 do not make sense, as that device is a server on my LAN - but if I add gateway addresses for the OPNsense device and give those a higher prio, then my internet connectivity fails for all machines on my LAN)

Code Select Expand

WAN:
Status up
DHCP         DHCPv4 up        DHCPv6 up   
MAC address 00:0d:b9:..:..:.. - PC Engines GmbH
MTU                 1500
IPv4 address 81.82.aaa.aaa/18
IPv4 gateway auto-detected: 81.82.192.1
IPv6 link-local fe80::20d:b9ff:fe45:cc08/64
IPv6 address 2a02:xxxx:xxxx:80e1:cc89:fc32:3be2:44db/128
IPv6 prefix 2a02:....:....:900::/56
IPv6 gateway auto-detected: fe80::217:10ff:fe2b:3173
DNS servers 195.130.130.2
                        195.130.131.2
                        2a02:1800:100::42:2
                        2a02:1800:100::42:1
Media         1000baseT <full-duplex>

I found some online reports related to multicast & IGMP settings, so I disabled all those functions in my Unifi Network application as well as in my Cisco central switch. I also reduced the Group Rekeying Interval and restarted my access points as some Unifi user reported very similar symptoms as mine: IPv6 connectivity seems to be working, but it is not.

Thank you again, Maurice. It looks like I've been spending way too much time down the wrong rabbit hole.

So, apparently, my IPv6 setup as such is fully functional, but then the new question is "Why are all apps on my android phone, except for the browser, timeing out or failing once I enable WiFi?"

I have been monitoring my firewall logs to see if anything is being blocked, and apart from a lot of Geo-related blocking I cannot see anything of relevance in the logs... On top of the automatically generated firewall rules to allow certain ICMP traffic, I have added a floating rule to allow *all* ICMP traffic.

Can you recommend me what to do next?

Hi, I have been struggling for two weeks now to get IPv6 to work. The initial problem was with my provider, and now I seemingly have everything working, but most apps on my android devices appear to fall back to IPv4 after 30s of trying.

The test at https://ipv6-test.com/ results in a score of 18 or 19/20, even on my android 14 phone, but the 'SLAAC' option always results in a (green) 'NO'. My devices do report having a fe80:...ff:fe... address, which I believe is formatted like a SLAAC address.

I have a very simple network, with 1 single subnet and no VLANs. My ISP assigns me a fixed /56 block, so I have set:
* WAN IPv6 to DHVPv6
* LAN IPv6 to 'Track interface: WAN'
* Assigned a fixed Virtual IP to my LAN interface aaaa:bbbb:cccc:dddd::1/128 -> now changed to /64 (thx to Maurice)

These are the options I have tried, without success:

* "Automatic mode":
      * simply disable the "Manually adjust router advertisements in the Track Interface section" - IIRC, Maurice posted on this forum that OPNsense then automatically advertises SLAAC.
      * disable radvd and DHCPv6 on the LAN interface
* "Unmanaged mode":
      * enable "Manually adjust router advertisements in the Track Interface section"
      * enable radvd daemon and configure it as Unmanaged
      * disable radvd and DHCPv6 on the LAN interface
* "Assisted mode" / "SLAAC mode":
      * enable "Manually adjust router advertisements in the Track Interface section"
      * enable radvd daemon and configure it as Assisted
      * enable the DHCPv6 server on my LAN interface (tried with and without setting a dhcp range)

In the radvd service section, I have tried with and without setting a prefix - if I did, I made sure that it was a /64 prefix.
I have used radvdump on my laptop to test the RA's I receive, both on my network and a similar network in my neighbourhood (only difference being OPNsense versus Unifi Router), and have been able to align the RA's perfectly.
I have tried running a radvd daemon on a RaspberryPi, making sure that that one did not advertise itself as the router/gateway on the network.

My questions:
1. I always restarted the services after changing settings - but I did not always restart OPNsense entirely - should I?
2. Where else can I look or what else can I try (in OPNsense, I think) for relevant settings?

Although far from a full-blown tutorial, I managed to get this to work earlier this week. These are the steps I took:
https://forum.opnsense.org/index.php?topic=38224.msg187483#msg187483

Just a quick note that everything is working, both outbound and inbound.

The main issue was resolved once I had access to a functional admin panel to change the DUID in my provider's records. After one more reboot I indeed received the correct /56 prefix.

Then setting up my internal network was a bit more involving than some online tutorials had led me to believe:
* I set the LAN interface to "track interface: WAN"
* I set the option "Request only prefix"
* I enabled the DHCPv6 server on the LAN interface
* I enabled the radvd service on the LAN interface
* I set a virtual IP on my LAN interface.

Thank you again!

Well, I'm a whole lot further now, and maybe the problem has been solved - thank you for your help! Without it, I would still be focusing on errors made by my provider...

I am able to successfully run several online IPv6 tests from my laptop, but the ping from the Virtual IP as you instructed is failing. I also don't seem to be able to reach a server on my LAN via its IPv6 address, so I need to test some more.

Since I could finally manage my 'Fixed IP address' settings in the management panel offered by my provider, I decided to just choose another DUID, update both on my provider's side and in OpnSense and then reboot OpnSense. That immediately fixed me getting the wrong IPv6 prefix. Since I had rebooted my appliance a few times already, I'm blaming their cache settings.

I went back to the most basic settings imaginable:
* WAN: DHCPv6 + prefix size set to 56 + request only prefix
* LAN: track WAN interface + disable 'manual adjustment'
* Added a Virtual IPv6 to my OpnSense appliance: 2a02:xx07:1020:902::1/128

I also tried with these additional settings, but they didn't make a difference:
* Added a Gateway to 2a02:xx07:1020:902::1
* Added and enabled a route in OpnSense to route all 2a02:xx07:1020:900::/56 traffic through this gateway

So, now all clients on my LAN receive IPv6 addresses within the IPv6 prefix that is allocated to me by my provider. But still I fail on all online IPv6 tests. I did restart the networking components on my laptop (iwd and dhcpcd) and even switched browsers.

Any idea?

Thank you so much for your fast reply. I appear to have some reading to do before scouring my provider :).

I don't know whether I will receive a fixed IPv6 address or prefix, but I would assume that the prefix is allocated to me exclusively, so I can reach my server.

I have indeed tried setting manually an address in the prefix:
My radvd configuration (running on a rPi on my LAN but I also tried with the radvd service on OpnSense):

Code Select Expand

interface eth0 {
  AdvSendAdvert on;
  MinRtrAdvInterval 3;
  MaxRtrAdvInterval 10;
  prefix 2a02:xx07:1020:902::/64 {
    AdvOnLink on;
    AdvAutonomous on;
    AdvRouterAddr on;
  };
  RDNSS 2a02:xx07:1020:902::15 {};

My laptop's ip addresses:

Code Select Expand

3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5c:e4:2a:d0:a5:2f brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.197/24 brd 192.168.2.255 scope global dynamic noprefixroute wlan0
       valid_lft 6778sec preferred_lft 5929sec
    inet6 2a02:xx12:1c0c:4802:708b:871b:556e:907e/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 86395sec preferred_lft 14395sec
    inet6 2a02:xx07:1020:902:2e1f:1791:743d:a738/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 86400sec preferred_lft 14400sec
    inet6 fe80::3354:e5e9:84ba:9992/64 scope link
       valid_lft forever preferred_lft forever

My laptop's ip routes:

Code Select Expand

default via 192.168.2.1 dev wlan0 proto dhcp src 192.168.2.197 metric 3003
10.66.71.0/24 via 192.168.2.1 dev wlan0
192.168.2.0/24 dev wlan0 proto dhcp scope link src 192.168.2.197 metric 3003

2a02:xx07:1020:902::/64 dev wlan0 proto ra metric 3003 pref medium
2a02:xx12:1c0c:4802::/64 dev wlan0 proto ra metric 3003 mtu 1500 pref medium
fe80::/64 dev wlan0 proto kernel metric 256 pref medium
default via fe80::7e5b:6b13:4b53:6f84 dev wlan0 proto ra metric 3003 pref medium

My laptop's resolv.conf:

Code Select Expand

# Generated by resolvconf
domain [my domain]
nameserver 192.168.2.15
nameserver 2a02:xx07:1020:902::15
options trust-ad edns0 single-request timeout:1

If I run the tests at https://test-ipv6.com, these are the results:

Code Select Expand

Test with IPv4 DNS record ok (0.952s) using ipv4
Test with IPv6 DNS record bad (0.785s)
Test with Dual Stack DNS record ok (1.137s) using ipv4
Test for Dual Stack DNS and large packet ok (0.168s) using ipv4
Test IPv6 large packet bad (0.554s)
Test if your ISP's DNS server uses IPv6 ok (2.645s) using ipv4
Find IPv4 Service Provider ok (0.648s) using ipv4 ASN 6848
Find IPv6 Service Provider bad (0.793s)

Apart from an internal DNS server, I don't have any other network-related services running on this LAN. The DNS server forwards all traffic to DNSSEC servers, and serves a few local hosts/addresses with A, AAAA and PTR records.

/edit: added resolv.conf and ip routes

[xxxf]

Hi,
Short and maybe stupid question, but my service provider is ignoring my requests to solve my issue (and I am not entirely sure that my configuration is not the culprit).

A few months ago, I requested a fixed IPv6 address and shared the DUID of my OpnSense appliance. The provider confirmed that all was set up correctly on their end. However, the IPv6 address that my OpnSense appliance receives through DHCPv6 is outside of the block that is supposed to be reserved for me, and it does not match the IPv6 prefix that OpnSense tells me it has received:

OpnSense interfaces overview:
IPv6 link-local   fe80::20d:b9ff:fe45:cc08/64
IPv6 address   2a02:xxxf:0:80e1:b032:6391:eea0:89df/128
IPv6 prefix   2a02:xxx2:1c0c:4800::/56
IPv6 gateway   auto-detected: fe80::217:10ff:fe87:b386

-> 1. The address is outside of the prefix
-> 2. The address actually has changed already at least once, so whatever they have issued to me, it is not a fixed IPv6
-> 3. The prefix is not the same as the one I was told I would receive (2a02:xxy7:1020:900::/56)

I have already released/renewed the IPv6 and IPv4 addresses, and have also rebooted twice. I have asked my provider to double check the modem ID in their systems (/my account), and they always confirm everything is set up correctly.

I can make outbound things work (as confirmed by an ipv6test website) by adding a virtual IP address inside the prefix and manually setting up a route, but inbound, computers on my LAN are not reachable from outside with that setup.

My configuration is pretty standard, I believe:
1. the WAN interface is configured to use DHCPv6, with 'send prefix hint' set and 'prefix delegation size' set to 56
2. the LAN interface is set to track the WAN interface, and I have tried both with the option to manually adjust RA and without.

Is there anything I could be doing wrong, or can I firmly demand a solution from my provider?

I believe I can chime in: I enabled the 20.7 update in the GUI, then waited until some internet-dependent tasks had completed (20-30 minutes), and then clicked the 'start update' button.

An icon started spinning and I continued working on my laptop. Suddenly my Wi-Fi connection disappeared, and when I checked back on the installation status, the icon was still spinning.

Using a usb-to-serial modem cable, I was able to see a lot of gibberish and some incomplete words that sounded a lot like 'segmentation fault - core dump'. I reinstalled using a new USB image and imported the config. First update in 3 (/4?) years that goes wrong. Maybe it can be attributed to my session being broken at the moment I clicked 'start update'?

I am still puzzled why my access points (Ubiquiti) suddenly stopped accepting connections - If my gateway/firewall is down, I would expect the internal network to stay up  :-\

Kr,
Vincent

I have enabled the IDS + IPS service on the WAN interface yesterday with the 'Aho-corasick'-patternmatcher. I then have enabled / downloaded a few rulesets, amongst which all the Abuse.ch ones, and the ET Malware, ET Mobile Malware and ET Exploit. All rulesets have been edited to 'drop' the packets. Tonight a first CRON run has been succesfully executed to download the newest versions of these rulesets.

This morning I am still able to download and open the EICAR test file and see no alert in my alerts.

I just tried it again with the opnsense-test-rules, but still nothing is blocked or alerted.

Please tell me what you need and I'll happily supply it to you.

Kr,
Vincent

I had some issues while setting up my OPNsense router with NAT, and after I had solved the base issues with my internal network, I couldn't get to work the simplest of NAT rules. I solved the issue by removing the NAT rules, Saving the changes and creating a new NAT rule (and including the option to 'add a firewall rule').
HTH

Dear all,

First of all, thank you for the OPNsense software - I have it more or less up and running by now, but as I have had some issues, I'd like to document them here. Maybe in my ignorance I have stumbled across a bug or two, and who knows maybe this leads to the bugs being squashed.

Secondly: The reason why I ordered a new PC Engines APU2C4 board and chose OPNsense is that I thought my Linksys EA3500 router had started dying on me: I received complaints of frequent drops in connections (http and sip), but today I must admit now that it seems that these drops have been caused by internal DHCP-issues: one of my devices used a fixed IP that was conflicting with a statically attributed address. Now this has been resolved, I don't see the connection drops anymore, so I hope I can mark the issue solved :).

My setup: MODEM - ROUTER (only NAT) - SG200 SWITCH - internal network, including a Debian Jessie server with IPTABLES, FAIL2BAN, DHCPD and BIND9.

In this setup, changing routers should be very straightforward, but it proved not that straightforward:
1. Installation was painless (once I had the correct null modem cable) with the 'serial'-image on a USB stick and installation to an mSATA disk
2. Configuration of the Intel ethernet ports was easy: igb0=WAN=DHCPv4 + igb1=LAN=fixed IPv4
3. In the webgui, I spoofed the former Linksys MAC Address to receive my fixed IPv4 address on WAN
4. Again in the webgui, I setup port forwarding for ~15 destinations, carefully leaving the 'bind to firewall rule'-option unchanged
5. I setup disk cache as per the instructions

Still thinking I had solved the issue with the dropped packets, I was very satisfied: the old router configuration (NAT only) was manually copied to the new device, and everything was accessible. At that moment, I witnessed only:
* a constantly high CPU load (30%+, although a max of 5 low-traffic clients were active on the LAN)
* a hanging JS-script when opening the "Interfaces > LAN" -page
* the NTPD service failed quite often

24 hours later, the connection drops reared their ugly head again, and I moved the old router back into place, which seemed to solve the issue in the short term. Yesterday I found the issue, but I had made a few changes to the router:
1. I moved all DHCPD-rules to the OPNsense router and disabled the DHCPD on the Debian Jessie server
2. I reset the SG200-switch as well as some other switches
3. I unplugged a lot of cables from the switch
4. I removed the igb0-interface and moved WAN to igb2, to move it back 30 minutes later
5. I stopped spoofing the MAC address of the old router
6. I tried to login into all the managed switches and resolved the dual use of one IP address for two devices
7. I stopped the Intrusion Detection and WebProxy services
8. I rebooted the modem and the router

Then suddenly,
* the CPU load dropped to 1%-3%
* the packet loss was gone
* the NTPD service was constantly up
* the LAN-page did not have the issue with the hanging JS-script any longer

Today I learnt that, although the port forwarding rules were still 'there', the corresponding firewall rules were 'gone' - in the firewall logs all attempts to forward traffic to the local server were 'blocked by the default deny rule'. To change this, it did not suffice to change the port forward rule to set the corresponding firewall rule to 'pass' - nor did it help to add an explicit firewall rule to allow the specific traffic on a certain port. Only after removing all specific firewall rules AND port forwarding rules, I could re-add port forwarding rules with the expected consequence that such traffic would be allowed to pass through the firewall.

Firewalling and routing network traffic are not entirely new to me, but my knowledge/experience is quite high level. Still, if I may cautiously conclude:
1. the above would not have happened if I did not have any collisions on the internal network, but I think OPNsense 'sensed' these issues while it was not able (or willing :)) to tell me what was wrong (or at least something was wrong - it even reported '0 collisions' and '0 packets lost'). If at all possible, this would be a great feature...
2. maybe I have messed up the NAT/Firewall rules myself by deleting the igb0/WAN interface, but given the consequences if the situation is restored (all rules have to be deleted and reconfigured anyway), wouldn't it be a useful action to automatically delete all NAT+FW rules that had been added to the network interface upon deletion of the network interface assignment?

Thank you for reading this long post - I hope my experiences with this distro can help making it even more newbie-proof :)

Kr,
Vincent