Messages

Hi again,

I meanwhile solved the problem with the help of mitmproxy to inspect the headers.

As it turns out nginx rewirtes several headers to lower-case. In my case the header field X-XSRF-TOKEN was changed to x-xsrf-token which caused the problem within the webapp.

I hot-fixed the problem by including

Code Select Expand

proxy_set_header X-XSRF-TOKEN $http_x_xsrf_token;
in the /usr/local/opnsense/service/templates/OPNsense/Nginx/location.conf template.

Is saw that the generated nginx.conf contains includes for each location in the form of:

Code Select Expand


include <guid>_post/*.conf;


Would you suggest to create that directory and put an include config there myself? Do you know of any other ways to have nginx not change the case of header-fields?

Edit: it appears that client <-> nginx is using http2 and nginx <-> webapp is using http 1.1. As http2 requires headers in lowercase format chrome actually sends them in lower case.... aye.

thx
Dark-Sider

Hi,

I have a web-application that up until now used a NAT port-forward. However I need URL-based filtering. As the application is "closed", my solution of choice was to setup a nginx reverse proxy in opnsense and add some ACL-based filtering. It all works great, except on small but important detail:

The web-app displays a logon-page. If I enter the correct username / password (while using nginx as reverse proxy) it displays an login error page. The web-app's log shows:

Code Select Expand


[ERROR] 2022-02-24 13:14:01,144 [qtp142733894-87857] Unauthorized access detected
com.appName.AuthenticationException: Invalid CSRF token


If I then press "reload" on the browser, I'm magically logged in and everything works. Since the web-app is also accessed by external users, I would like to get it 100% working though :)

The reverse proxy configuration is very basic at this stage:
Upstream, and Upstream server are configured with correct ssl certs.
I tried the Upstream configuration with Proxy Protocol enabled and disabled (no difference)

Location configuration is as basic as it can get (just enforce HTTPS) I also tried to enable and disable the response/request buffering (no idea what this actually does though)

The HTTP-Server configuration is also very basic. It just listens on a specific virtual IP on specific ports. Location is set and SSL-Cert is set.
I also tried enabling proxy protocol within the HTTP-Server options, and setting the real ip source to all options. Nothing worked (I restarted nginx after each configuration change)

I have not defined any security headers.

Any ideas what my configuration is missing?

regards
Dark-Sider

Here are some updates on this issue. I have had opened a ticket with AVM the manufacturer of the VOIP-Router. I sent them packet captures and a lot of details. They concluded (and I might have to agree) that issue has to be on the firewall. So I did some more testing tonight and the results are really strange. The MTU for my PPPoE device is set to 1492, the LAN interface MTU is 1500.

The SIP response going out to sipgate is too big fit in one packet, so the Fritz!Box already devides it into 2 packets: 1st packet has a length of 1502 and the 2nd packet a length of 237. Those lengths are the full physical ethernet frames reaching opnsense. So 18 bytes ethernet header and footer are deducted, yeilding a full IPv6 size of 1484 which is way below 1492. But opnsense replies to my Fritz!Box with ICMPv6 Packet Size too big.

I dumped the SIP-Payload of both fragments and recombined them to the total size of 1604 Bytes (just the SIP response), opened and piped it to netcat on a linux box in my LAN. I chose sipgates server as destination and used the 5060 port. The packet was again fragmented by nc/linux to the exact same 2 fragment sizes as the fritzBox did, but it went through (!!!). When I compared the headers I only found that the fritzBox chose a flow label of 0x0 and nc put in some (maybe not so) random hex.

When doing a packet capture on the pfsense (filtering to the packet sending machine), I always get those messages:

Code Select Expand


LAN
vmx2 20:44:11.309778 IP6 2001:a61:xxx > 2001:ab7::4: frag (0|1440) 38065 > 5060: UDP, bad length 1604 > 1432
LAN
vmx2 20:44:11.309787 IP6 2001:a61:xxx > 2001:ab7::4: frag (1440|172)


however when the Fritz!Box sends it, it gets dropped and that ICMPv6 message is returned. - the other numbers (except for the source ip) stay the same...

over at pfsense I found that there apparently existed a FreeBSD bug: https://forum.netgate.com/topic/123169/problems-with-mtu-and-dropped-packets/10 https://redmine.pfsense.org/issues/8165 but as opnsense is already at 12.x I think this should be fixed?!

Edit: The issue persists after upgrading to 21.1.3

Edit2: more testing:
This actually might be related to the bug I linked above. I did the following testing:
pfctl -d      The packets go through
pfctl -e      The packets get blocked again with the "too big" message


regards
Darky

Hi,

I'm running an AVM Fritz!Box router as a LAN client for my VOIP needs. It's a neat all-in-one PBX solution for internal ISDN, DECT and SIP phones. Internet is provided by the German ISP M-Net through FTTH/GPON. OPNsense connects via PPPoE to the Internet (MTU set to 1492 in the pppoe options).

To avoid NAT issues with SIP I configured all SIP-accounts as IPv6 only. Under Firewall settings I have allowed sipagte's servers so they can talk to my AVM Fritz!Box.

I'm using both, sipagte and my ISP's voip service. While my ISP's voip works fine, incoming sipagate calls cannot be answered (caller still hears the phone ringing although the call was picked up).

To troubleshoot the problem I did some packet capturing. As it turns out the SIP/SDP 200 OK packet that is sent  to sipgate is "rejected" by OPNsense with an ICMPv6 "Packet too big" and therefore never reaching my voip provider. The ICMPv6 Packet Too Big contains the correct MTU of 1484 (the value that is displayed und calculated MTU for PPP in the pppoe options), the Packet that is rejected has a packet size of 1494.

What would be the expected behavior there? Is the Fritz!Box expected to rentransmit the packet with a smaller packet size or is OPNsense expected to refragment the packet to fit the MTU? If so, what setting do I miss?

regards,
Fabian

Thanks mate, made the changes and at a first glance it works.

Is this the recommended setup to have one box as "master" and let the "client" (DG) initiate the connection?
https://docs.opnsense.org/manual/how-tos/wireguard-s2s.html -- I assumed that the connection settings have to be mutual. Giving it a second thought though, roadwarriors don't have reachable DNS entries well...

Hey guys,

I'm running a wireguard VPN between two dial-up sites in Germany. One provider in M-Net (local carrier in Munich) and the other provider is Deutsche Glasfaser (a FTTH provider, active in several parts of Germany).

For the ease of this post I'll refer to the two sites as MNET and DG. Both sites have installed identical hardware, opnsense is running in an ESXi 6.7 VM on a qotom headless PC (core i7). I have plenty of experience with that hardware and it gets the job done quite well.

Although both sites connect via FTTH (gpon) mnet requires a pppoe connection which featurese IPv4/IPv6 Dual Stack. The IPv6 prefix (a /56) is received via the IPv4 connectivity through the pppoe connection. The pppoe interface does not receive a public but a linklocal IPv6 address. DG just issues IPv4 and IPv6 addresses via DHCP (no pppoe required). The WAN interface also gets a public /128 IPv6 address. Since DG only offers cg NAT for IPv4 connectivity, I'm using wireguard with IPv6 to connect the two sites.

As the IPv6 subnets assigned by both providers are not static, I use dynv6 to update my IPv6 addresses on both sites. Since MNet does not provide a public v6 address on the pppoe/wan I selected the LAN-IPv6-adresses to be served on dynv6. This is checked, and both boxes are pingable with their respective hostnames from the internet using IPv6.

The wireguard part of the configuration was quite straight forward:
- install wirguard
- configure the local part on both boxes
- configure the remote-endpoint on both boxes with the public key of the other box
- activate the endpoints within the respective local config.

For the allowed IP-addresses I selected the corresponding remote network, and the remote tunnel address with /32

MNET uses 172.20.0.0/16, 172.19.0.0.2/32
DG uses 192.168.0.0/24, 172.19.0.0.1/32
as allowed nets. The allowed net is the remote LAN-Subnet of the other box and the other boxe's tunnell address.

I don't need IPv6 within the tunnel so it's only IPv4 for the tunnel.

For the sake of ease both sites have the firewall settings wg0 to IPv4+IPv6 any to any. (Protocol, sourc, dest, port...)

Both boxes run the latest 20.7 opnSense version.

Now the strange part: The tunnel sometimes works and sometimes doesn't work - although the IPv6 prefixes of both sites have not changed and DNS returns the correct IPv6 address. The DNS record also has only an AAAA record configured, to avoid IPv4 connections.

When I look at the wireguard "List Configuration" output on each site i see sth like this:

Code Select Expand


interface: wg0
  public key: XXX
  private key: (hidden)
  listening port: 51820

peer: XXX
  endpoint: [2a00:6020:1000:xxx]:51820
  allowed ips: 172.19.0.2/32, 172.20.0.0/16
  latest handshake: 19 minutes, 38 seconds ago
  transfer: 22.06 KiB received, 14.01 KiB sent
  persistent keepalive: every 1 minute


The other site displays similiar information. If the connection is working, then the received and sent counters go up as expected and traffic passes through the network. But I very often reach a state where both boxes just send packets and the other box won't receive any, or it receives the packet and even sends out an answer (check with tcpdump) but the sending box won't receive any packets.

I also noticed, that the IPv6 address noted in the "peer" section of the List Configuration view on the MNET box, does not match the configured address in the endpoint dialogue or the the resolved address from hostname. (tried both) but keeps displaying the WAN address of the DG box instead of it's LAN address. So it seems to me that wg tries to origniate traffic from DGs WAN instead of LAN IP. I updated the DYNV6 configuration to match the WAN - but no luck there either.

WG runs on 51820 on both boxes and the ports are opened in the Firewall (wan/pppoe) with "THIS FIREWALL" as target. To allow the ICMP-Echo for both boxes I use the same rule but with ICMP...

any ideas? any fundamental flaws in my thinking?

the odd thing is, that it sometimes works and sometimes doesn't, and I can't see whe :(

regards,
Fabian

@Dark-Sider:
So, you set the gateway IP to the real IP instead of "dynamic" and tick "disable routes" and have what exactly???

I set the gateway IP to the real IP so I'm not affected by that bug that won't let me add the gateway. I personally would prefer the dynamic solution. Since I control both endpoints of this setup the gateway's IP adress won't suddenly change so this setup works for me right now.

I have to tick disable routes since I have to add 0.0.0.0/0 in the wireguard's allowed list. If I don't tick it - a new route with highest priority is added to opnsense once the tunnel is established and all traffic goes down that new route. If this should not happen anymore when the other gateways have that "upstream" box ticked, than there is still a bug present, as it happens to me in 19.7.2

Without ticking the box and adding the route for the tunnel network manually on console my setup starts working as it is supposed to be.

So a option, what kind of routes should be added once the tunnel IF goes live would be cool. Usually this would be just the route for the tunnel-network.

regards,

Fabian

some additional input:

if I uncheck disable routes mit routing table looks like this:

Code Select Expand


Internet:
Destination        Gateway            Flags     Netif Expire
0.0.0.0/1          wg0                US          wg0
default            ppp-default.m-onli UGS      pppoe0
[...]


and all traffic is sent through wg0 - upstream flags are set correctly

Hi,

thanks for your swift reply. I changed the interface from DHCP to ipv4 none. I also tried to add a gateway like in the guide:

To do this, go to System ‣ Gateways ‣ Single and add a new gateway. Choose your WireGuard interface and set the Gateway to dynamic.

If I do so and put "dynamic" in the gateway's ip address field the gateway just won't appear in the list. Also there is no errormessage. If I put in the ip adress of my VPS server's tunnel interface (192.168.4.1) the gateway shows up. Maybe this is part of the new gateway code.

I already put my ISP's v6 and v4 gateway as upstream gateway, the vpn_gw does not have that option set.

As for the routing stuff, I'll check this once I'm home tonight. When I  check "disable routes" I now successfully created the route to the tunnel via commandline:

route add -inet 192.168.4.0/24 -link -iface wg0
Is there any way to make this persistant via the gui?

while I did so I noticed another strange thing:

netstat -r lists the following:
192.168.4.2        link#13            UH          wg0
While the .2 IP-Adress is the local ip of my wg interface it is routed out via wg0. If I ping 192.168.4.2 from my opnsense box, packets go into the tunnel, and come back after 180ms (from canada):

root@OPNsense:~ # ping 192.168.4.2
PING 192.168.4.2 (192.168.4.2): 56 data bytes
64 bytes from 192.168.4.2: icmp_seq=0 ttl=63 time=188.488 ms

traceroute to 192.168.4.2 (192.168.4.2), 3 hops max, 40 byte packets
1  192.168.4.1 (192.168.4.1)  94.379 ms  94.387 ms  94.121 ms

If I ping 192.168.4.2 from a device from my LAN it shows my local reply times <1ms

the local ip address of an openvpn tunnel however is bound to lo0
10.41.0.2          link#11            UHS         lo0

is this intended?`

With 19.7 gateway code changed (not Wireguard Code). You have to verify that your default gateway has "upstream" checked and your Wireguard gateaway not. Then this thing can't happen.

So you would say, that ticking the "disable routes" box is not necessary anymore with 19.7, when the ISP gateways have set the upstream flags and the vpn-gw does not?

regards,
Fabian

Hi,

to set the general picture: I installed wireguard devel on 19.1.x a couple of weeks ago. As I now had time play around with my planned wg VPN tunnel I finally did so yesterday.

I already have multiple ipsec and openvpn tunnels on my opnsense box running - so I thought this would be an easy and straight forward task :-)

The purpose for my setup is to route some hosts of my network through an VPS server in canada. This VPS runs ubuntu 18.04 with wireguard. I did manage to bring up the tunnel quite easily. The tunnel network is 192.168.4.0/24 with my wireguard box being .2 and the VPS .1

Without defining any other virtual interfaces (just setting the allow rule for the wirguard interface) I was able to ping the remote location - ping times and tcpdump also show that it actually is the VPS who answered.

My first try was to put 192.168.4.0/24 as "allowed ips" in the wireguard config on the opnsense box. Once the tunnel was established the routes were set and the tunnel worked.

The next step was to route actual internet upstream traffic through the vpn. I looked in the forum and found those threads:
https://forum.opnsense.org/index.php?topic=8998.0 (ovpn)
https://forum.opnsense.org/index.php?topic=4979.0 (ovpn)
https://forum.opnsense.org/index.php?topic=11737.0 (wg)

So I went ahead and did
- define an alias for one host in my network,
- setup the NAT rules (but not needed as NAT can be achieved on the VPS, just makes allowed ips config easier)
- created the virtual VPN interface with assigned wg0 to it and set it to ipv4 DHCP
- created the firewall rule with the alias and the gateway of the virtual VPN interface.

It did not work as I had expected. tcpdump on the VPS shows, that 1962.168.4.0/24 traffic is sent through the tunnel to the VPS but all other traffic just doesn't go through the tunnel - but also does not exit my opnsense on any other interface (ping shows a timeout).

I did a bit of thinking and replaced the allowed ips in the wg config with 0.0.0.0/0 as this might be the problem.

After restarting  the box my whole network was routed through the VPN not just this one host. However, the traffic actually went over the VPS in Canada. It took me some time to figure out that I probably had to check that disable routes checkbox in the wireguard config, das my default route was now set to 0.0.0.0/0 -> VPN

before doing so however I also read that 19.7 was now released and has wg as stable on board. So I reset my opnsense VM to a config state before my wireguard efforts and upgraded the box to 19.7.2 (uninstalled wg-devel and installed the stable wg package)

After setting up the wg tunnel again and verifying that the VPS is pingable from my LAN I again defined the virtual interface.

Then things changed.

First thing I noticed is, that there is no more gateway created automatically once you choose IPV4 DHCP for the virtual interface --- why?

I just stayed with ipv4 "none". Restarted  the tunnel and the VPN interface got the correct ip 192.168.4.2 - a minor change I thought.

I created the gateway manually leaving all IP fields blank just setting the Interface to VPN. After pressing OK and Applying there was no gateway added. I retried this a couple of times - no error message - no gateway.

I then created the gateway with the IP 192.168.4.1 and Interface VPN - which actually worked.

I thend changed the allowed ip again to 0.0.0.0/0 - and all traffic was again forced through the tunnel.

After that I ticked the "disable routes" option in the wg config and restarted the box (just to be safe). This is when things got really bad.

The default route was gone, however so was any route pointing 192.168.4.0/24 to go out through the VPN interface. Thus, 192.168.4.1 (my VPS) was no longer reachable. The routing table also shows only 192.168.4.2 (my opnsense box) on the VPN interface without any netmask.

I then tried to add the route manually as read on the forums, however this is not (not anymore?) possible. Whenn adding the route I can't chose my VPN interface. I only have choice of my PPPoE gws for my dial-up connection.

This is where I hit the roadblock, as I was not able to set
- 0.0.0.0/0 as allowed ips
- without setting a default route through the tunnel
- and I failed adding the 192.168.4.0/24 route manually

Were there changes in 19.7 how this is done?
Is there an obvious mistake I just don't see?

Thanks for helping out!

bye
Fabian

Hey guys,

thanks for bringing wireguard to opnsense! Maybe my question will become obsolete since a fix that should stop crashing is apparently on its way:

Is the crashing only related to configuration tasks while setting up the tunnels / config or is the whole setup unstable while in use?

regards,

Darky

Hi,

When I uncheck "only request a prefix" no ipv6 connectivity is established whatsoever. I also changed the prefix id to 1 on the lan interface if WAN would grab id 0. (also rebooted opnsense several times).

When I got home today ipv6 routing was working great, however half an hour ago or so it just stopped working, prefixes etc. stayed the same. I noticed that IPv6 routing died as several websited wouldn't load since the browser had used v6 for them apparently.

I checked some logs on the opnsense box and found this:

Code Select Expand


Apr 19 22:20:12 OPNsense apinger: No usable targets found, exiting
Apr 19 22:21:43 OPNsense apinger: Starting Alarm Pinger, apinger(40601)
Apr 19 22:21:43 OPNsense apinger: No usable targets found, exiting
Apr 19 22:21:50 OPNsense apinger: Starting Alarm Pinger, apinger(44357)
Apr 19 22:21:50 OPNsense apinger: No usable targets found, exiting
Apr 19 22:21:56 OPNsense apinger: Starting Alarm Pinger, apinger(33336)
Apr 19 22:21:56 OPNsense apinger: No usable targets found, exiting
Apr 19 22:21:56 OPNsense apinger: Starting Alarm Pinger, apinger(51798)
Apr 19 22:21:56 OPNsense apinger: No usable targets found, exiting
Apr 20 22:21:47 OPNsense apinger: Starting Alarm Pinger, apinger(65383)
Apr 20 22:21:47 OPNsense apinger: No usable targets found, exiting


so apinger was dead for like 24 hrs and when it started my v6 routing died - could this be related? Can I start this tool manually What actually does apinger do?

EDIT:
So I did more digging through the logs: ppps.log shows that my Internet-connection was reset after 24 hrs of online-time. This was (until recently) usual behaviour in Germany, however my plan should not have any timed disconnects. I'll have to open a ticket with my provider.

ppps.log, showing the 24hrs disconnect (look at the day):

Code Select Expand


Apr 19 22:21:38 OPNsense ppp: [wan]   93.104.101.XX -> 82.135.16.28
Apr 20 22:21:36 OPNsense ppp: [wan_link0] LCP: rec'd Terminate Request #133 (Opened)


However the pppoe connection is reestablished shorty after the disconnect and the same IPv4 address is issued.
dhcpd.log shows that a NEW v6 prefix was received.

The public v6 address on the LAN-Interface however won't update! It stayed the same and is still the old v6 address (as I'm typing). So there might be some kinks with the "track interface" once the pppoe connection get's reset.

DHCPD.log now looks like this, every few minutes a NEW /56 is received - however the LAN interface or the LAN clients won't get updated

Code Select Expand


Apr 20 23:24:56 OPNsense dhcp6c[61772]: Sending Solicit
Apr 20 23:24:56 OPNsense dhcp6c[61772]: set client ID (len 14)
Apr 20 23:24:56 OPNsense dhcp6c[61772]: set elapsed time (len 2)
Apr 20 23:24:56 OPNsense dhcp6c[61772]: set option request (len 4)
Apr 20 23:24:56 OPNsense dhcp6c[61772]: set IA_PD prefix
Apr 20 23:24:56 OPNsense dhcp6c[61772]: set IA_PD
Apr 20 23:24:56 OPNsense dhcp6c[61772]: send solicit to ff02::1:2%pppoe0
Apr 20 23:24:56 OPNsense dhcp6c[61772]: reset a timer on pppoe0, state=SOLICIT, timeo=38, retrans=126960
Apr 20 23:24:56 OPNsense dhcp6c[22061]: receive advertise from fe80::5a00:bbff:fe09:f8b0%pppoe0 on pppoe0
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option client ID, len 14
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   DUID: 00:01:00:01:22:67:cd:3e:00:0c:29:8c:38:00
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option server ID, len 26
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   DUID: 00:02:00:00:05:83:35:38:3a:30:30:3a:62:62:3a:30:39:3a:66:66:3a:63:63:00:00:00
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option IA_PD, len 41
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   IA_PD: ID=0, T1=1800, T2=2880
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option IA_PD prefix, len 25
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   IA_PD prefix: 2001:a61:47b:9200::/56 pltime=3600 vltime=8199546692136082464
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option DNS, len 32
Apr 20 23:24:56 OPNsense dhcp6c[22061]: XID mismatch
Apr 20 23:24:56 OPNsense dhcp6c[62386]: Sending Solicit
Apr 20 23:24:56 OPNsense dhcp6c[62386]: set client ID (len 14)
Apr 20 23:24:56 OPNsense dhcp6c[62386]: set elapsed time (len 2)
Apr 20 23:24:56 OPNsense dhcp6c[62386]: set option request (len 4)
Apr 20 23:24:56 OPNsense dhcp6c[62386]: set IA_PD prefix
Apr 20 23:24:56 OPNsense dhcp6c[62386]: set IA_PD
Apr 20 23:24:56 OPNsense dhcp6c[62386]: send solicit to ff02::1:2%pppoe0
Apr 20 23:24:56 OPNsense dhcp6c[62386]: reset a timer on pppoe0, state=SOLICIT, timeo=38, retrans=126960
Apr 20 23:24:56 OPNsense dhcp6c[22061]: receive advertise from fe80::5a00:bbff:fe09:f8b0%pppoe0 on pppoe0
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option client ID, len 14
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   DUID: 00:01:00:01:22:67:cd:3e:00:0c:29:8c:38:00
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option server ID, len 26
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   DUID: 00:02:00:00:05:83:35:38:3a:30:30:3a:62:62:3a:30:39:3a:66:66:3a:63:63:00:00:00
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option IA_PD, len 41
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   IA_PD: ID=0, T1=1800, T2=2880
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option IA_PD prefix, len 25
Apr 20 23:24:56 OPNsense dhcp6c[22061]:   IA_PD prefix: 2001:a61:47b:9200::/56 pltime=3600 vltime=8199546692136082464
Apr 20 23:24:56 OPNsense dhcp6c[22061]: get DHCP option DNS, len 32
Apr 20 23:24:56 OPNsense dhcp6c[22061]: XID mismatch
Apr 20 23:25:38 OPNsense dhcp6c[40907]: Sending Solicit
Apr 20 23:25:38 OPNsense dhcp6c[40907]: set client ID (len 14)
Apr 20 23:25:38 OPNsense dhcp6c[40907]: set elapsed time (len 2)
Apr 20 23:25:38 OPNsense dhcp6c[40907]: set option request (len 4)
Apr 20 23:25:38 OPNsense dhcp6c[40907]: set IA_PD prefix
Apr 20 23:25:38 OPNsense dhcp6c[40907]: set IA_PD
Apr 20 23:25:38 OPNsense dhcp6c[40907]: send solicit to ff02::1:2%pppoe0
Apr 20 23:25:38 OPNsense dhcp6c[40907]: reset a timer on pppoe0, state=SOLICIT, timeo=518, retrans=127512
Apr 20 23:25:38 OPNsense dhcp6c[22061]: receive advertise from fe80::5a00:bbff:fe09:f8b0%pppoe0 on pppoe0
Apr 20 23:25:38 OPNsense dhcp6c[22061]: get DHCP option client ID, len 14
Apr 20 23:25:38 OPNsense dhcp6c[22061]:   DUID: 00:01:00:01:22:67:cd:3e:00:0c:29:8c:38:00
Apr 20 23:25:38 OPNsense dhcp6c[22061]: get DHCP option server ID, len 26
Apr 20 23:25:38 OPNsense dhcp6c[22061]:   DUID: 00:02:00:00:05:83:35:38:3a:30:30:3a:62:62:3a:30:39:3a:66:66:3a:63:63:00:00:00
Apr 20 23:25:38 OPNsense dhcp6c[22061]: get DHCP option IA_PD, len 41
Apr 20 23:25:38 OPNsense dhcp6c[22061]:   IA_PD: ID=0, T1=1800, T2=2880
Apr 20 23:25:38 OPNsense dhcp6c[22061]: get DHCP option IA_PD prefix, len 25
Apr 20 23:25:38 OPNsense dhcp6c[22061]:   IA_PD prefix: 2001:a61:47b:a400::/56 pltime=3600 vltime=8199546692136082464
Apr 20 23:25:38 OPNsense dhcp6c[22061]: get DHCP option DNS, len 32
Apr 20 23:25:38 OPNsense dhcp6c[22061]: XID mismatch
Apr 20 23:27:03 OPNsense dhcp6c[61772]: Sending Solicit
Apr 20 23:27:03 OPNsense dhcp6c[61772]: set client ID (len 14)
Apr 20 23:27:03 OPNsense dhcp6c[61772]: set elapsed time (len 2)
Apr 20 23:27:03 OPNsense dhcp6c[61772]: set option request (len 4)
Apr 20 23:27:03 OPNsense dhcp6c[61772]: set IA_PD prefix
Apr 20 23:27:03 OPNsense dhcp6c[61772]: set IA_PD
Apr 20 23:27:03 OPNsense dhcp6c[61772]: send solicit to ff02::1:2%pppoe0
Apr 20 23:27:03 OPNsense dhcp6c[61772]: reset a timer on pppoe0, state=SOLICIT, timeo=39, retrans=120360
Apr 20 23:27:03 OPNsense dhcp6c[22061]: receive advertise from fe80::5a00:bbff:fe09:f8b0%pppoe0 on pppoe0
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option client ID, len 14
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   DUID: 00:01:00:01:22:67:cd:3e:00:0c:29:8c:38:00
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option server ID, len 26
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   DUID: 00:02:00:00:05:83:35:38:3a:30:30:3a:62:62:3a:30:39:3a:66:66:3a:63:63:00:00:00
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option IA_PD, len 41
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   IA_PD: ID=0, T1=1800, T2=2880
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option IA_PD prefix, len 25
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   IA_PD prefix: 2001:a61:47b:a400::/56 pltime=3600 vltime=8199546692136082464
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option DNS, len 32
Apr 20 23:27:03 OPNsense dhcp6c[22061]: XID mismatch
Apr 20 23:27:03 OPNsense dhcp6c[62386]: Sending Solicit
Apr 20 23:27:03 OPNsense dhcp6c[62386]: set client ID (len 14)
Apr 20 23:27:03 OPNsense dhcp6c[62386]: set elapsed time (len 2)
Apr 20 23:27:03 OPNsense dhcp6c[62386]: set option request (len 4)
Apr 20 23:27:03 OPNsense dhcp6c[62386]: set IA_PD prefix
Apr 20 23:27:03 OPNsense dhcp6c[62386]: set IA_PD
Apr 20 23:27:03 OPNsense dhcp6c[62386]: send solicit to ff02::1:2%pppoe0
Apr 20 23:27:03 OPNsense dhcp6c[62386]: reset a timer on pppoe0, state=SOLICIT, timeo=39, retrans=120360
Apr 20 23:27:03 OPNsense dhcp6c[22061]: receive advertise from fe80::5a00:bbff:fe09:f8b0%pppoe0 on pppoe0
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option client ID, len 14
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   DUID: 00:01:00:01:22:67:cd:3e:00:0c:29:8c:38:00
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option server ID, len 26
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   DUID: 00:02:00:00:05:83:35:38:3a:30:30:3a:62:62:3a:30:39:3a:66:66:3a:63:63:00:00:00
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option IA_PD, len 41
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   IA_PD: ID=0, T1=1800, T2=2880
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option IA_PD prefix, len 25
Apr 20 23:27:03 OPNsense dhcp6c[22061]:   IA_PD prefix: 2001:a61:47b:a400::/56 pltime=3600 vltime=8199546692136082464
Apr 20 23:27:03 OPNsense dhcp6c[22061]: get DHCP option DNS, len 32
Apr 20 23:27:03 OPNsense dhcp6c[22061]: XID mismatch
Apr 20 23:27:45 OPNsense dhcp6c[40907]: Sending Solicit
Apr 20 23:27:45 OPNsense dhcp6c[40907]: set client ID (len 14)
Apr 20 23:27:45 OPNsense dhcp6c[40907]: set elapsed time (len 2)
Apr 20 23:27:45 OPNsense dhcp6c[40907]: set option request (len 4)
Apr 20 23:27:45 OPNsense dhcp6c[40907]: set IA_PD prefix
Apr 20 23:27:45 OPNsense dhcp6c[40907]: set IA_PD
Apr 20 23:27:45 OPNsense dhcp6c[40907]: send solicit to ff02::1:2%pppoe0
Apr 20 23:27:45 OPNsense dhcp6c[40907]: reset a timer on pppoe0, state=SOLICIT, timeo=519, retrans=125580
Apr 20 23:27:45 OPNsense dhcp6c[22061]: receive advertise from fe80::5a00:bbff:fe09:f8b0%pppoe0 on pppoe0
Apr 20 23:27:45 OPNsense dhcp6c[22061]: get DHCP option client ID, len 14
Apr 20 23:27:45 OPNsense dhcp6c[22061]:   DUID: 00:01:00:01:22:67:cd:3e:00:0c:29:8c:38:00
Apr 20 23:27:45 OPNsense dhcp6c[22061]: get DHCP option server ID, len 26
Apr 20 23:27:45 OPNsense dhcp6c[22061]:   DUID: 00:02:00:00:05:83:35:38:3a:30:30:3a:62:62:3a:30:39:3a:66:66:3a:63:63:00:00:00
Apr 20 23:27:45 OPNsense dhcp6c[22061]: get DHCP option IA_PD, len 41
Apr 20 23:27:45 OPNsense dhcp6c[22061]:   IA_PD: ID=0, T1=1800, T2=2880
Apr 20 23:27:45 OPNsense dhcp6c[22061]: get DHCP option IA_PD prefix, len 25
Apr 20 23:27:45 OPNsense dhcp6c[22061]:   IA_PD prefix: 2001:a61:47b:b600::/56 pltime=3600 vltime=8199546692136082464
Apr 20 23:27:45 OPNsense dhcp6c[22061]: get DHCP option DNS, len 32
Apr 20 23:27:45 OPNsense dhcp6c[22061]: XID mismatch
Apr 20 23:29:03 OPNsense dhcp6c[61772]: Sending Solicit
Apr 20 23:29:03 OPNsense dhcp6c[61772]: set client ID (len 14)
Apr 20 23:29:03 OPNsense dhcp6c[61772]: set elapsed time (len 2)
Apr 20 23:29:03 OPNsense dhcp6c[61772]: set option request (len 4)
Apr 20 23:29:03 OPNsense dhcp6c[61772]: set IA_PD prefix
Apr 20 23:29:03 OPNsense dhcp6c[61772]: set IA_PD
Apr 20 23:29:03 OPNsense dhcp6c[61772]: send solicit to ff02::1:2%pppoe0
Apr 20 23:29:03 OPNsense dhcp6c[61772]: reset a timer on pppoe0, state=SOLICIT, timeo=40, retrans=129864
Apr 20 23:29:03 OPNsense dhcp6c[22061]: receive advertise from fe80::5a00:bbff:fe09:f8b0%pppoe0 on pppoe0
Apr 20 23:29:03 OPNsense dhcp6c[22061]: get DHCP option client ID, len 14
Apr 20 23:29:03 OPNsense dhcp6c[22061]:   DUID: 00:01:00:01:22:67:cd:3e:00:0c:29:8c:38:00
Apr 20 23:29:03 OPNsense dhcp6c[22061]: get DHCP option server ID, len 26
Apr 20 23:29:03 OPNsense dhcp6c[22061]:   DUID: 00:02:00:00:05:83:35:38:3a:30:30:3a:62:62:3a:30:39:3a:66:66:3a:63:63:00:00:00
Apr 20 23:29:03 OPNsense dhcp6c[22061]: get DHCP option IA_PD, len 41
Apr 20 23:29:03 OPNsense dhcp6c[22061]:   IA_PD: ID=0, T1=1800, T2=2880
Apr 20 23:29:03 OPNsense dhcp6c[22061]: get DHCP option IA_PD prefix, len 25
Apr 20 23:29:03 OPNsense dhcp6c[22061]:   IA_PD prefix: 2001:a61:47b:b600::/56 pltime=3600 vltime=8199546692136082464
Apr 20 23:29:03 OPNsense dhcp6c[22061]: get DHCP option DNS, len 32
Apr 20 23:29:03 OPNsense dhcp6c[22061]: XID mismatch


Edit2:
just now - a few minutes ago - a new ipv6 prefix was propagated to my LAN and LAN-IF. Here is the system log from the relavant time frame. Interestingly "rc.newwanipv6" was just executed before connectivity was restored:

Code Select Expand


Apr 20 22:21:44 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'pppoe0'
Apr 20 22:21:44 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: fe80::2c40:3052:ceaf:5446) (interface: MNET[wan]) (real interface: pppoe0).
Apr 20 22:21:46 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'pppoe0'
Apr 20 22:21:46 OPNsense opnsense: /usr/local/etc/rc.newwanip: On (IP address: 93.104.101.XX) (interface: MNET[wan]) (real interface: pppoe0).
Apr 20 22:21:46 OPNsense opnsense: /usr/local/etc/rc.newwanip: Accept router advertisements on interface pppoe0
Apr 20 22:21:47 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Apr 20 22:21:47 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv4 default gateway set, trying wan on 'pppoe0' (82.135.16.28)
Apr 20 22:21:47 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, trying wan on 'pppoe0' (fe80::5a00:bbff:fe09:f8b0)
Apr 20 22:21:47 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 82.135.16.28
Apr 20 22:21:47 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv6 default route to fe80::5a00:bbff:fe09:f8b0
Apr 20 22:21:48 OPNsense ipsec_starter[79816]: configuration 'con1' unrouted
Apr 20 22:21:48 OPNsense ipsec_starter[79816]:
Apr 20 22:21:48 OPNsense opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface MNET.
Apr 20 22:21:48 OPNsense kernel: ovpnc1: link state changed to DOWN
Apr 20 22:21:49 OPNsense ipsec_starter[79816]: 'con1' routed
Apr 20 22:21:49 OPNsense ipsec_starter[79816]:
Apr 20 22:21:52 OPNsense opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS (xxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Apr 20 22:21:53 OPNsense kernel: ovpnc1: link state changed to UP
Apr 20 22:21:53 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpnc1'
Apr 20 22:21:53 OPNsense opnsense: /usr/local/etc/rc.newwanip: Interface '' is disabled or empty, nothing to do.
Apr 20 22:32:16 OPNsense opnsense: /index.php: Session timed out for user 'root' from:
Apr 20 22:32:19 OPNsense opnsense: /index.php: Web GUI authentication error for 'root' from
Apr 20 22:32:23 OPNsense opnsense: /index.php: Successful login for user 'root' from:
Apr 20 22:43:18 OPNsense opnsense: user 'root' could not authenticate.
Apr 20 22:43:18 OPNsense sshd[7223]: error: PAM: authentication error for root from
Apr 20 22:43:18 OPNsense sshd[7223]: error: PAM: authentication error for root from
Apr 20 22:43:21 OPNsense opnsense: user 'root' authenticated successfully
Apr 20 22:43:21 OPNsense sshd[7223]: Accepted keyboard-interactive/pam for root from port 49181 ssh2
Apr 20 22:54:35 OPNsense kernel: pid 9128 (apinger), uid 65534: exited on signal 11
Apr 20 23:52:00 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'pppoe0'
Apr 20 23:52:00 OPNsense opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: fe80::2c40:3052:ceaf:5446) (interface: MNET[wan]) (real interface: pppoe0).


regards,
Fabian

Hi,

a quick update:

when I came back home from work, IPv6 Routing of the prefix that wasn't routed in the morning now works just fine. Maybe there is some cronjob / timed issue at hand. I read on some threads that a tool called arpinger (or compareble spelling) checks the availability of gateways etc.

Using the above settings, my LAN clients receive IPv6 addresses but my WAN does not. The LAN clients can still ping ipv6 locations, such as ipv6.google.com.

If I remove the option

Code Select Expand

Request only a ipv6 prefix then both the WAN and LAN interfaces receive an ipv6 address. In your case, can you try unchecking this option and see if your WAN and LAN receive a stable address range?

I can try to do this, however my ISP uses a "private" prefix for routing IPv6 Packets between the provider's gateway and the customer's firewall - so I don't know if a public IPv6-address on my WAN would solve anything or if it actually might make things worse.

regards,
Fabian

Hi,

as it turns out, the issue is not yet resolved. Today in the morning opnsense got a new v6 prefix from the provider which was correctly handed out to clients- however those clients were unable to connect to v6-addresses on the internet. opnsense itself was perfectly able to use it's own new address to connect to v6 hosts.

So I don't think that it's a routing issue on the side of my provider. Since the subnet/prefix stays the same between opnsense and the clients. Why would opnsense be routed by my provider and another client not? The only difference is that the clients use opnsense as gateway.

regards,
Fabian

Hi,

a firewall rule one the WAN side should only be needed if I wanted to ping from the outside into my home network.

However: I just returned from and the problem magically is gone. I did not reboot the firewall nor was the internet connection reset.

Both my laptop and my iphone had no v6 connectivity yesterday, but today everything works just fine. I'll monitor this behaviour - seems a bit sketchy to me.