[solved] no ipv6 outbound possible

[securid]

I suspect I am missing something obvious but I don't see it yet  .

My ISP offers ipv6 /48. I enabled dhcpv6 on WAN and set my LAN to track interface. Everything is getting ipv6 addresses on my LAN. I can ping and connect to local services over ipv6. DNS seems to work as well, I can ping6 hostname and it returns replies with the ipv6 address.

I created a firewall rule on LAN to allow ipv6 from LAN net to all. Basically I cloned my ipv4 rule and changed it to ipv6.

When I go to test-ipv6.net and some other test websites, its not detecting ipv6 at all.

When I `curl https://\[2a02:2e0:3fe:1001:302::\]` I get:
curl: (7) Failed to connect to 2a02:2e0:3fe:1001:302:: port 443 after 4006 ms: Couldn't connect to server

When I do `curl -k https://\[opnsense lan ipv6 address\]` it connects to my opnsense.

In the logging I don't see any blocks. I tried several hosts and nothing is able to connect to the outside on ipv6.

I have not setup outbound NAT because I don't think it requires that.
I checked dhcp6 gateway has been created, its up and green.

I checked my local ipv6 default routes for a gateway and internet6 default route is set to the pfsense address using its fe80:: address. This the only thing that has me wondering wether that is actually correct?

No ipv6 expert here I admit. Any ideas what I am missing?

Thanks a bunch!

[bartjsmit]

Are you running a router advertisement on your LAN side?

Services: Router Advertisements: [LAN]

Bart...

[securid]

Are you running a router advertisement on your LAN side?

Services: Router Advertisements: [LAN]

Bart...

Hey Bart. Yes. Should have mentioned that, sorry . When track interface alone didn't work, I:
enabled `Allow manual adjustment of DHCPv6 and Router Advertisements`
enabled `Enable DHCPv6 server on LAN interface`
Set the DHCPv6 range to include the LAN prefix ID (added to the already present IPv6 prefix): `:0::` and end to `:0:ffff:ffff:ffff:ffff`

I then went into Services, Router Advertisements, LAN and set to `Assisted` with `Advertise Default Gateway` ticked.

I also rebooted OPNsense, just to be sure.

Still no outbound connections on IPv6.

Thanks!

[securid]

Its still not working. I've been randomly trying different settings because I'm at a loss. I searched and found several guides specifically for opnsense and pfsense in combination with my isp. I can see nothing wrong.

Since I wasn't seeing any traffic passing opnsense with tcpdump, I added my opnsense ipv6 LAN address to the routes list on the router advertisement. I now see the following when I start tcpdump on my opnsense box. I can see the curl request from my client coming in on igc1 (or should I say leaving igc1?). (ipv6 addresses redacted):

Code: [Select]

root@opnsense:~ # tcpdump -i igc1 -vvvv -nnnn host 2a02:my_client_ipv6:c662
tcpdump: listening on igc1, link-type EN10MB (Ethernet), capture size 262144 bytes
09:05:27.000583 IP6 (flowlabel 0xb0600, hlim 64, next-header TCP (6) payload length: 44) 2a02:my_client_ipv6:c662.51026 > 2a00:1450:400e:80c::2004.443: Flags [SEW], cksum 0xfc79 (correct), seq 3166761984, win 65535, options [mss 1440,nop,wscale 6,nop,nop,TS val 433254785 ecr 0,sackOK,eol], length 0
09:05:27.000696 IP6 (hlim 64, next-header ICMPv6 (58) payload length: 92) 2a02:opnsense_lan_ipv6::1 > 2a02:my_client_ipv6:c662: [icmp6 sum ok] ICMP6, destination unreachable, unreachable route 2a00:1450:400e:80c::2004
Above 2 lines repeats 4 more times
09:05:31.578066 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) 2a02:opnsense_lan_ipv6::1 > 2a02:my_client_ipv6:c662: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 2a02:my_client_ipv6:c662
  source link-address option (1), length 8 (1): 00:opnsense_lan_mac:b3
    0x0000:  00e2 6960 5db3

So I'm not sure how to interpret the line that says destination unreachable. The return packet (from www.google.com in this case) found its way back to my opnsense, but opnsense cannot find a route back to my client?

I enabled logging on all relevant rules  and I still don't see ipv6 blocks.

Any ideas please what I can check to get this working?

Thanks!

[doktornotor]

Since I wasn't seeing any traffic passing opnsense with tcpdump, I added my opnsense ipv6 LAN address to the routes list on the router advertisement.

That is not good, it will (if anything) advertise a route to your FW LAN address, not needed and won't help. For debugging, try ::/0 instead.

Also, unplug the network on the client(s) temporarily after making changes on the firewall.

Assumimg a Windows client:

Code: [Select]

route -6 print
On OPNsense:

Code: [Select]

netstat -rn6
Enough of guessing here.

[securid]

Since I wasn't seeing any traffic passing opnsense with tcpdump, I added my opnsense ipv6 LAN address to the routes list on the router advertisement.

That is not good, it will (if anything) advertise a route to your FW LAN address, not needed and won't help. For debugging, try ::/0 instead.

Thanks for the suggestion! I can use all the help I can get haha

Do you mean in the router advertisement, for Advertise Routes, fill in :: for prefix, and 0 for length?

If so, that doesn't seem to help either. I can still see traffic from the client with a tcpdump on opnsense, but no return traffic.

A difference now is that with tcpdump filtering on client ipv6 address alone, I now see a lot of packets flying over the screen which wasn't the case before. Maybe I should make a capture and look at it in wireguard. I was hoping that wasn't necessary though its way beyond my paygrade.

In the live logging view, I still don't see ipv6 passed, or blocked.

[securid]

ps didnt see your edit until now. Hang on 

[doktornotor]

Do you mean in the router advertisement, for Advertise Routes, fill in :: for prefix, and 0 for length?

Yes. Should not be needed, of course.

A difference now is that with tcpdump filtering on client ipv6 address alone, I now see a lot of packets flying over the screen which wasn't the case before. Maybe I should make a capture and look at it in wireguard. I was hoping that wasn't necessary though its way beyond my paygrade.

Should really be last resort, there's some obvious error in your setup, these things just work normally.

See the other suggestions for less low-level debugging.

P.S. Yeah, hanging. 

[securid]

See the other suggestions for less low-level debugging.

P.S. Yeah, hanging. 

Some clients are MacOS, others are Linux. This is Linux:

Code: [Select]

[root@arch01 ~]# netstat -rn6
Kernel IPv6 routing table
Destination                    Next Hop                   Flag Met Ref  Use If
2a02:pre:fix::1594/128       ::                         U    100 1      0 ens33
2a02:pre:fix::/64            ::                         U    100 1      0 ens33
fe80::/64                      ::                         U    1024 1      0 ens33
::/0                           fe80::2e2:69ff:fe60:5db3   UG   20100 3      0 ens33
::1/128                        ::                         Un   0   5      0 lo
2a02:pre:fix::1594/128       ::                         Un   0   3      0 ens33
2a02:pre:fix:0:46b8:96c8:4eb0:26b8/128 ::                         Un   0   3      0 ens33
fe80::8e96:a476:33a7:d5be/128  ::                         Un   0   3      0 ens33
ff00::/8                       ::                         U    256 3      0 ens33
::/0                           ::                         !n   -1  1      0 lo
And ip a output:

Code: [Select]

2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0c:29:d5:b5:20 brd ff:ff:ff:ff:ff:ff
    altname enp2s1
    inet 10.1.2.118/24 brd 10.1.2.255 scope global dynamic noprefixroute ens33
       valid_lft 6922sec preferred_lft 6922sec
    inet6 2a02:pre:fix::1594/128 scope global dynamic noprefixroute
       valid_lft 6843sec preferred_lft 4143sec
    inet6 2a02:pre:fix:0:46b8:96c8:4eb0:26b8/64 scope global dynamic noprefixroute
       valid_lft 86345sec preferred_lft 14345sec
    inet6 fe80::8e96:a476:33a7:d5be/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
Ping and curl:

Code: [Select]

[root@arch01 ~]# ping -6 www.google.com
PING www.google.com(ams16s32-in-x04.1e100.net (2a00:1450:400e:80c::2004)) 56 data bytes
From opnsense (2a02:opn:sense::1) icmp_seq=1 Destination unreachable: No route
From opnsense (2a02:opn:sense::1) icmp_seq=2 Destination unreachable: No route
From opnsense (2a02:opn:sense::1) icmp_seq=3 Destination unreachable: No route
^[[AFrom opnsense (2a02:opn:sense::1) icmp_seq=4 Destination unreachable: No route
^C
--- www.google.com ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3029ms

[root@arch01 ~]# curl -k https://\[2a00:1450:400e:80c::2004\]
curl: (7) Failed to connect to 2a00:1450:400e:80c::2004 port 443 after 0 ms: Couldn't connect to server
[root@arch01 ~]#


[securid]

Missed the opnsense routing table request:

Code: [Select]

root@opnsense:~ # netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::1                               link#5                        UHS         lo0
2a02:pre:fix:::/64               link#2                        U          igc1
2a02:pre:fix:::/48               ::1                           USB         lo0
2a02:pre:fix:::1                 link#2                        UHS         lo0
2a02:pre:fix::10::/64            link#11                       U      igc1_vla
2a02:pre:fix::10::1              link#11                       UHS         lo0
fe80::%igc1/64                    link#2                        U          igc1
fe80::2e2:69ff:fe60:5db3%igc1     link#2                        UHS         lo0
fe80::%lo0/64                     link#5                        U           lo0
fe80::1%lo0                       link#5                        UHS         lo0
fe80::%igc1_vlan10/64             link#11                       U      igc1_vla
fe80::2e2:69ff:fe60:5db3%igc1_vlan10 link#11                    UHS         lo0
fe80::%pppoe0/64                  link#19                       U        pppoe0
fe80::2e2:69ff:fe60:5db3%pppoe0   link#19                       UHS         lo0


[doktornotor]

Missed the opnsense routing table request:

Code: [Select]

root@opnsense:~ # netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::1                               link#5                        UHS         lo0
2a02:pre:fix:::/64               link#2                        U          igc1
2a02:pre:fix:::/48               ::1                           USB         lo0
2a02:pre:fix:::1                 link#2                        UHS         lo0
2a02:pre:fix::10::/64            link#11                       U      igc1_vla
2a02:pre:fix::10::1              link#11                       UHS         lo0
fe80::%igc1/64                    link#2                        U          igc1
fe80::2e2:69ff:fe60:5db3%igc1     link#2                        UHS         lo0
fe80::%lo0/64                     link#5                        U           lo0
fe80::1%lo0                       link#5                        UHS         lo0
fe80::%igc1_vlan10/64             link#11                       U      igc1_vla
fe80::2e2:69ff:fe60:5db3%igc1_vlan10 link#11                    UHS         lo0
fe80::%pppoe0/64                  link#19                       U        pppoe0
fe80::2e2:69ff:fe60:5db3%pppoe0   link#19                       UHS         lo0


You are missing the default route there.

Code: [Select]

# netstat -rn6 | grep default
default                           fe80::e681:84ff:fec3:3734%pppoe0 UG    pppoe0


[securid]

You are missing the default route there.

Code: [Select]

# netstat -rn6 | grep default
default                           fe80::e681:84ff:fec3:3734%pppoe0 UG    pppoe0


Shouldn't that be set automatically?

I mean, I can set it manually but that shouldn't be required?

[doktornotor]

It should. There's a bunch of threads here about possible track interface and PPPoE issues.

It the WAN v6 gateway shown as up and running? Also try restarting the dpinger service.

[securid]

It should. There's a bunch of threads here about possible track interface and PPPoE issues.

It the WAN v6 gateway shown as up and running? Also try restarting the dpinger service.

Awesome. Like I said in my opening post, it must be something obvious.

The IPv6 gateway was there (I had checked it), but, the tickbox for upstream gateway was not ticked.

In the end so much trouble for such a small thing haha!

Thanks for the help, all working now!

PS. I removed the ::/0 from the router advertisements and removed all the any-any rules.

[doktornotor]

The IPv6 gateway was there (I had checked it), but, the tickbox for upstream gateway was not ticked.

In the end so much trouble for such a small thing haha!

Not convinced ticking this checkbox should be necessary. The PPPoE GW is dynamic and should be selected automatically.

https://docs.opnsense.org/manual/gateways.html

Anyway, with single WAN setups, I have a habit of ticking the "consider GW to be always up" since the related logic on WAN GW going down does not do anything useful there anyway, the internet is down and will remain down.