haproxy for ipv4 and ipv6 to ipv4; ipv6 doesnt work?

[abraxxa]

Port 56571 or 56573?

[Bytechanger]

I'm trying both.

[Maurice]

To approach this from a different angle: Can you access any IPv6 service running on OPNsense from the Internet? VPN, SSH, ...?
If you're not sure, you could e. g. allow SSH access from the Internet for testing.

[Bytechanger]

OK, you are right.

It seems to be an problem with opnsense/pppoe, not haproxy.

On InternetServer I can reach ipv6

Code Select Expand


[b]dig AAAA +short www.heise.de[/b]
2a02:2e0:3fe:1001:7777:772e:2:85

[b]wget --no-check-certificate https://[2a02:2e0:3fe:1001:7777:772e:2:85][/b]
--2020-05-27 05:48:12--  https://[2a02:2e0:3fe:1001:7777:772e:2:85]/
Connecting to [2a02:2e0:3fe:1001:7777:772e:2:85]:443... connected.
    WARNING: certificate common name 'www.heise.de' doesn't match requested host name '2a02:2e0:3fe:1001:7777:772e:2:85'.
HTTP request sent, awaiting response... 200 OK
Length: 76 [text/plain]
Saving to: 'index.html.6'

index.html.6                  100%[===============================================>]      76  --.-KB/s    in 0s

2020-05-27 05:48:13 (10.8 MB/s) - 'index.html.6' saved [76/76]

[b]works fine[/b]


SSH to OPNSense over PPPOE works over ipv4 but also not over ipv6:

Code Select Expand


ssh -i /home/blabla/.ssh/homekey -p 56561 -vvv testuser@2003:(WAN address)
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "2003:(WAN address)" port 56561
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 2003:xxx [2003:(WAN address)] port 56561.

nothing....


on client, try to connect

Code Select Expand


sudo tcpdump -ni ens192 'tcp port 56561'
[sudo] password for blabla:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
05:54:19.387975 IP6 2001:(IP Client).54516 > 2003:(WAN Firewall).56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackOK,TS val 2362155641 ecr 0,nop,wscale 6], length 0
05:54:20.396577 IP6 2001:(IP Client).54516 > 2003:(WAN Firewall).56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackOK,TS val 2362156649 ecr 0,nop,wscale 6], length 0
05:54:22.412581 IP6 2001:(IP Client).54516 > 2003:(WAN Firewall).56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackOK,TS val 2362158665 ecr 0,nop,wscale 6], length 0
05:54:26.604603 IP6 2001:(IP Client).54516 > 2003:(WAN Firewall).56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackOK,TS val 2362162857 ecr 0,nop,wscale 6], length 0
05:54:34.796572 IP6 2001:(IP Client).54516 > 2003:(WAN Firewall).56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackOK,TS val 2362171049 ecr 0,nop,wscale 6], length 0

---------------------------
sudo tcpdump -vv -ni ens192 'tcp port 56561'
tcpdump: listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
06:27:20.701131 IP6 (flowlabel 0xd3867, hlim 64, next-header TCP (6) payload length: 40) 2001:xxx.54520 > 2003:xxx.56561: Flags [S], cksum 0xcbfd (incorrect -> 0xc709), seq 1683830560, win 64800, options [mss 1440,sackOK,TS val 2364136905 ecr 0,nop,wscale 6], length 0
06:27:21.708591 IP6 (flowlabel 0xf0fdf, hlim 64, next-header TCP (6) payload length: 40) 2001:xxx.54520 > 2003:xxx.56561: Flags [S], cksum 0xcbfd (incorrect -> 0xc31a), seq 1683830560, win 64800, options [mss 1440,sackOK,TS val 2364137912 ecr 0,nop,wscale 6], length 0


on OPNSense-Firewall

Code Select Expand


sudo tcpdump -ni pppoe0 'tcp port 56561'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pppoe0, link-type NULL (BSD loopback), capture size 262144 bytes
07:54:19.396907 IP6 2003:(IP Client).54516 > 2003:(FIRST PART of IP OPNSense WAN)                                                                                 e92:8583.56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackO                                                                                 K,TS val 2362155641 ecr 0,nop,wscale 6], length 0
07:54:19.396972 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362155641], length 0
07:54:20.405540 IP6 2003:(IP Client).54516 > 2003:(FIRST PART of IP OPNSense WAN)                                                                                 e92:8583.56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackO                                                                                 K,TS val 2362156649 ecr 0,nop,wscale 6], length 0
07:54:20.405579 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362156649], length 0
07:54:22.421526 IP6 2003:(IP Client).54516 > 2003:(FIRST PART of IP OPNSense WAN)                                                                                 e92:8583.56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackO                                                                                 K,TS val 2362158665 ecr 0,nop,wscale 6], length 0
07:54:22.421564 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362158665], length 0
07:54:25.427714 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362158665], length 0
07:54:26.613695 IP6 2003:(IP Client).54516 > 2003:(FIRST PART of IP OPNSense WAN)                                                                                 e92:8583.56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackO                                                                                 K,TS val 2362162857 ecr 0,nop,wscale 6], length 0
07:54:26.613735 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362162857], length 0
07:54:29.613738 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362162857], length 0
07:54:32.867815 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362162857], length 0
07:54:34.805559 IP6 2003:(IP Client).54516 > 2003:(FIRST PART of IP OPNSense WAN)                                                                                 e92:8583.56561: Flags [S], seq 1051216040, win 64800, options [mss 1440,sackO                                                                                 K,TS val 2362171049 ecr 0,nop,wscale 6], length 0
07:54:34.805592 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362171049], length 0
07:54:37.805475 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362171049], length 0
07:54:41.006098 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362171049], length 0
07:54:44.205453 IP6 2003:(IP OPNSense WAN).56561 > 2001:(FIRST PART of IP Client)                                                                                 :803c::1.54516: Flags [S.], seq 1796056796, ack 1051216041, win 65228, option                                                                                 s [mss 1432,nop,wscale 9,sackOK,TS val 1751506195 ecr 2362171049], length 0


------------
sudo tcpdump -vv -ni pppoe0 'tcp port 56561'
tcpdump: listening on pppoe0, link-type NULL (BSD loopback), capture size 262144 bytes
08:27:20.709034 IP6 (flowlabel 0xd3867, hlim 57, next-header TCP (6) payload length: 40) 2001:xxx.54520 > 2003:xxx.56561: Flags [S], cksum 0xc709 (correct), seq 1683830560, win 64800, options [mss 1440,sackOK,TS val 2364136905 ecr 0,nop,wscale 6], length 0
08:27:20.709120 IP6 (flowlabel 0x3245d, hlim 63, next-header TCP (6) payload length: 40) 2003:xxx.56561 > 2001:xxx.54520: Flags [S.], cksum 0xcbfd (incorrect -> 0xf118), seq 3931317341, ack 1683830561, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 472230242 ecr 2364136905], length 0
08:27:21.739742 IP6 (flowlabel 0xf0fdf, hlim 57, next-header TCP (6) payload length: 40) 2001:xxx.54520 > 2003:xxx.56561: Flags [S], cksum 0xc31a (correct), seq 1683830560, win 64800, options [mss 1440,sackOK,TS val 2364137912 ecr 0,nop,wscale 6], length 0
08:27:21.739780 IP6 (flowlabel 0x3245d, hlim 63, next-header TCP (6) payload length: 40) 2003:xxx.56561 > 2001:xxx.54520: Flags [S.], cksum 0xcbfd (incorrect -> 0xed29), seq 3931317341, ack 1683830561, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 472230242 ecr 2364137912], length 0




So, I don´t know what´s my problem....
It seems, ipv6 doesn´t work from Internet to WAN (over pppoe), but otherwhise from LAN to Internet works fine.

In dumps there is somthing like checksum incorrect?!

Greets

Byte

[Maurice]

Another user reported similar issues with a Deutsche Telekom DSL line on the German forum:
IPv6 auf WAN nicht erreichbar
Maybe you can work on this together. I don't have a PPPoE line so I can only guess.

[Bytechanger]

Thanks,

seems it could be the same problem.
I´m also telekom. But robgnu writes on pfsense it works.
So is it possible, it´s a opnsense bug?

Greets

Byte

[Bytechanger]

OK, It seems, there is something special crazy?!

When I disable automatic created Gateway in

Gateway->Single->WAN_DHCP6
  WAN_DHCP6    WAN    IPv6    254    fe80::f6b5:2fff:fef0:a2eb

(crazyly it has a link-local address)

it works, I can acces ssh from outside!!
So I think, there is something wrong with opnsense ?!?!


Greets

Byte

[abraxxa]

IPv6 default gateways are always link-local addresses (at least if your product follows the RfCs).

Check the destination layer 2 fields of your outgoing IPv6 packets and compare the syn-ack of an incoming connection with a regular packet going through the firewall.

[Bytechanger]

Thanks,
but what/how exactly should I do?

Greets

Byte

[abraxxa]

Compare the destination mac address of IPv6 packets from your network to the Internet with syn-ack response packets when accessing a service on the firewall.

[Bytechanger]

OK, can you tell me the command for this please?

with
sudo tcpdump -eni pppoe0 'tcp port 56561'
on OPNSense, I see only ipv6 no mac?!

[abraxxa]

Just the tcpdump you already did before but write it to a file using -w filename.pcap and then copy it to your PC and load it in Wireshark.
Alternatively you can use the -e flag.

[Bytechanger]

I test it, on OPNSense -e doesn´t show MACs, only ipv6 adresses....
On pcap file also??

Code Select Expand


sudo tcpdump -eni pppoe0 'tcp port 56561'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pppoe0, link-type NULL (BSD loopback), capture size 262144 bytes
12:30:50.888384 AF IPv6 (28), length 84: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [S], seq 459488888, win 64800, options [mss 1440,sackOK,TS val 3410368836 ecr 0,nop,wscale 6], length 0
12:30:50.888496 AF IPv6 (28), length 84: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [S.], seq 2539208736, ack 459488889, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 2303200564 ecr 3410368836], length 0
12:30:50.906850 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 1, win 1013, options [nop,nop,TS val 3410368854 ecr 2303200564], length 0
12:30:50.907048 AF IPv6 (28), length 117: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1:42, ack 1, win 1013, options [nop,nop,TS val 3410368855 ecr 2303200564], length 41
12:30:50.907064 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 42, win 128, options [nop,nop,TS val 2303200583 ecr 3410368855], length 0
12:30:50.920355 AF IPv6 (28), length 133: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 1:58, ack 42, win 128, options [nop,nop,TS val 2303200596 ecr 3410368855], length 57
12:30:50.938217 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 58, win 1013, options [nop,nop,TS val 3410368886 ecr 2303200596], length 0
12:30:50.938242 AF IPv6 (28), length 1132: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 58:1114, ack 42, win 128, options [nop,nop,TS val 2303200614 ecr 3410368886], length 1056
12:30:50.938759 AF IPv6 (28), length 1436: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 42:1402, ack 58, win 1013, options [nop,nop,TS val 3410368886 ecr 2303200596], length 1360
12:30:50.938778 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 1402, win 126, options [nop,nop,TS val 2303200614 ecr 3410368886], length 0
12:30:50.959396 AF IPv6 (28), length 124: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1402:1450, ack 1114, win 1002, options [nop,nop,TS val 3410368907 ecr 2303200614], length 48
12:30:50.959418 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 1450, win 128, options [nop,nop,TS val 2303200635 ecr 3410368907], length 0
12:30:50.967390 AF IPv6 (28), length 584: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 1114:1622, ack 1450, win 128, options [nop,nop,TS val 2303200643 ecr 3410368907], length 508
12:30:50.989428 AF IPv6 (28), length 92: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1450:1466, ack 1622, win 1002, options [nop,nop,TS val 3410368937 ecr 2303200643], length 16
12:30:50.989449 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 1466, win 128, options [nop,nop,TS val 2303200665 ecr 3410368937], length 0
12:30:51.007402 AF IPv6 (28), length 120: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1466:1510, ack 1622, win 1002, options [nop,nop,TS val 3410368955 ecr 2303200665], length 44
12:30:51.007444 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 1510, win 128, options [nop,nop,TS val 2303200683 ecr 3410368955], length 0
12:30:51.007523 AF IPv6 (28), length 120: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 1622:1666, ack 1510, win 128, options [nop,nop,TS val 2303200683 ecr 3410368955], length 44
12:30:51.025398 AF IPv6 (28), length 144: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1510:1578, ack 1666, win 1002, options [nop,nop,TS val 3410368973 ecr 2303200683], length 68
12:30:51.025441 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 1578, win 128, options [nop,nop,TS val 2303200701 ecr 3410368973], length 0
12:30:51.030829 AF IPv6 (28), length 120: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 1666:1710, ack 1578, win 128, options [nop,nop,TS val 2303200707 ecr 3410368973], length 44
12:30:51.048840 AF IPv6 (28), length 216: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1578:1718, ack 1710, win 1002, options [nop,nop,TS val 3410368997 ecr 2303200707], length 140
12:30:51.048861 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 1718, win 128, options [nop,nop,TS val 2303200725 ecr 3410368997], length 0
12:30:51.049712 AF IPv6 (28), length 176: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 1710:1810, ack 1718, win 128, options [nop,nop,TS val 2303200725 ecr 3410368997], length 100
12:30:51.069546 AF IPv6 (28), length 304: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1718:1946, ack 1810, win 1002, options [nop,nop,TS val 3410369017 ecr 2303200725], length 228
12:30:51.069566 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 1946, win 128, options [nop,nop,TS val 2303200745 ecr 3410369017], length 0
12:30:51.075459 AF IPv6 (28), length 104: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 1810:1838, ack 1946, win 128, options [nop,nop,TS val 2303200751 ecr 3410369017], length 28
12:30:51.093479 AF IPv6 (28), length 188: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 1946:2058, ack 1838, win 1002, options [nop,nop,TS val 3410369041 ecr 2303200751], length 112
12:30:51.093550 AF IPv6 (28), length 872: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 1838:2634, ack 2058, win 128, options [nop,nop,TS val 2303200769 ecr 3410369041], length 796
12:30:51.155121 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 2634, win 1002, options [nop,nop,TS val 3410369103 ecr 2303200769], length 0
12:30:51.155143 AF IPv6 (28), length 120: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 2634:2678, ack 2058, win 128, options [nop,nop,TS val 2303200831 ecr 3410369103], length 44
12:30:51.173341 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 2678, win 1002, options [nop,nop,TS val 3410369121 ecr 2303200831], length 0
12:30:51.173376 AF IPv6 (28), length 520: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 2058:2502, ack 2678, win 1002, options [nop,nop,TS val 3410369121 ecr 2303200831], length 444
12:30:51.173390 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 2502, win 127, options [nop,nop,TS val 2303200849 ecr 3410369121], length 0
12:30:51.175004 AF IPv6 (28), length 184: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 2678:2786, ack 2502, win 128, options [nop,nop,TS val 2303200850 ecr 3410369121], length 108
12:30:51.175154 AF IPv6 (28), length 176: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 2786:2886, ack 2502, win 128, options [nop,nop,TS val 2303200850 ecr 3410369121], length 100
12:30:51.175294 AF IPv6 (28), length 440: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 2886:3250, ack 2502, win 128, options [nop,nop,TS val 2303200851 ecr 3410369121], length 364
12:30:51.175348 AF IPv6 (28), length 360: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 3250:3534, ack 2502, win 128, options [nop,nop,TS val 2303200851 ecr 3410369121], length 284
12:30:51.182243 AF IPv6 (28), length 112: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 3534:3570, ack 2502, win 128, options [nop,nop,TS val 2303200858 ecr 3410369121], length 36
12:30:51.193282 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 2886, win 1002, options [nop,nop,TS val 3410369141 ecr 2303200850], length 0
12:30:51.193733 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 3534, win 1002, options [nop,nop,TS val 3410369142 ecr 2303200851], length 0
12:30:51.242917 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 3570, win 1002, options [nop,nop,TS val 3410369191 ecr 2303200858], length 0
12:31:08.458459 AF IPv6 (28), length 112: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [P.], seq 2502:2538, ack 3570, win 1002, options [nop,nop,TS val 3410386406 ecr 2303200858], length 36
12:31:08.458506 AF IPv6 (28), length 76: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [.], ack 2538, win 128, options [nop,nop,TS val 2303218134 ecr 3410386406], length 0
12:31:08.458883 AF IPv6 (28), length 112: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 3570:3606, ack 2538, win 128, options [nop,nop,TS val 2303218134 ecr 3410386406], length 36
12:31:08.458978 AF IPv6 (28), length 112: 2003:(IP of WAN OPNSense).56561 > 2001:(IP of InternetServer).49268: Flags [P.], seq 3606:3642, ack 2538, win 128, options [nop,nop,TS val 2303218134 ecr 3410386406], length 36
12:31:08.476409 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 3606, win 1002, options [nop,nop,TS val 3410386424 ecr 2303218134], length 0
12:31:08.476645 AF IPv6 (28), length 76: 2001:(IP of InternetServer).49268 > 2003:(IP of WAN OPNSense).56561: Flags [.], ack 3642, win 1002, options [nop,nop,TS val 3410386424 ecr 2303218134], length 0


OK, with sudo tcpdump -w test3.pcap
I get a looooot of data and don´t know, what to do...


Greets

Byte

[robgnu]

Hello,

I am the user from the german forum with the same issue:
I have two separate OPNsense boxes with 20.1.7 connected by PPPoE (VDSL, Deutsche Telekom). Both machines do exactly the same:
- Connecting to the ISP works perfect. MTU was not set, but calculated to 1492, which is correct.
- All PCs behind the Firewall get clean IPv6 connections and traffic to the internet.
- One machine BEHIND the firewall is reachable from the internet (SSH; I configured the according rule)
- IPv6 connections from the Internet to the WAN port of the OPNsense are getting a timeout (SSH, HTTPS etc.)
- IPv4 connections from the Internet to the WAN port of the OPNsense working as usual and fast.
- Both OPNsense machines were fresh installed with 20.1.0, then I configured the PPPoE connection and then I upgraded to the latest version.

I figured out, that disabling the Gateway "WAN_DHCP6" solves that issue. On one machine I have now a continuously restarting radvd daemon (every 2 seconds) which is another IPv6 problem, I think.

I logged into these two machines an did a "clog -f /var/log/system.log". There is a massive flooding of messages (both OPNsenses). In parallel I did the same on another pfSense, there aren't those messages:

Code Select Expand


May 28 19:31:12 sense dhcp6c[67533]: script "/var/etc/dhcp6c_wan_script.sh" terminated
May 28 19:31:12 sense dhcp6c[67533]: removing an event on pppoe0, state=REQUEST
May 28 19:31:12 sense dhcp6c[67533]: removing server (ID: 00:02:00:00:05:83:34:30:3a:37:31:3a:38:33:3a:61:38:3a:63:38:3a:30:30:00:00:00)
May 28 19:31:12 sense dhcp6c[67533]: got an expected reply, sleeping.
May 28 19:31:12 sense dhcp6c[67533]: Sending Solicit
May 28 19:31:12 sense dhcp6c[67533]: a new XID (15754b) is generated
May 28 19:31:12 sense dhcp6c[67533]: set client ID (len 14)
May 28 19:31:12 sense dhcp6c[67533]: set identity association
May 28 19:31:12 sense dhcp6c[67533]: set elapsed time (len 2)
May 28 19:31:12 sense dhcp6c[67533]: set option request (len 4)
May 28 19:31:12 sense dhcp6c[67533]: send solicit to ff02::1:2%pppoe0
May 28 19:31:12 sense dhcp6c[67533]: reset a timer on pppoe0, state=SOLICIT, timeo=0, retrans=1016
May 28 19:31:12 sense dhcp6c[67533]: receive advertise from fe80::200:ff:fe00:0%pppoe0 on pppoe0
May 28 19:31:12 sense dhcp6c[67533]: get DHCP option client ID, len 14
May 28 19:31:12 sense dhcp6c[67533]:   DUID: 00:01:00:01:21:39:d6:87:00:0d:b9:54:2b:6c
May 28 19:31:12 sense dhcp6c[67533]: get DHCP option server ID, len 26
May 28 19:31:12 sense dhcp6c[67533]:   DUID: 00:02:00:00:05:83:34:30:3a:37:31:3a:38:33:3a:61:38:3a:63:38:3a:30:30:00:00:00
May 28 19:31:12 sense dhcp6c[67533]: get DHCP option identity association, len 59
May 28 19:31:12 sense dhcp6c[67533]:   IA_NA: ID=0, T1=0, T2=0
May 28 19:31:12 sense dhcp6c[67533]: get DHCP option status code, len 43
May 28 19:31:12 sense dhcp6c[67533]:   status code: no addresses
May 28 19:31:12 sense dhcp6c[67533]: get DHCP option DNS, len 32
May 28 19:31:12 sense dhcp6c[67533]: server ID: 00:02:00:00:05:83:34:30:3a:37:31:3a:38:33:3a:61:38:3a:63:38:3a:30:30:00:00:00, pref=-1
May 28 19:31:12 sense dhcp6c[67533]: reset timer for pppoe0 to 0.979450
May 28 19:31:13 sense dhcp6c[67533]: picked a server (ID: 00:02:00:00:05:83:34:30:3a:37:31:3a:38:33:3a:61:38:3a:63:38:3a:30:30:00:00:00)
May 28 19:31:13 sense dhcp6c[67533]: Sending Request
May 28 19:31:13 sense dhcp6c[67533]: a new XID (ac85ba) is generated
May 28 19:31:13 sense dhcp6c[67533]: set client ID (len 14)
May 28 19:31:13 sense dhcp6c[67533]: set server ID (len 26)
May 28 19:31:13 sense dhcp6c[67533]: set status code
May 28 19:31:13 sense dhcp6c[67533]: set identity association
May 28 19:31:13 sense dhcp6c[67533]: set elapsed time (len 2)
May 28 19:31:13 sense dhcp6c[67533]: set option request (len 4)
May 28 19:31:13 sense dhcp6c[67533]: send request to ff02::1:2%pppoe0
May 28 19:31:13 sense dhcp6c[67533]: reset a timer on pppoe0, state=REQUEST, timeo=0, retrans=1059
May 28 19:31:13 sense dhcp6c[67533]: receive reply from fe80::200:ff:fe00:0%pppoe0 on pppoe0
May 28 19:31:13 sense dhcp6c[67533]: get DHCP option client ID, len 14
May 28 19:31:13 sense dhcp6c[67533]:   DUID: 00:01:00:01:21:39:d6:87:00:0d:b9:54:2b:6c
May 28 19:31:13 sense dhcp6c[67533]: get DHCP option server ID, len 26
May 28 19:31:13 sense dhcp6c[67533]:   DUID: 00:02:00:00:05:83:34:30:3a:37:31:3a:38:33:3a:61:38:3a:63:38:3a:30:30:00:00:00
May 28 19:31:13 sense dhcp6c[67533]: get DHCP option identity association, len 59
May 28 19:31:13 sense dhcp6c[67533]:   IA_NA: ID=0, T1=0, T2=0
May 28 19:31:13 sense dhcp6c[67533]: get DHCP option status code, len 43
May 28 19:31:13 sense dhcp6c[67533]:   status code: no addresses
May 28 19:31:13 sense dhcp6c[67533]: get DHCP option DNS, len 32
May 28 19:31:13 sense dhcp6c[67533]: Received REPLY for REQUEST
May 28 19:31:13 sense dhcp6c[67533]: nameserver[0] 2003:180:2:7000::53
May 28 19:31:13 sense dhcp6c[67533]: nameserver[1] 2003:180:2:9000::53
May 28 19:31:13 sense dhcp6c[67533]: make an IA: NA-0
May 28 19:31:13 sense dhcp6c[67533]: status code for NA-0: no addresses
May 28 19:31:13 sense dhcp6c[67533]: IA NA-0 is invalidated
May 28 19:31:13 sense dhcp6c[67533]: remove an IA: NA-0
May 28 19:31:13 sense dhcp6c[67533]: reset a timer on pppoe0, state=INIT, timeo=0, retrans=113
May 28 19:31:13 sense dhcp6c[67533]: executes /var/etc/dhcp6c_wan_script.sh
May 28 19:31:13 sense dhcp6c: dhcp6c REQUEST on pppoe0 - running newipv6
May 28 19:31:14 sense opnsense: /usr/local/etc/rc.newwanipv6: IPv6 renewal is starting on 'pppoe0'
May 28 19:31:14 sense opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: 2003:a:xxxx:xxxx:20d:b9ff:fe54:2b6c) (interface: WAN[wan]) (real interface: pppoe0).


For now, I'm out of ideas. I switched over from pfSense for some other good reasons, but I never had any problems with IPv6 on pfSense.

Robert.

[robgnu]

Hello,

I did some more investigation this evening. One post in this forum (https://forum.opnsense.org/index.php?topic=17434.msg79254#msg79254) gave the hint, to disable reply-to in the firewall settings. After this tip and re-enableing the IPv6 gateway back, I am able to connect to the WAN interface (TCP/UDP).

I don't understand why - I don't have a Multi-WAN setup, just simple PPPoE. Is there a difference to pfSense?

Robert.