haproxy for ipv4 and ipv6 to ipv4; ipv6 doesnt work?

[Bytechanger]

Hi,

On my opnsense haproxy is running.
Set a ipv4 Backend.

Frontend hearing on ipv4 0.0.0.0:56573 ipv6 [::1]:56573
but only ipv4 is working??
WAN ipv4 -> haproxy runs great
WAN ipv6 -> haproxy no reaction.

SSH on OPNSense:

Code: [Select]

sudo sockstat -6 | grep haproxy
www      haproxy    36535 22 tcp6   ::1:56573             *:*
So I think, haproxy is hearing on right ports.

Firewall is open on WAN to ipv4 and ipv6 for 56573

Where is my fault?

Greets

Byte

[Maurice]

The IPv6 equivalent of '0.0.0.0' is '::' (all zeros, unspecified address).

Cheers

Maurice

[Bytechanger]

Hm, OPNSense tells you to use

Code: [Select]

"Please provide a valid listen address, i.e. 127.0.0.1:8080, [::1]:8080 or www.example.com:443. Port range as start-end, i.e. 127.0.0.1:1220-1240."

[::]:56573 doesn´t work

but in ssh looks good

Code: [Select]

sudo sockstat -6 | grep haproxy
www      haproxy    42268 22 tcp6   *:56573               *:*


[Bytechanger]

Hi,

so internaly it works fine!
When I choose https://[2003:xx:xxx:xxxx:xxx:xxxx:xxxx:8584]:56573/ (IPv6 LAN-Adapter adress) it works fine.
But when I test and come from internet to WAN, nothing happens!?

I ssh into an IONOS-VServer and try to connect to my opnsense at home
  ping6 2003:xx:xxx:xxxx:xxx:xxxx:xxxx:8583
works fine.
But

wget  --no-check-certificate https://[2003:xx:xxx:xxxx:xxx:xxxx:xxxx:8583]:56573
--2020-05-22 13:46:55--  https://[2003:xx:xxx:xxxx:xxx:xxxx:xxxx:8583]:56573/
Connecting to 2003:xx:xxx:xxxx:xxx:xxxx:xxxx:8583]:56573...

ends there.....

wget --no-check-certificate https://87.xxx.xxx.16:56573 works fine also....

Any Idea??
Firewallrules are set to ipv4 and ipv6 opened on this ports...

Greets

Byte

[Maurice]

So, connecting to the LAN interface address from a host in the LAN works, but connecting to the WAN interface address from the Internet doesn't work, correct?

1. What about connecting to the WAN address from a host in the LAN?
2. Anything in the firewall logs when trying to connect from the Internet?

[johnsmi]

[::]:56573 doesn´t work

Tho it looks funny, the listen address:port is 0.0.0.0:443 :::443 localhost:443

Code: [Select]

:::56573

[Bytechanger]

OK, you mean, localy I put the IPv6 Adress of WAN-Interface?
This works too.

I´m not sure, I can handle filterlogs correct.
But when I filter my log (don´t know if it´s right) i can find some entry:

Code: [Select]

filterlog
134,,,0,pppoe0,match,pass,in,6,0x00,0xb70a5,58,tcp,6,40,2001:XX(IP from my IONOS Server),2003:(IP from my WAN),44608,56573,0,S,3312441647,,64800,,mss;sackOK;TS;nop;wscale

Can´t read all of it, but I think, it´s right and passing my firewall??

Greets

Byte

[johnsmi]

Seems like haproxy isn't responding on WAN-IPv6.

Listening on all addresses:port is three colons in a row followed by port

Code: [Select]

:::443 not [::]:443, not [::1]:443, not ::1:443.

Frontend hearing on ipv4 0.0.0.0:56573 ipv6 [::1]:56573

Does haproxy public service now listen on

Code: [Select]

:::56573
2003:xx:xxx:xxxx:xxx:xxxx:xxxx:8583:56573?
all IPv6 and/or WAN-IP6?
Without [] and NOT ::1?
[::1]:56573
[::]:56573
:::56573

wget  -O- --no-check-certificate https://[2003::LAN]:56573 from LAN is fine?
wget  -O- --no-check-certificate https://[2003::WAN:8583]:56573 from LAN is fine?

wget  -O- --no-check-certificate https://[2003::WAN:8583]:56573 from WAN passes firewall with datalen=0.
wget  -O- --no-check-certificate https://[2003::LAN]:56573 from WAN?

[Bytechanger]

Hi,

:::56573 or [::]:56573 has the same result in haproxy
Especially when you ssh into opnsense and

Code: [Select]

sudo sockstat -6 | grep haproxy
Password:
www      haproxy    2683  6  tcp6   *:56573               *:*

So sockstat tells it is listening to *:56573, and I think it´s for all interfaces.


wget  -O- --no-check-certificate https://[2003::LAN]:56573 from LAN is fine? YES
wget  -O- --no-check-certificate https://[2003::WAN:8583]:56573 from LAN is fine? YES


wget  -O- --no-check-certificate https://[2003::WAN:8583]:56573 from WAN passes firewall with datalen=0.
wget  -O- --no-check-certificate https://[2003::LAN]:56573 same as above... datalen=0

hm, crazy

Greets

Byte

[johnsmi]

Hi,

interesting.

How much information provides tcpdump?

Code: [Select]

sudo tcpdump -ni WAN-Interface 'tcp port 56573'
Is there any response from haproxy?

At least some TCP-stuff?

Might be something with MTU? You're using PPPoE so we expect mss 1452.

[Bytechanger]

Hi,

thanks for helping, here is my output for tcpdump
(I change to port 56571)

Code: [Select]

sudo tcpdump -ni pppoe0 'tcp port 56571'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pppoe0, link-type NULL (BSD loopback), capture size 262144 bytes
21:59:31.099942 IP6 2001:(IP of my IONOS Server).52084 > 2003:(IP of WAN).56571: Flags [S], seq 2826379982, win 64800, options [mss 1440,sackOK,TS val 3003487412 ecr 0,nop,wscale 6], length 0
21:59:31.100008 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003487412], length 0
21:59:32.127004 IP6 2001:(IP of my IONOS Server).52084 > 2003:(IP of WAN).56571: Flags [S], seq 2826379982, win 64800, options [mss 1440,sackOK,TS val 3003488438 ecr 0,nop,wscale 6], length 0
21:59:32.127051 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003488438], length 0
21:59:34.143015 IP6 2001:(IP of my IONOS Server).52084 > 2003:(IP of WAN).56571: Flags [S], seq 2826379982, win 64800, options [mss1440,sackOK,TS val 3003490454 ecr 0,nop,wscale 6], length 0
21:59:34.143054 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003490454], length 0
21:59:37.144058 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003490454], length 0
21:59:38.303321 IP6 2001:(IP of my IONOS Server).52084 > 2003:(IP of WAN).56571: Flags [S], seq 2826379982, win 64800, options [mss1440,sackOK,TS val 3003494614 ecr 0,nop,wscale 6], length 0
21:59:38.303358 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003494614], length 0
21:59:41.303356 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003494614], length 0
21:59:44.503085 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003494614], length 0
21:59:46.494985 IP6 2001:(IP of my IONOS Server).52084 > 2003:(IP of WAN).56571: Flags [S], seq 2826379982, win 64800, options [mss1440,sackOK,TS val 3003502806 ecr 0,nop,wscale 6], length 0
21:59:46.495047 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003502806], length 0
21:59:49.496584 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003502806], length 0
21:59:52.696136 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003502806], length 0
21:59:55.896200 IP6 2003:(IP of WAN).56571 > 2001:(IP of my IONOS Server).52084: Flags [S.], seq 983975308, ack 2826379983, win 65228, options [mss 1432,nop,wscale 9,sackOK,TS val 4023452836 ecr 3003502806], length 0

I don´t see any response from haproxy, in protocol I also can´t see anything

MTU?
On Interfaces->WAN->MTU is empty, under field is shown: Calculated PPP MTU: 1492
MSS is also empty

when accessing with ipv4, working is so

Code: [Select]

listening on pppoe0, link-type NULL (BSD loopback), capture size 262144 bytes
22:17:19.118020 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [S], seq 1756100036, win 64240, options [mss 1452,sackOK,TS val 1030447999 ecr 0,nop,wscale 6], length 0
22:17:19.118083 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [S.], seq 3225430295, ack 1756100037, win 65228,options [mss 1452,nop,wscale 9,sackOK,TS val 4149591175 ecr 1030447999], length 0
22:17:19.134375 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [.], ack 1, win 1004, options [nop,nop,TS val 1030448016 ecr 4149591175], length 0
22:17:19.135332 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [P.], seq 1:319, ack 1, win 1004, options [nop,nop,TS val 1030448017 ecr 4149591175], length 318
22:17:19.135354 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], ack 319, win 126, options [nop,nop,TS val 4149591192 ecr 1030448017], length 0
22:17:19.152703 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], seq 1:1441, ack 319, win 127, options [nop,nop,TS val 4149591209 ecr 1030448017], length 1440
22:17:19.152722 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], seq 1441:2881, ack 319, win 127, options [nop,nop,TS val 4149591209 ecr 1030448017], length 1440
22:17:19.152734 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [P.], seq 2881:3623, ack 319, win 127, options [nop,nop,TS val 4149591209 ecr 1030448017], length 742
22:17:19.170552 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [.], ack 1441, win 1002, options [nop,nop,TS val1030448052 ecr 4149591209], length 0
22:17:19.171227 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [.], ack 2881, win 1002, options [nop,nop,TS val1030448053 ecr 4149591209], length 0
22:17:19.171929 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [.], ack 3623, win 1002, options [nop,nop,TS val1030448053 ecr 4149591209], length 0
22:17:19.172845 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [P.], seq 319:399, ack 3623, win 1002, options [nop,nop,TS val 1030448054 ecr 4149591209], length 80
22:17:19.172866 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], ack 399, win 127, options [nop,nop,TS val 4149591229 ecr 1030448054], length 0
22:17:19.173059 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [P.], seq 3623:3702, ack 399, win 127, options [nop,nop,TS val 4149591230 ecr 1030448054], length 79
22:17:19.173139 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [P.], seq 3702:3781, ack 399, win 127, options [nop,nop,TS val 4149591230 ecr 1030448054], length 79
22:17:19.189451 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [P.], seq 399:576, ack 3623, win 1002, options [nop,nop,TS val 1030448071 ecr 4149591229], length 177
22:17:19.189481 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], ack 576, win 127, options [nop,nop,TS val 4149591247 ecr 1030448071], length 0
22:17:19.190099 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [.], ack 3781, win 1002, options [nop,nop,TS val1030448071 ecr 4149591230], length 0
22:17:19.208669 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], seq 3781:5221, ack 576, win 127, options [nop,nop,TS val 4149591266 ecr 1030448071], length 1440
22:17:19.208701 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], seq 5221:6661, ack 576, win 127, options [nop,nop,TS val 4149591266 ecr 1030448071], length 1440
22:17:19.208713 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], seq 6661:8101, ack 576, win 127, options [nop,nop,TS val 4149591266 ecr 1030448071], length 1440
22:17:19.208725 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [P.], seq 8101:8206, ack 576, win 127, options [nop,nop,TS val 4149591266 ecr 1030448071], length 105
22:17:19.208805 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [P.], seq 8206:9591, ack 576, win 127, options [nop,nop,TS val 4149591266 ecr 1030448071], length 1385
22:17:19.227697 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [.], ack 6661, win 1002, options [nop,nop,TS val1030448109 ecr 4149591266], length 0
22:17:19.229938 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [.], ack 8206, win 1002, options [nop,nop,TS val1030448111 ecr 4149591266], length 0
22:17:19.231738 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [F.], seq 576, ack 9591, win 1002, options [nop,nop,TS val 1030448113 ecr 4149591266], length 0
22:17:19.231759 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [.], ack 577, win 127, options [nop,nop,TS val 4149591288 ecr 1030448113], length 0
22:17:19.231813 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [P.], seq 9591:9615, ack 577, win 127, options [nop,nop,TS val 4149591288 ecr 1030448113], length 24
22:17:19.231871 IP 87:(IP of my WAN).56571 > 82:(IP of my IONOS Server).40268: Flags [F.], seq 9615, ack 577, win 127, options [nop,nop,TS val 4149591288 ecr 1030448113], length 0
22:17:19.248161 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [R], seq 1756100613, win 0, length 0
22:17:19.248183 IP 82:(IP of my IONOS Server).40268 > 87:(IP of my WAN).56571: Flags [R], seq 1756100613, win 0, length 0
[code]
Greets

Byte

[johnsmi]

There are MSS of 1440 (default) and 1432 + 8Bytes for PPPoE.

Seems like Path MTU Discovery is firewalled.

Can you try permitting ICMP on WAN? At least IPv6-ICMP type "Packet too big".

[Bytechanger]

If you mean a firewall rule on WAN allowing ICMP, it´s aleready there as last rule (IPv6 ICMP pass).
Because of this, ping6 is possible to WAN address from outsite.

Greets

Byte

[Bytechanger]

Should I set MTU to 1452 in Interface->WAN?
Or to anything else?
OR MSS to 1452 ?


Greets

Byte

[Bytechanger]

Any idea?
Is this a haproxy problem?
How can I check this? Any other traffic to WAN without haproxy?
Need help, please


Greets

Byte