Messages

I think this part is not needed:

Step 4a - Setup Firewall Site A
Go to Firewall ‣ Settings ‣ Normalization and add a new rule to prevent fragmentation of traffic going through the wireguard tunnel.

https://docs.opnsense.org/manual/how-tos/wireguard-s2s.html


At least disabling it solved my problem
https://forum.opnsense.org/index.php?topic=39717.0

FYI

disable MSS clamping  :-[


Problem was caused by bad config, probably i had followed some tutorial or this is not necessary anymore.

@ Firewall: Settings: Normalization
I had a setting for the WG-Group enforcing a max MSS.
Disabling this resolved all problems with WG


Clients did not reach LAN in a site to site setup.


Symptoms were with opn being the "client" i.e. initiating the connection to another "server".
( Client <-> WG relay server <-> opn <-> LAN )

While opn being the "server" was working fine, but not a reliable option due to dynamic IP.
( Client <-> opn <-> LAN )

Man kann die Range ohne das Präfix in der Form 

Code Select Expand

:: - ::ffff:ffff:ffff:ffff angeben. Auch z.B. als ::100-::abc.

# echo "keymap='de'">>/etc/rc.conf

;)

Hi,

hard to guess, I'd start with playing around with the settings at -> Interfaces -> Settings
starting with toggle: Hardware TSO


And keep an eye on interrupt/CPU utilisation during network activity
e.g.
sudo top -PSH

According to your screenshot there is no vlan10 on igb2. Thus you can't connect to vlan10  in the DMZ.



Is there no switch involved anymore?

Don't crate a 2nd vlan 10 on igb2, let the switch handle that single vlan10 on the igb1 port.

Also I'd use a 'real' Trunk between igb1 and switch, with tagged LAN and tagged Mngmt and let the switch hand out an untagged LAN to the clients port.

Yes, leave MSS empty.

We expect the loss at 1445 due to PPPoE.


ok,
setting MTU to 1492 and max packet of 1436 does mean that it is correctly set automatically, i.e. pppoe subtracts its 8 Byte header automatically.
The default of 1500 -8 B pppoe results in max packet of 1444.

So it's fine having MTU blank/default/1500 on the pppoe interface.



I'm setting MTU 1492 everywhere on the LANs with traffic through WAN/PPPoE and JumboFrames on the local-only LAN. So my LAN and DMZ get 1492 and NAS-network is at 9k.




Seems like your webserver doesn't like the reduced MTU? Can you set MTU 1492 directly on its interface.

Sorry for late reply.

Use MTU 1492

IP4 with (MSS = 1452) + ( 20B TCP + 20B IP4 +8B PPPoE) = MTU 1500 default ethernet.
IP6-Header is 40 Bytes, thus MSS 1432.

With 1500 -8B PPPoE -40B IPv6 -8B ICMP = 1444 packetsize you can
ping6 -c1 -s 1444 example.net
wich passes, while
ping6 -c1 -s 1445 example.net
times out.

Please test which is largest packetsize for ping.

There might be some encapsulating overhad on the route (VXLAN VPN ...)

Your internal DNS-server knows the internal names, the external servers know about the other ones.

The answer is either cached, or forwarded to the external server or forwarded to the internal server. You'd need to be lucky getting the internal server for lan-adresses.


You can either forward ALL queries to your internal server and let that one resolve internal and internet names
or
create forward-zones for your internal names with the internal server and a default zone like in 'Config b'.



Another option for only a few names would be overrides.

There are MSS of 1440 (default) and 1432 + 8Bytes for PPPoE.

Seems like Path MTU Discovery is firewalled.

Can you try permitting ICMP on WAN? At least IPv6-ICMP type "Packet too big".

So, it's possible to wire all 4 ports of opnsense to the switch as trunk and opnsense performs some kind of 'load balancing' (4x GBit)?

Configured as LAGG on switch and opnsense. You don't want that.



You could use your ports like
* untagged WAN
* several tagged OPTs and/or tagged LAN1 -> to managed switch
* untagged LAN2 (optional) -> for a unmanaged switch
* spare Port or management

The switch will tag/untag packets. You get a Trunk port with tagged VLANs between opnsense and switch.

E.g. one port on the switch gets the VLAN 90 untagged with PVID 90 if the servers NIC is not vlan-aware. Another server connects to a switch-port with tagged VLAN90 and VLAN20 then the server has to be configured with a VLAN90 and a VLAN20 on this Trunk. A client gets a port on the switch with untagged VLAN1/PVID1.

I'd try using e.g. 192.168.253.0/24 for Wireguard.

10/8 might be routed/NATed somewhere else already.

Hi,

interesting.

How much information provides tcpdump?

Code Select Expand

sudo tcpdump -ni WAN-Interface 'tcp port 56573'

Is there any response from haproxy?

At least some TCP-stuff?

Might be something with MTU? You're using PPPoE so we expect mss 1452.

[now]

Seems like haproxy isn't responding on WAN-IPv6.

Listening on all addresses:port is three colons in a row followed by port

Code Select Expand

:::443 not [::]:443, not [::1]:443, not ::1:443.

Frontend hearing on ipv4 0.0.0.0:56573 ipv6 [::1]:56573

Does haproxy public service now listen on

Code Select Expand

:::56573
2003:xx:xxx:xxxx:xxx:xxxx:xxxx:8583:56573
?
all IPv6 and/or WAN-IP6?
Without [] and NOT ::1?
[::1]:56573
[::]:56573
:::56573

wget  -O- --no-check-certificate https://[2003::LAN]:56573 from LAN is fine?
wget  -O- --no-check-certificate https://[2003::WAN:8583]:56573 from LAN is fine?

wget  -O- --no-check-certificate https://[2003::WAN:8583]:56573 from WAN passes firewall with datalen=0.
wget  -O- --no-check-certificate https://[2003::LAN]:56573 from WAN?

[::]:56573 doesn´t work

Tho it looks funny, the listen address:port is 0.0.0.0:443 :::443 localhost:443

Code Select Expand

:::56573