Slow bandwidth from lan to wan (OPNsense on Proxmox VE)

[teapot9]

Hi,
I want to share with you my internet speed problem between lan and wan.
I have OPNsense installed as a VM in Proxmox virtual environment.

Problem :
Normally my internet connection provides me 900down and almost 300up, but OPNsense only gives lan 250-300down and 300up.
Install is new, not a lot of configuration done except vlan, dhcp and unbound.
Iperf3 from lan device to firewall gives 2-3 gbps speed.
And the speed seems fine for the firewall itself (used iperf3 to public servers and got 900down/300up) so the problem is between lan and wan.

Proxmox version : pve-manager/5.3-11/d4907f84 (running kernel: 4.15.18-12-pve)
OPNsense version : 19.1.4

Host hardware :
CPU : AMD Threadripper 1950x (16 cores @ 3.7 GHz)
Memory : 64 GB

VM config :
Memory : 8 GiB (no balloon)
CPU : EPYC, numa, 8 cores (2 sockets 4 cores)
Hard disk : SCSI 64 GB
Network device : virtio linux bridge (for LAN, VLAN aware)
PCIe passthrough : Intel I210-T1 gigabit NIC (for WAN)
BIOS : OVMF (UEFI)
Qemu agent : disabled

OPNsense configuration :
WAN : DHCP and DHCPv6
LAN : Static ipv4 and ipv6 with DHCP and DHCPv6 server (no prefix delegation available), router advertisements set to assisted
LAN is configured on VLAN 10 of the virtio bridge
WAN is a direct PCIe passthrough of the Intel NIC

Firewall configuration :
- Floating rule : Allow DNS from anywhere to this firewall
- LAN : Anti-Lockout rule
- LAN : Default allow LAN to any rule
- LAN : Default allow LAN IPv6 to any rule

Other problems that can be related :
- IPv6 works fine on firewall but no ipv6 for lan
- Unbound only works in forward mode

Any help really welcome
Thanks

[hbc]

Did you try the "new" netmap enabled kernel? Should perform better with virtual nics.

See here:
https://forum.opnsense.org/index.php?topic=11477.0

[teapot9]

Did you try the "new" netmap enabled kernel? Should perform better with virtual nics.

See here:
https://forum.opnsense.org/index.php?topic=11477.0

Thank you for your reply, I have just installed the netmap enabled kernel and restarted, but no speed change unfortunately

[TeKK]

I've installed OPNSense on Proxmox as well. And have had the same trouble with WAN speeds. Have you tried bridging the WAN interface to Proxmox rather than passing the device through directly to VM? PCI Passthrough is experimental in Proxmox from what I read.

[teapot9]

Thanks for your reply

However, I resolved the problem, it happened to be a configuration error of mine :
I changed a configuration in TCP configuration autotuninglevel from disabled to auto (default) and it instantly solved my problem.

Thank you for your help

N.B. Should I create a topic for my ipv6 problems ?

[tbard1]

I changed a configuration in TCP configuration autotuninglevel from disabled to auto (default) and it instantly solved my problem.

Hi yoshi_26_02,

Where did you found this parameter on OPNsense's Gui ?

Thanks

[guest15389]

It isn't in the GUI as it's a tunable.

I'm assuming they set:

  net.inet.tcp.sendbuf_auto=1
  net.inet.tcp.recvbuf_auto=1