[SOLVED] Low upload speed on VirtIO network + PPPoE + VLAN

[chadima]

Hi,

I'm a long OpenWRT user (routing a backbone network 5x 25Gbps).
After an OpenWRT upgrade at home, which caused a complete wipe,
I have decided to give OPNsense a try.

My home network parameters (GPON Cetin, Czech Republic):
PPPoE - user name and password
VLAN - 848
MTU - 1492
download - 2000Mbps
upload - 1000Mbps

I have force-installed the FreeBSD 13 package of Ookla Speedtest
and i get 2020Mbps for download and 150-350Mbps for upload
on the OPNsense firewall itself (means no NAT involved etc.).
I believe that this is caused by a wrong MTU somewhere,
but I'm unable to fix it (and I have tried many options).

Configuration of Proxmox:
Linux bridge vmbr0 - IP:192.168.1.2 GW:192.168.1.1 Bridge ports:enp2s0f0
Linux bridge vmbr1 - Bridge ports:enp2s0f1
Virtual machine net0 - Model: VirtIO Multiqueue: 8 Bridge: vmbr0
Virtual machine net1 - Model: VirtIO Multiqueue: 8 Bridge: vmbr1

Configuration of OPNsense:
Interfaces, Other Types, VLAN add vlan01 - Vlan tag: 848 Parent: vtnet1
Interfaces, Point-to-Point, Devices add pppoe0 - Link Type: PPPoE Parent interface: vlan01
(+ Username and Password)
Interfaces, Asignments interface [WAN] change Device to pppoe0

I have nothing else changed, all is left at the default values.
On the [WAN] Interface is shown - Calculated PPP MTU: 1492
The maximum ping size I can send from the firewall is ping -D -s 1464 1.1.1.1
which seems OK to me.

Can anybody help me please?

[chadima]

I have tried to delete the vlan01 and to set the PPPoE Parent interface to vtnet1
and at the same time set the Virtual machine net1 VLAN tag to 848,
which is correctly working, but has the same upload speed issue.

I didn't have this problem with upload speed using OpenWRT.

[bartjsmit]

Can you pass through the LAN and WAN NIC to OPNsense rather than bridge them with Proxmox?

https://www.wundertech.net/how-to-install-opnsense-in-proxmox/

[chadima]

I'm using PCIe passthrough a lot and I can give it a try.

The reason to use virtualized network cards was,
that I didn't have to plug the LAN from the firewall into the switch
and afterwards plug the Proxmox server itself into the switch... 

[chadima]

OK, problem solved using PCIe passthrough.

It looks like a problem in the FreeBSD network VirtIO driver... 

[bartjsmit]

You may want to try the various Proxmox NIC emulators (if you haven't already).

If only in the interest of science