Show Posts
1
Hardware and Performance / Wan speed with opnsense and a r86s u4 is slower than expected
« on: May 22, 2024, 01:19:24 pm »
Hi,
I've been using pfsense for a long time but my isp recently installed a 10/10 fiber in my home, so I took the opportunity to reinstall everything and finally try opnsense.
This is my final/desired configuration:
The isp router is connected in "dmz mode" with a rj45 to an opnsense router (I'll replace it with a bridge in a few days).
On the router there are some vlans with some rules (lan, lansrv, security, linuxisosdownload, ...), haproxy, unbound,..
Everything is trunked to a MikroTik CRS305 via SFP+, with no untagged traffic (the physical interface is enabled but without ip assigned).
On the MikroTik:
- the rj45 port goest to an unifi switch (wap, appletv, server's ipmi port).
- one sfp+ goest to my proxmox machine (network card: connectx-3) (trunk).
- one sfp+ goes to my pc (network card: connectx-3) (untagged).
The problem is that with a gowin r86s u4 i can "only" reach max 5000/5000.
So I did some tests removing the switch and vlans from the equation.
I though the problem was caused by the r86s but since it seems to be a lot faster with pfsense there must be something else (freebsd version, some misterious settings...).
I also did some crossvlan iperf between the pc and a vm, with both devices I can reach around 6 Gbits/sec.
Any ideas on how to investigate this? I don't think i will go back to pfsense because 5000/5000 is not really a problem
but I'd like to understand why this happens.
Thanks!
I've been using pfsense for a long time but my isp recently installed a 10/10 fiber in my home, so I took the opportunity to reinstall everything and finally try opnsense.
This is my final/desired configuration:
The isp router is connected in "dmz mode" with a rj45 to an opnsense router (I'll replace it with a bridge in a few days).
On the router there are some vlans with some rules (lan, lansrv, security, linuxisosdownload, ...), haproxy, unbound,..
Everything is trunked to a MikroTik CRS305 via SFP+, with no untagged traffic (the physical interface is enabled but without ip assigned).
On the MikroTik:
- the rj45 port goest to an unifi switch (wap, appletv, server's ipmi port).
- one sfp+ goest to my proxmox machine (network card: connectx-3) (trunk).
- one sfp+ goes to my pc (network card: connectx-3) (untagged).
The problem is that with a gowin r86s u4 i can "only" reach max 5000/5000.
So I did some tests removing the switch and vlans from the equation.
| from | to | notes | speed |
| isp router | directly to pc | 10gbase-t transceiver | 7900/7900 |
| r86s u4 opnsense | directly to pc | spectre/meltdown mitigations ON | 5000/5000 |
| r86s u4 opnsense | directly to pc | spectre/meltdown mitigations OFF microcodes update | 5000/5000 |
| r86s u4 opnsense | directly to pc | spectre/meltdown mitigations OFF | 5000/5000 |
| x9scm-f e3-1270v2 opnsense | directly to pc | spectre/meltdown mitigations ON | 5000/5000 |
| x9scm-f e3-1270v2 opnsense | directly to pc | spectre/meltdown mitigations OFF | 7600/7600 |
| r86s u4 pfsense | directly to pc | run the test only once so I don't have a real avg | 7500/7500 |
I though the problem was caused by the r86s but since it seems to be a lot faster with pfsense there must be something else (freebsd version, some misterious settings...).
I also did some crossvlan iperf between the pc and a vm, with both devices I can reach around 6 Gbits/sec.
Any ideas on how to investigate this? I don't think i will go back to pfsense because 5000/5000 is not really a problem
but I'd like to understand why this happens. Thanks!