Messages

hello , the nic is
SFP+ 2 Ports LC - CHIPSET BROADCOM BCM57810 - 10GbE Ethernet Fiber Network Adapter

I will try iperf3 later

Hello

firewall in proxmox (on pve itself, bridge, nic) is disabled.

In fact, i have switched to untangle with lots of performance improvement.

But i will try to find why this problem with opnsense

Hello all (and happy new year)

i'm asking expert after lots of time trying and searching.

My lab is

Fiber Freebox Delta bridge mode (Free / France) 10Gb/s Download / 700 MB Upload

Storage
  Truenas Scale = 1x10GB SFP+

A cluster of 3 proxmox
  Proxmox 1 = 2x10Gb SFP+
  Proxmox 2 = 1x1Gb ethernet
  Proxmox 3 = 1x2.5Gb Ethernet



On proxmox 1:
  linux brige (vmbr0) link with nic0 LAN
  linux brige (vmbr1) link with nic1 WAN

  First VM (opnsense) with 2 NIC
    NIC0 LAN (on vmbr0)
    NIC1 WAN (on vmbr1)

  Second VM (alpine linux) with one NIC
    NIC0 LAN (on vmbr0)

On proxmox 2:
  linux brige (vmbr0) link with nic0 LAN

  First VM (alpine linux) with one NIC
    NIC0 LAN (on vmbr0)

Truenas Scale
  NIC0 = LAN
 
 
VM Opnsense = Gateway (firewall/NAT)

Everything work fine (even IPV6 with Freebox TV player IPV6) Except one "little" thing, NAT outside proxmox 1 (so, all my lan)



LAN
Trafic performance (done with iperf3)

  proxmox 1 -> truenas = 9Gb/s
  truenas -> proxmox 1   = 9Gb/s
 
  proxmox 2 -> truenas = 2.5Gb/s
  truenas -> proxmox 2 = 2.5Gb/s
 
  vm(proxmox 1) -> truenas = 9Gb/s
  truenas -> vm (proxmox 1) = 9Gb/s
 
  vm(proxmox 2) -> truenas = 2.5Gb/s
  truenas -> vm (proxmox 2) = 2.5Gb/s 



INTERNET (NAT Thrue opnsense)
Trafic performance (done with speedtest)

  Proxmox 1 -> Internet = 1.5 Gb/s : OK for me
  Internet -> Proxmox 1 = 700MB/s : OK for me

  VM Opensense -> Internet = 1.5 Gb/s : OK for me
  Internet -> VM Opensense = 700MB/s : OK for me

  vm(proxmox 1) -> Internet = 1.5 Gb/s : OK for me
  Internet -> vm(proxmox 1) = 700MB/s : OK for me

Anything else not directly INSIDE proxmox 1 (where opnsense is hosted)

=============== BBR
      Server: LaFibre.info BBR-IPv4 - Palaiseau (id: 45446)
    Download:  1427.95 Mbps (data used: 957.4 MB)
      Upload:   590.06 Mbps (data used: 1.1 GB)


      Server: BOUYGUES TELECOM BBR - Meudon (id: 47318)
    Download:  1762.61 Mbps (data used: 2.5 GB)
      Upload:   614.28 Mbps (data used: 641.0 MB)


      Server: LaFibre.info BBR - Massy (id: 2231)
    Download:  1513.09 Mbps (data used: 1.4 GB)
      Upload:   618.12 Mbps (data used: 586.8 MB)




=============== CUBIC
      Server: LaFibre.info CUBIC - Orly (id: 45454)
    Download:    34.60 Mbps (data used: 55.2 MB)
      Upload:   575.48 Mbps (data used: 806.2 MB)


      Server: BOUYGUES TELECOM CUBIC - Vélizy (id: 47424)
    Download:    32.60 Mbps (data used: 49.4 MB)
      Upload:   597.55 Mbps (data used: 856.8 MB)


      Server: Hivane NetWork Cubic - Ivry-sur-Seine (id: 49781)
    Download:    30.76 Mbps (data used: 46.6 MB)
      Upload:   120.26 Mbps (data used: 190.6 MB)
    
    
So, any VM, computer or whatever (not directrly inside proxmox 1) downloading internet things thrue opnsense got worst download speed ever.

Speedtest is the best i can do. but when i try do download alpine iso (exemple)

im 700Mb/s from proxmox 1, (or any vm hosted by proxmox one)
im 1Mbs/s from anything not hosted inside proxmox one itself.


If someone have any idea ???

Thx a lot