Messages

Hello,

Did you apply all VATES recommandation on configuration your VIF ? ( disabling rx checksumming for example ?)

There is a complete documentation here: https://docs.xcp-ng.org/guides/pfsense/

Do you have more details about your setup maybe ?

Best regards

Hello,

Thanks @patrick

I didn't know it was introduced by CARP.

It's okay for me :)

Best regards.

Hello,

no, it's not enabled, i don't use it.

Hello,

I have promiscious mode enabled on ALL interfaces except the one on which i make all my ha states pass thought.

However, i didn't enable at all promiscuous mode on any interface, i don't know why the mode is present.

I tried to disable if using `ifconfig XXXX -promisc` but it does nothing.

I run opnsense in a virtual machine inside a Xen cluster.

```
xn0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9004
   description: WAN (opt8)
   options=0
   ether xx:xxx:xx:xxx
        XXXX
   carp: MASTER vhid 1 advbase 1 advskew 0
         peer 224.0.0.18 peer6 ff02::12
   media: Ethernet manual
   status: active
   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
```

Do you have an idea why ?

I have opnsense latest version

I don't have any ids or something like this, i just have unbound dns configured.


In my dmesg i have
```
root@xxxx:~ # dmesg |grep "promiscuous"
pflog0: permanently promiscuous mode enabled
xn0: promiscuous mode enabled
vlan0.1: promiscuous mode enabled
vlan0.2: promiscuous mode enabled
vlan0.5: promiscuous mode enabled
vlan0.4: promiscuous mode enabled
vlan0.5: promiscuous mode enabled
....
vlan0.1: promiscuous mode disabled
vlan0.2: promiscuous mode disabled
vlan0.3: promiscuous mode disabled
vlan0.4: promiscuous mode disabled
vlan0.5: promiscuous mode disabled
...
xn0: promiscuous mode disabled
xn0: promiscuous mode enabled
vlan0.1: promiscuous mode enabled
vlan0.2: promiscuous mode enabled
vlan0.3: promiscuous mode enabled
vlan0.4: promiscuous mode enabled
vlan0.5: promiscuous mode enabled

vlan0.10: promiscuous mode enabled
vlan0.10: promiscuous mode disabled
vlan0.10: promiscuous mode enabled
vlan0.10: promiscuous mode disabled
vlan0.1: promiscuous mode disabled
vlan0.1: promiscuous mode enabled
```

VLAN 10 ( in my obsfuscated output ), correspond to the vlan i made all my ha states pass thought.

Do you have an idea ?

Thanks !

Hello,

i have issue with the opnsense 25.1



Here is my setup:
OPNsense 25.1.6_4-amd64
FreeBSD 14.2-RELEASE-p3
OpenSSL 3.0.16
AMD EPYC 4464P 12-Core Processor (4 cores, 4 threads)
RAM: 8GB DDR4
Disk: 50GB ZFS.


My docker container got random timeout to other networks.

in tcpdump, i see incoming packet ( on the server interface ) with length higher than 1500 ( 1506 ) i don't understand why.

My router cannot send ping higher than my MTU, normal

Code Select Expand

root@gco-001-router-001:~ # ping -s 1472 -D PUBLIC_IP
PING PUBLIC_IP (PUBLIC_IP): 1472 data bytes
1480 bytes from PUBLIC_IP: icmp_seq=0 ttl=64 time=2.179 ms
1480 bytes from PUBLIC_IP: icmp_seq=1 ttl=64 time=2.302 ms
1480 bytes from PUBLIC_IP: icmp_seq=2 ttl=64 time=2.151 ms
1480 bytes from PUBLIC_IP: icmp_seq=3 ttl=64 time=2.152 ms

root@gco-001-router-001:~ # ping -s 1500 -D PUBLIC_IP
PING PUBLIC_IP(PUBLIC_IP: 1500 data bytes
ping: sendto: Message too long
ping: sendto: Message too long

However, from a computer inside the LAN.

Code Select Expand

[root@local-vm ~]# ping -s 3000 -D PUBLIC_IP
PING PUBLIC_IP (PUBLIC_IP) 3000(3028) bytes of data.
[1747332539.430593] 3008 bytes from PUBLIC_IP: icmp_seq=1 ttl=63 time=3.56 ms
[1747332540.431041] 3008 bytes from PUBLIC_IP: icmp_seq=2 ttl=63 time=2.34 ms


I also post it here https://github.com/opnsense/src/issues/235#issuecomment-2885157470

I have some difficulty to understand why i got randomly this issue.

Best regards

Hello,

Thanks !

This button is definitly too discret :D

For peoples who search it, here is a screenshot



Best regards,

Hello,

It would be great to be able to reset / remove all custom tunnable settings defines.

At this time, i use the "value" vs "default" to determine what to remove, but could be great to be able to reset it easily.

Maybe there is a way i don't know ?

Best regards

1.1.1.1 It's an example .... i will not show your my real networks / ips and the issue gone when i select my lan interfaces

Here is a network diagram of my infrastructure

Resume of the situation.
1.1.1.1 -> 1.1.1.2 = OK
INTERNET -> 1.1.1.2 = OK
192.168.1.2 -> 1.1.1.2 = Webui of the interface, other port are not working.

I have a no-nat from 192.168.0.0/16 to 1.0.0.0/24 to preserve client ip on our "public server" which can enter in the network thought the opnsense which serve of gateway.

And a global nat for the rest of internet by the ip 1.1.1.253

But IF on my port-forward rule, i enable also the local interface, it works !

In the screen of my opnsense, i show that if i check openvpn ( for example, but it could have been whatever interface ) it works.

PS: i move the admin port and cchange the listen interface to admin vlan only, it's doesn't change anything except now, i haven't anything answering.

Hello,

I enabled it, i don't see difference, making a curl on my wan ip from inside show me the opnsense webui instead of making the redirection.

But if on the port forward rule, i choose my lan interface, then it works. but why ?

Hello,

I have a opnsense 25.1.

I have one vlan per customer. ( and multiple customers of course ).
I have a WAN range ips: 1.0.0.0/24

I want to make a port-foward from 1.0.0.1 to one customer vlan ip.

So i configured:
- Interface: WAN
- Source: any
- Destination: 1.0.0.1
- Translate to: 192.168.2.4

I also create the firewall rule to permit the trafic.

That point is OK !

I have a no-nat rules from all my local subnets to 1.0.0.0/24 ( to preserve client ip on the wan devices ).

However, if the trafic come from one local ip to the port-forward ip, it's the firewall which handle the connection and it's not redirected to the local ip.

External connections are ok !

In opnsense, i need to edit the port-forward and select all interfaces one / one.

The problem, is if i add a new network, i won't add on all my port-foward the new interface.

How simplify this ?

Can you add the possibility to listen on "Any interface" instead of selecting interfaces one / one ?

Best regards,

Hello,

At this time, i don't "really need this bandwith".

But it will be when my new backup server will arrive with good disks.

iperf in the same interface or accross vlan doesn't change anything in term of bandwith ( i tested it ).

the Opnsense VM have 2 nics. ( which is phyiscally, the same 25Gb/s interface ).

one dedicated for the "upstream", the other for the internal with vlan.

When there is no trafic, my idle is never at 100%, but around 65-85%

Code Select Expand

6 processes:   1 running, 5 sleeping
CPU:  0.0% user,  0.0% nice,  0.0% system, 22.5% interrupt, 77.5% idle
Mem: 67M Active, 153M Inact, 113M Laundry, 1332M Wired, 56K Buf, 277M Free
ARC: 937M Total, 168M MFU, 602M MRU, 1926K Anon, 19M Header, 144M Other
     662M Compressed, 1674M Uncompressed, 2.53:1 Ratio
Swap: 8192M Total, 8540K Used, 8184M Free


When i run an iperf from one ip of my vlan, to the ip of the opnsense ( so, no interval-routing )

Code Select Expand

6 processes:   1 running, 5 sleeping
CPU:  0.8% user,  0.0% nice,  0.0% system, 99.2% interrupt,  0.0% idle
Mem: 67M Active, 153M Inact, 113M Laundry, 1336M Wired, 56K Buf, 272M Free
ARC: 936M Total, 167M MFU, 604M MRU, 190K Anon, 19M Header, 144M Other
     662M Compressed, 1675M Uncompressed, 2.53:1 Ratio

( at this point, the bandwith of the iperf is at 2.3Gb/s ).

Best regards,

Hello,

My hypervisor has 2 nics:
- 1x 1Gb/s
- 1x 25Gb/s connected to a switch ( which lead to my upstream ).


I use OPNsense 24.7.12_2-amd64 in a XCP-NG virtual machine which has 4c / 8Gb ram / 50Gb nvme.

The CPU of the server is AMD EPYC 4464P (3,7 GHz )

i'm in Hardware virtualization with paravirtualization drivers enabled (PVHVM) with Realtek 8139
i attached 2x the 25gb/s link ( 1 for wan, 1 for the local VLAN ).

I can't get better than 3Gb/s

Code Select Expand

[18:39 server-1 ~]# iperf3 -c 10.255.0.254
Connecting to host 10.255.0.254, port 5201
[  4] local 10.255.1.3 port 38188 connected to 10.255.0.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   351 MBytes  2.95 Gbits/sec  362    536 KBytes       
[  4]   1.00-2.00   sec   331 MBytes  2.78 Gbits/sec    4    582 KBytes       
[  4]   2.00-3.00   sec   354 MBytes  2.97 Gbits/sec   21    465 KBytes       
[  4]   3.00-4.00   sec   341 MBytes  2.86 Gbits/sec    5    556 KBytes       
[  4]   4.00-5.00   sec   310 MBytes  2.60 Gbits/sec   15    441 KBytes       
[  4]   5.00-6.00   sec   354 MBytes  2.97 Gbits/sec   64    644 KBytes       
[  4]   6.00-7.00   sec   345 MBytes  2.89 Gbits/sec    6    530 KBytes       
[  4]   7.00-8.00   sec   312 MBytes  2.62 Gbits/sec    7    671 KBytes       
[  4]   8.00-9.00   sec   348 MBytes  2.92 Gbits/sec   14    581 KBytes       
[  4]   9.00-10.00  sec   326 MBytes  2.74 Gbits/sec   50    571 KBytes       

When i run the iperf, the cpu is full !


When i disable the firewall in settings, the iperf is 2x more performant.

Code Select Expand

[18:45 server-1 ~]# iperf3 -c 10.255.0.254 -t 10000
Connecting to host 10.255.0.254, port 5201
[  4] local 10.255.1.3 port 38318 connected to 10.255.0.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   617 MBytes  5.17 Gbits/sec  464    742 KBytes       
[  4]   1.00-2.00   sec   540 MBytes  4.53 Gbits/sec   57    671 KBytes       
[  4]   2.00-3.00   sec   639 MBytes  5.36 Gbits/sec  118    522 KBytes       
[  4]   3.00-4.00   sec   646 MBytes  5.42 Gbits/sec   58    636 KBytes       
[  4]   4.00-5.00   sec   665 MBytes  5.58 Gbits/sec   35    599 KBytes       
[  4]   5.00-6.00   sec   699 MBytes  5.86 Gbits/sec  150    698 KBytes       
[  4]   6.00-7.00   sec   616 MBytes  5.16 Gbits/sec  128    702 KBytes       
[  4]   7.00-8.00   sec   692 MBytes  5.82 Gbits/sec  156    735 KBytes       
^C[  4]   8.00-8.53   sec   380 MBytes  6.03 Gbits/sec   50    509 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-8.53   sec  5.37 GBytes  5.40 Gbits/sec  1216             sender
[  4]   0.00-8.53   sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: interrupt - the client has terminated


I precise that i also tried to remove ALL MY RULES, i don't see any difference in term of cpu usage.

Why do i use all theses CPU ?

And the final question, yesterday, after a ton of test ( which i don't note somewhere of course ) i figured to have my 17GB/s !!! , but after a reboot, my bandwith come back to 3GB/s...

Code Select Expand

[19:52 server-1 ~]# iperf3 -c 10.255.1.254
Connecting to host 10.255.1.254, port 5201
[  4] local 10.255.1.3 port 59300 connected to 10.255.1.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.80 GBytes  15.4 Gbits/sec  265    682 KBytes       
[  4]   1.00-2.00   sec  1.79 GBytes  15.3 Gbits/sec  245    743 KBytes       
[  4]   2.00-3.00   sec  1.88 GBytes  16.2 Gbits/sec  216   1014 KBytes       
[  4]   3.00-4.00   sec  1.91 GBytes  16.4 Gbits/sec  138   1.60 MBytes       
[  4]   4.00-5.00   sec  1.85 GBytes  15.9 Gbits/sec  153   1.98 MBytes       
[  4]   5.00-6.00   sec  2.00 GBytes  17.2 Gbits/sec  262    638 KBytes       
[  4]   6.00-7.00   sec  1.92 GBytes  16.5 Gbits/sec  351    944 KBytes       
[  4]   7.00-8.00   sec  1.78 GBytes  15.3 Gbits/sec  241   2.06 MBytes       
[  4]   8.00-9.00   sec  1.97 GBytes  16.9 Gbits/sec  240    655 KBytes       
[  4]   9.00-10.00  sec  1.58 GBytes  13.6 Gbits/sec  210   2.01 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  18.5 GBytes  15.9 Gbits/sec  2321             sender
[  4]   0.00-10.00  sec  18.5 GBytes  15.9 Gbits/sec                  receiver

When i run no trafic, my CPU usage is at 13%-20%.

Thanks !

Hello,

Indeed, my virtual machine is currently in Hardware virtualization with paravirtualization drivers enabled (PVHVM).

Crap, i'm wrong i'm using RTL8139. sorry

Hello,

My hypervisor has 2 nics:
- 1x 1Gb/s
- 1x 25Gb/s connected to a switch ( which lead to my upstream ).


I use OPNsense 24.7.12_2-amd64 in a XCP-NG virtual machine which has 4c / 8Gb ram / 50Gb nvme with the guest tools installed.

I use virtio driver.
i attached 2x the 25gb/s link ( 1 for wan, 1 for the local VLAN ).


The interface link speed seems to be "manually configured", but to which value ?

Code Select Expand

xn1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
description: LAN (opt9)
options=503<RXCSUM,TXCSUM,TSO4,LRO>
ether 7a:10:c4:65:82:ca
inet6 fe80::7810:c4ff:fe65:82ca%xn1 prefixlen 64 scopeid 0x6
media: Ethernet manual
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Of course, i did'nt configure anything.

I want to be sure the link speed is to 25Gb/s.

I run a iperf on 127.0.0.1, it's seems to be the case, but i want to be sure.



Thanks !