Messages

1

Hardware and Performance / Re: 10Gbe massive performance issues *SOLVED*

« on: August 19, 2024, 06:00:01 pm »

did you use the same 10G card on this server which worked ? Like taking the 10G card out from the server and putting in to this server ? I am just wondering if there was something with the firmware of the 10G card ?

2

23.7 Legacy Series / Re: Qlogic 57810 10Gbit Ethernet

« on: August 16, 2024, 12:39:33 pm »

I am in similar kind of problem, my 10G card is QLogic 577xx/578xx 10 Gigabit Ethernet (BCM57840) and I have used the iommu to map it to my virtual machine which runs opnsense, when I boot my opnsense I get following errors (one interface works but the other doesn't).  The problem is not with the card, if I switch it to a different PCI slot the bxe1 starts working and bxe0 gives the same error. The following logs are with bxe0 working and bxe1 with errors. I cleared the NVRAM on the server but nothing helped.

Code: [Select]

bxe0: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380000000000-0x3800007fffff,0x380000800000-0x380000ffffff,0x380002000000-0x38000200ffff irq 11 at device 8.0 on pci0
bxe0: PCI BAR0 [10] memory allocated: 0x380000000000-0x3800007fffff (8388608) -> 0xfffffe008d400000
bxe0: PCI BAR2 [18] memory allocated: 0x380000800000-0x380000ffffff (8388608) -> 0xfffffe008dc00000
bxe0: PCI BAR4 [20] memory allocated: 0x380002000000-0x38000200ffff (65536) -> 0xfffffe0084678000
bxe0: Found 10Gb Fiber media.
bxe0: IFMEDIA flags : 20
bxe0: Using defaults for TSO: 65518/35/2048
bxe0: Ethernet address: d4:5d:64:09:0b:6a
bxe0: MSI-X vectors Requested 2 and Allocated 2
bxe1: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380001000000-0x3800017fffff,0x380001800000-0x380001ffffff,0x380002010000-0x38000201ffff irq 10 at device 9.0 on pci0
bxe1: PCI BAR0 [10] memory allocated: 0x380001000000-0x3800017fffff (8388608) -> 0xfffffe008e800000
bxe1: PCI BAR2 [18] memory allocated: 0x380001800000-0x380001ffffff (8388608) -> 0xfffffe008f000000
bxe1: PCI BAR4 [20] memory allocated: 0x380002010000-0x38000201ffff (65536) -> 0xfffffe0084688000
bxe1: ERROR: Invalid SHMEM validity signature: 0x00010008
bxe1: ERROR: Invalid phy config in NVRAM (PHY1=0x00010008 PHY2=0x00010008)
bxe1: Unknown media!
bxe1: IFMEDIA flags : 20
bxe1: Using defaults for TSO: 65518/35/2048
bxe1: ERROR: FW failed to respond!
bxe1: ERROR: MCP response failure, aborting
bxe1: ERROR: Failed to unload previous driver! time_counter 10 rc -1
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: <System Container> port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: <System Container> port 0xae00-0xae17 on acpi0
Has it got anything to do with memory addresses ? the addresses gotten allocated for port 1(bxe0), then the same memory addresses are tried to be allocated for the port 2 (bxe1) which can't happen, could this be something like that ?

3

General Discussion / Re: iommu & QLogic NetXtreme II BCM57840

« on: August 14, 2024, 12:49:04 pm »

Since both the ports are on the same card, their ID is same i.e. 14e4:16a1 and 1043:8591, please check following

Code: [Select]

61:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x
61:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x

I don't think you can pass both through in this way, since as you well point out, the pci id are the same. Only one.
Why not add the _whole_ pci device to the VM though? In other words, without vfio.
 Is it Proxmox?

How to passthrough without vfio ? it is opnsense and the vm-host is centos/kvm

4

General Discussion / Re: iommu & QLogic NetXtreme II BCM57840

« on: August 14, 2024, 10:35:22 am »

The server (Server1) on which both ports work, PCI-E slot is

Code: [Select]

Type:  PCIExpressGen4
And the server (Server2) on which one port works the PCI-E slot is

Code: [Select]

Type:  PCIExpressGen3

5

General Discussion / Re: iommu & QLogic NetXtreme II BCM57840

« on: August 14, 2024, 12:27:43 am »

Since both the ports are on the same card, their ID is same i.e. 14e4:16a1 and 1043:8591, please check following

Code: [Select]

61:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x
61:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x
My grub has following parameters,

Code: [Select]

rhgb quiet intel_iommu=on iommu=pt pci-stub.ids=1043:8591

Following files exists with the data as shown

Code: [Select]

[root@server2]# cat /etc/modules-load.d/vfio-pci.conf
vfio-pci
[root@server2]# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=14e4:16a1
[root@server2]#


I have the same configuration on two servers, the servers are a little different, the 10G cards are the same, on server2 irqbypass is missing as shown below. I don't remember what did I do with server1 to get it to work.

Code: [Select]

[root@server1 etc]# lsmod | grep kvm
kvm_intel             442368  0
kvm                  1347584  1 kvm_intel
irqbypass              16384  2 vfio_pci_core,kvm    ##   << this isn't shown on server2

[root@server2 etc]# lsmod | grep kvm
kvm_intel             442368  0
kvm                  1347584  1 kvm_intel

6

General Discussion / Re: iommu & QLogic NetXtreme II BCM57840

« on: August 13, 2024, 10:13:27 am »

Hi again,
 Thanks for the reply
lspci -nnk shows following for the 10G cards, both have the same PCI_ids.

Code: [Select]

61:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x
61:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x

7

General Discussion / iommu & QLogic NetXtreme II BCM57840

« on: August 12, 2024, 04:19:44 pm »

Hi,
 I have virtualized the opnsense and pass-through QLogic NetXtreme II BCM57840  via iommu.
I can't get both the ports work, one of them works but the other has the problem with the address mapping. I am just wondering if someone else has faced the same problem, the logs are as follow, please guide !
If I insert the card in the other PCI-express slot the bxe0 starts working and bxe1 start having the problem which is mentioned below for bxe0

Code: [Select]

bxe0: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380000000000-0x3800007fffff,0x380000800000-0x380000ffffff,0x380002000000-0x38000200ffff irq 10 at device 9.0 on pci0
bxe0: PCI BAR0 [10] memory allocated: 0x380000000000-0x3800007fffff (8388608) -> 0xfffffe008d400000
bxe0: PCI BAR2 [18] memory allocated: 0x380000800000-0x380000ffffff (8388608) -> 0xfffffe008dc00000
bxe0: PCI BAR4 [20] memory allocated: 0x380002000000-0x38000200ffff (65536) -> 0xfffffe0084678000
bxe0: ERROR: Invalid SHMEM validity signature: 0x00010008
bxe0: ERROR: Invalid phy config in NVRAM (PHY1=0x00010008 PHY2=0x00010008)
bxe0: Unknown media!
bxe0: IFMEDIA flags : 20
bxe0: Using defaults for TSO: 65518/35/2048
bxe0: ERROR: FW failed to respond!
bxe0: ERROR: MCP response failure, aborting
bxe0: ERROR: Failed to unload previous driver! time_counter 10 rc -1
bxe1: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380001000000-0x3800017fffff,0x380001800000-0x380001ffffff,0x380002010000-0x38000201ffff irq 10 at device 10.0 on pci0
bxe1: PCI BAR0 [10] memory allocated: 0x380001000000-0x3800017fffff (8388608) -> 0xfffffe008e800000
bxe1: PCI BAR2 [18] memory allocated: 0x380001800000-0x380001ffffff (8388608) -> 0xfffffe008f000000
bxe1: PCI BAR4 [20] memory allocated: 0x380002010000-0x38000201ffff (65536) -> 0xfffffe0084688000
bxe1: Found 10Gb Fiber media.
bxe1: IFMEDIA flags : 20
bxe1: Using defaults for TSO: 65518/35/2048
bxe1: Ethernet address: d4:5d:64:09:0b:6a
bxe1: MSI-X vectors Requested 2 and Allocated 2
Any ideas ?

8

General Discussion / Re: Backup from google drive

« on: June 05, 2024, 03:55:49 pm »

Atleast I now know that the full configuration is at /conf/config.xml

But please someone answer this  https://forum.opnsense.org/index.php?topic=40896.0

9

General Discussion / Decrypting the backup from Google-Drive

« on: June 05, 2024, 03:54:29 pm »

Hi,
 I want to decrypt the backup file which I got from the google-drive, I want to make changes to it before I restore it (e.g. changing the name of the network interface), does anyone know the command I can use to decrypt the backup of the configuration.xml ?
The following doesn't work for me. 

Code: [Select]

openssl  enc -d -AES-256-CBC -md SHA512 -pbkdf2 -iter=100000 -in config-encrypted -pass pass:XXXXXXX
Thanks in advance

Regards

10

General Discussion / Backup from google drive

« on: May 31, 2024, 01:10:55 am »

Hi,
 I want to decrypt the backup file which I got from the google-drive, I want to make changes to it, does anyone know the command I can use to decrypt the backup of the configuration.xml ?
And secondly, if I have a running opnsense, where can I find the full configuration along with the user database ? full configuration which I can copy and restore to a new instance of opnsense.

Thanks in advance

Best Regards

11

Virtual private networks / User database

« on: January 08, 2024, 09:12:18 pm »

Hi,
 Is it possible that I can import the opnsense's user database along with the passwords ?
I have done that in the past using the ldapsearch with another system which was running LDAP.
I want to take the usernames and the passwords from the opnsense and import them to another linux's user database (/etc/passwd & /etc/shadow).

Please advice.

Regards

12

General Discussion / opnsense router on kvm-based Virtual machine

« on: March 27, 2023, 05:43:05 am »

Hi all,
 I currently have my opnsense installed & configured on a physical machine as a router, I am thinking to move it to a KVM-based virtual machine, has anyone already tried it ?
I am thinking to dedicate two physical interfaces of the physical machine to the virtual machine, install the opnsense to that virtual machine and then restore the backup from the opnsense which is my current router on a physical machine to this virtual machine.
Has anyone tried this already ? are there some limitations ?

Regards

13

General Discussion / restoring from a backup

« on: January 22, 2023, 09:59:10 pm »

I had opnsense which has the network interfaces as follows.
bxe0, bxe1, igd0 and igd1 

 I am trying to restore on a new machine with a backup which was taken on the old one. The interfaces on the new machine are like eno0, eno1 etc


I want to move them to names like eno, eth1 and so on. I know that I can do it in the /etc/rc.conf

ifconfig_bxe0_name=eth0
but during the boot time the network interfaces are still bxe0, bxe1 etc. Is there any way I can change interface names I want before the kernel gets loaded ?

Thanks in advance