Messages

did you use the same 10G card on this server which worked ? Like taking the 10G card out from the server and putting in to this server ? I am just wondering if there was something with the firmware of the 10G card ?

[bxe1]

I am in similar kind of problem, my 10G card is QLogic 577xx/578xx 10 Gigabit Ethernet (BCM57840) and I have used the iommu to map it to my virtual machine which runs opnsense, when I boot my opnsense I get following errors (one interface works but the other doesn't).  The problem is not with the card, if I switch it to a different PCI slot the bxe1 starts working and bxe0 gives the same error. The following logs are with bxe0 working and bxe1 with errors. I cleared the NVRAM on the server but nothing helped.

Code Select Expand

bxe0: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380000000000-0x3800007fffff,0x380000800000-0x380000ffffff,0x380002000000-0x38000200ffff irq 11 at device 8.0 on pci0
bxe0: PCI BAR0 [10] memory allocated: 0x380000000000-0x3800007fffff (8388608) -> 0xfffffe008d400000
bxe0: PCI BAR2 [18] memory allocated: 0x380000800000-0x380000ffffff (8388608) -> 0xfffffe008dc00000
bxe0: PCI BAR4 [20] memory allocated: 0x380002000000-0x38000200ffff (65536) -> 0xfffffe0084678000
bxe0: Found 10Gb Fiber media.
bxe0: IFMEDIA flags : 20
bxe0: Using defaults for TSO: 65518/35/2048
bxe0: Ethernet address: d4:5d:64:09:0b:6a
bxe0: MSI-X vectors Requested 2 and Allocated 2
bxe1: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380001000000-0x3800017fffff,0x380001800000-0x380001ffffff,0x380002010000-0x38000201ffff irq 10 at device 9.0 on pci0
bxe1: PCI BAR0 [10] memory allocated: 0x380001000000-0x3800017fffff (8388608) -> 0xfffffe008e800000
bxe1: PCI BAR2 [18] memory allocated: 0x380001800000-0x380001ffffff (8388608) -> 0xfffffe008f000000
bxe1: PCI BAR4 [20] memory allocated: 0x380002010000-0x38000201ffff (65536) -> 0xfffffe0084688000
bxe1: ERROR: Invalid SHMEM validity signature: 0x00010008
bxe1: ERROR: Invalid phy config in NVRAM (PHY1=0x00010008 PHY2=0x00010008)
bxe1: Unknown media!
bxe1: IFMEDIA flags : 20
bxe1: Using defaults for TSO: 65518/35/2048
bxe1: ERROR: FW failed to respond!
bxe1: ERROR: MCP response failure, aborting
bxe1: ERROR: Failed to unload previous driver! time_counter 10 rc -1
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: <System Container> port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: <System Container> port 0xae00-0xae17 on acpi0

Has it got anything to do with memory addresses ? the addresses gotten allocated for port 1(bxe0), then the same memory addresses are tried to be allocated for the port 2 (bxe1) which can't happen, could this be something like that ?

[14e4:16a1]

Since both the ports are on the same card, their ID is same i.e. 14e4:16a1 and 1043:8591, please check following

Code Select Expand

61:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x
61:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x

I don't think you can pass both through in this way, since as you well point out, the pci id are the same. Only one.
Why not add the _whole_ pci device to the VM though? In other words, without vfio.
Is it Proxmox?

How to passthrough without vfio ? it is opnsense and the vm-host is centos/kvm

The server (Server1) on which both ports work, PCI-E slot is

Code Select Expand

Type:  PCIExpressGen4

And the server (Server2) on which one port works the PCI-E slot is

Code Select Expand

Type:  PCIExpressGen3
:o

[14e4:16a1]

Since both the ports are on the same card, their ID is same i.e. 14e4:16a1 and 1043:8591, please check following

Code Select Expand

61:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x
61:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x

My grub has following parameters,

Code Select Expand

rhgb quiet intel_iommu=on iommu=pt pci-stub.ids=1043:8591


Following files exists with the data as shown

Code Select Expand

[root@server2]# cat /etc/modules-load.d/vfio-pci.conf
vfio-pci
[root@server2]# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=14e4:16a1
[root@server2]#



I have the same configuration on two servers, the servers are a little different, the 10G cards are the same, on server2 irqbypass is missing as shown below. I don't remember what did I do with server1 to get it to work.

Code Select Expand

[root@server1 etc]# lsmod | grep kvm
kvm_intel             442368  0
kvm                  1347584  1 kvm_intel
irqbypass              16384  2 vfio_pci_core,kvm    ##   << this isn't shown on server2

[root@server2 etc]# lsmod | grep kvm
kvm_intel             442368  0
kvm                  1347584  1 kvm_intel

Hi again,
Thanks for the reply :)
lspci -nnk shows following for the 10G cards, both have the same PCI_ids.

Code Select Expand

61:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x
61:00.1 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57840 NetXtreme II 10 Gigabit Ethernet [14e4:16a1] (rev 11)
Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
Kernel driver in use: vfio-pci
Kernel modules: bnx2x

Hi,
I have virtualized the opnsense and pass-through QLogic NetXtreme II BCM57840  via iommu.
I can't get both the ports work, one of them works but the other has the problem with the address mapping. I am just wondering if someone else has faced the same problem, the logs are as follow, please guide !
If I insert the card in the other PCI-express slot the bxe0 starts working and bxe1 start having the problem which is mentioned below for bxe0

Code Select Expand

bxe0: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380000000000-0x3800007fffff,0x380000800000-0x380000ffffff,0x380002000000-0x38000200ffff irq 10 at device 9.0 on pci0
bxe0: PCI BAR0 [10] memory allocated: 0x380000000000-0x3800007fffff (8388608) -> 0xfffffe008d400000
bxe0: PCI BAR2 [18] memory allocated: 0x380000800000-0x380000ffffff (8388608) -> 0xfffffe008dc00000
bxe0: PCI BAR4 [20] memory allocated: 0x380002000000-0x38000200ffff (65536) -> 0xfffffe0084678000
bxe0: ERROR: Invalid SHMEM validity signature: 0x00010008
bxe0: ERROR: Invalid phy config in NVRAM (PHY1=0x00010008 PHY2=0x00010008)
bxe0: Unknown media!
bxe0: IFMEDIA flags : 20
bxe0: Using defaults for TSO: 65518/35/2048
bxe0: ERROR: FW failed to respond!
bxe0: ERROR: MCP response failure, aborting
bxe0: ERROR: Failed to unload previous driver! time_counter 10 rc -1
bxe1: <QLogic NetXtreme II BCM57840 4x10GbE (B1) BXE v:1.78.91> mem 0x380001000000-0x3800017fffff,0x380001800000-0x380001ffffff,0x380002010000-0x38000201ffff irq 10 at device 10.0 on pci0
bxe1: PCI BAR0 [10] memory allocated: 0x380001000000-0x3800017fffff (8388608) -> 0xfffffe008e800000
bxe1: PCI BAR2 [18] memory allocated: 0x380001800000-0x380001ffffff (8388608) -> 0xfffffe008f000000
bxe1: PCI BAR4 [20] memory allocated: 0x380002010000-0x38000201ffff (65536) -> 0xfffffe0084688000
bxe1: Found 10Gb Fiber media.
bxe1: IFMEDIA flags : 20
bxe1: Using defaults for TSO: 65518/35/2048
bxe1: Ethernet address: d4:5d:64:09:0b:6a
bxe1: MSI-X vectors Requested 2 and Allocated 2

Any ideas ?

Atleast I now know that the full configuration is at /conf/config.xml

But please someone answer this  https://forum.opnsense.org/index.php?topic=40896.0

Hi,
I want to decrypt the backup file which I got from the google-drive, I want to make changes to it before I restore it (e.g. changing the name of the network interface), does anyone know the command I can use to decrypt the backup of the configuration.xml ?
The following doesn't work for me.  :-[

Code Select Expand

openssl  enc -d -AES-256-CBC -md SHA512 -pbkdf2 -iter=100000 -in config-encrypted -pass pass:XXXXXXX

Thanks in advance

Regards

Hi,
I want to decrypt the backup file which I got from the google-drive, I want to make changes to it, does anyone know the command I can use to decrypt the backup of the configuration.xml ?
And secondly, if I have a running opnsense, where can I find the full configuration along with the user database ? full configuration which I can copy and restore to a new instance of opnsense.

Thanks in advance :)

Best Regards

Hi,
Is it possible that I can import the opnsense's user database along with the passwords ?
I have done that in the past using the ldapsearch with another system which was running LDAP.
I want to take the usernames and the passwords from the opnsense and import them to another linux's user database (/etc/passwd & /etc/shadow).

Please advice.

Regards

Hi all,
I currently have my opnsense installed & configured on a physical machine as a router, I am thinking to move it to a KVM-based virtual machine, has anyone already tried it ?
I am thinking to dedicate two physical interfaces of the physical machine to the virtual machine, install the opnsense to that virtual machine and then restore the backup from the opnsense which is my current router on a physical machine to this virtual machine.
Has anyone tried this already ? are there some limitations ?

Regards

I had opnsense which has the network interfaces as follows.
bxe0, bxe1, igd0 and igd1 

I am trying to restore on a new machine with a backup which was taken on the old one. The interfaces on the new machine are like eno0, eno1 etc


I want to move them to names like eno, eth1 and so on. I know that I can do it in the /etc/rc.conf

ifconfig_bxe0_name=eth0
but during the boot time the network interfaces are still bxe0, bxe1 etc. Is there any way I can change interface names I want before the kernel gets loaded ?

Thanks in advance