PCI Passthrough Intel X710-X2 Opnsense

[wondaah]

Hi everyone

I'm currently facing a really weird issue that I cannot find any info of in the internet  :-\

So I've just set up PCI passthrough, and added both of them to the OPNsense VM. So far so good. Everything there seems to work. IOMMU is working fine.

The WAN Interface also gets an IP so traffic is flowing there.

The LAN Interface is where my problem is; it has IP 192.168.1.1, its connected to a Mikrotik switch that has no VLANs and traffic over it is running fine.

But I cannot access the OPNsense Webinterface nor ping it.

The WAN Interface can ping for example 8.8.8.8 perfectly.

Now the errors I'm getting:

ixl1 (LAN Interface) i40e_aq_add_macvlan status I40E_ERR_ADMIN_QUEUE_ERROR, error I40E_AQ_RC_BAD_ADDR

I cannot seem to find any info about this anywhere, does someone of you know how I can fix this?

Edit: now both interfaces seem to get these error messages and I've lost the IP of my ISP...
Do I need drivers?

Edit 2: I think OPNsense 23.7 doesn't have ixl support for X710?

[bartjsmit]

So I've just set up PCI passthrough, and added both of them to the OPNsense VM. 

What is your hypervisor?

Do I need drivers?

No, but check the HCL https://www.freebsd.org/releases/13.2R/hardware/

Install OPNsense on bare metal first and confirm the X710 works in that setup.

Bart...

[0zzy]

Sounds you're using Proxmox, if yes:
add the NICs via PCI Passthrough like I done it (works 100% depending on your card you have to assign another RAW Device ID).
Take a look on the Screenshots.

For me it works very well ;)
Also at first start OPNSense is using the IP 192.168.1.1, if your modem is configured to use DHCP (what I not prefer), disable it and give it static IP Address.

On my side the modem have the ip 192.168.1.1, OPNSense is configured at first as I described at the top, from Proxmox Web Interface in the OPNSense Shell, I only add the Interfaces, and set the IP on igb0 to 192.168.11.252/24.

Change this to your needs.

Also it's not possible out of the box to ping the lan and wan interface on OPNSense, you need to add a Firewall Rule on both interfaces (not a good idea on wan).

[wondaah]

Yes I'm using Proxmox.

But it still seems the driver does not load correctly. Still getting the same error both in pfsense and opnsense. Will try to do a bare metal install now...

[wondaah]

Bare Metal install also throws the same errors, so its definitely not related to proxmox or the PCI passthrough

[Patrick M. Hausen]

The freebsd-net@freebsd.org mailing list might be the best place to go now.

[0zzy]

which kind of Nic Card do you have?
I need the exact Device Identifier (eventually a link to the site where you bought it)

[Patrick M. Hausen]

It's in the thread subject: X710-X2.

[0zzy]

Patrick this is why i meant the Device Identifier because there are many of them available (different versions, identifiers and also product names of these card.

There's not only one card out there from Intel ;)
especially this one product name "X710-X2" i couldn't find on the intel site.

Many X710's but not this one...

What i know is that for the most cards intel have drivers to compile by yourself.

So eventuelly this is the trick?

I had this on an older card, the driver wasn't on linux installed, compiled it by myself and then after a reboot everything working fine.
Sometime, if the card is too new there are the same problems.

[wondaah]

Hi the Card is an CNA X710-DA2 Manufactured 07/2018.

Yes it seems like I may have to compile the drivers myself.

Also it may be some issue with PCI Passthrough but I don't think thats the case since as stated it didnt work on Bare-Metal either.

Would you be able to assist me in Compiling them?

I also upgraded de Firmware of the Card to the newest version, didnt help.

[Patrick M. Hausen]

Again - I'd contact the freebsd-net mailing list about that issue.

[wondaah]

I just did, am I correct on just sending an E-Mail to that address?

This issue is driving me nuts lol

[Patrick M. Hausen]

You need to subscribe first to be permitted to send.

https://lists.freebsd.org/subscription/freebsd-net