OPNsense does not boot with more than 7 nics

[vupibi]

Hey there!

System: OPNsense 17.1.4 installed from CD on XENserver 7 with xenTools.

Booting with 7 or less nics is working properly, adding additional nics results in the following error and the system will stop booting:

Code: [Select]

xn7: failed to allocate tx grant refs
run_inerrupt_driven_hooks: still waiting (...)


It worked with OPNsense 16.x. Any Ideas?

Thanks for your support!

[bartjsmit]

Can you give some details on your use case? Do you require eight separate security zones, or are you merely finding the limits of the OS?

Bart...

[vupibi]

Hey,

we have four different internet connections, a DMZ and different other LANs for different jobs like printing, VOIP and so on. The security zones are important to us and its a real production network

At the moment we have on our XENserver 7 a VM with OPNSense 16.x which has 14 nics attached and it runs like a charm. I just wanted to upgrade it to OPNsense 17.x but it does not even boot from CD. If if I remove the nics, it works.

Could it be a limit of the new FreeBSD 11.0?

Thanks for your help!

[bartjsmit]

Actually, the problem seems to be with Xen. I was able to boot up a OPNsense 17.1 VM with eight vNIC's quite happily on ESXi 6.0U2.

Bart...

[vupibi]

Okay, thats a good information, thanks!

Does anyone knows what the error "failed to allocate tx grant refs" is telling me? I could not find anything about it.

Tanks for your help!

[vupibi]

Okay, after some testing I noticed that even a clean FreeBSD 11 does not boot on XEN with more than 7 nics attached. It will stuck at the same error message.

Seems really to be a problem with XEN.

Tanks for your support!

[Nnyan]

Found some references (but not sure how helpful they will be to you) that mention that error in one way or another:

https://xenbits.xen.org/docs/4.5-testing/misc/grant-tables.txt

https://github.com/freebsd/freebsd/blob/master/sys/dev/xen/netfront/netfront.c

http://old-list-archives.xenproject.org/xen-users/2005-11/msg00161.html

[aos1]

I believe we're getting this same problem under KVM/qemu.  That would make it not a XEN issue.

17.1 boots, but we never get the console.  Updating to the latest makes the firewall rules and GUI never actually come up.

the 16.x series also worked fine.

We have 9 interfaces.