High Ping with passthrough on ESXI 7.0u1

[xboarder56]

Hi all,

I just install OPNsense 20.7.7 onto a virtual machine using a intel x710-T4 passthrough to the opnsense VM. However, I'm running into a strange latency issue with packets dropping and taking insane amounts of time to respond back.

I have tested this on a fresh default installation with the 192.168.1.1 interface IP address and I can't determine what exactly is going on. With the intel igb0/1 interfaces no such ping issue exists. Live booting a ubuntu VM and giving it a static IP it pings as expected on the IXL0-4 interfaces as expected.

Testing the same ESXi Host with a live opnsense CD and assigning the interface to the same one (IXL0) works as expected (highest ping is 3ms), indicating this is something happening under OPNsense when the machine is being virtualized.

Below is a list of the ping results:

Reply from 192.168.1.1: bytes=32 time=204ms TTL=64
Reply from 192.168.1.1: bytes=32 time=584ms TTL=64
Reply from 192.168.1.1: bytes=32 time=498ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=587ms TTL=64
Reply from 192.168.1.1: bytes=32 time=968ms TTL=64
Reply from 192.168.1.1: bytes=32 time=37ms TTL=64
Reply from 192.168.1.1: bytes=32 time=18ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=857ms TTL=64
Reply from 192.168.1.1: bytes=32 time=78ms TTL=64
Reply from 192.168.1.1: bytes=32 time=7ms TTL=64
Reply from 192.168.1.1: bytes=32 time=62ms TTL=64
Reply from 192.168.1.1: bytes=32 time=61ms TTL=64
Reply from 192.168.1.1: bytes=32 time=16ms TTL=64
Reply from 192.168.1.1: bytes=32 time=128ms TTL=64
Reply from 192.168.1.1: bytes=32 time=6ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=155ms TTL=64
Reply from 192.168.1.1: bytes=32 time=155ms TTL=64
Reply from 192.168.1.1: bytes=32 time=144ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=291ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=11ms TTL=64
Reply from 192.168.1.1: bytes=32 time=97ms TTL=64
Reply from 192.168.1.1: bytes=32 time=187ms TTL=64
Reply from 192.168.1.1: bytes=32 time=212ms TTL=64
Reply from 192.168.1.1: bytes=32 time=56ms TTL=64
Reply from 192.168.1.1: bytes=32 time=606ms TTL=64
Reply from 192.168.1.1: bytes=32 time=587ms TTL=64
Reply from 192.168.1.1: bytes=32 time=263ms TTL=64
Reply from 192.168.1.1: bytes=32 time=237ms TTL=64
Reply from 192.168.1.1: bytes=32 time=202ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=215ms TTL=64
Reply from 192.168.1.1: bytes=32 time=849ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=25ms TTL=64
Reply from 192.168.1.1: bytes=32 time=115ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=248ms TTL=64
Reply from 192.168.1.1: bytes=32 time=463ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=19ms TTL=64
Reply from 192.168.1.1: bytes=32 time=31ms TTL=64
Reply from 192.168.1.1: bytes=32 time=77ms TTL=64
Reply from 192.168.1.1: bytes=32 time=326ms TTL=64
Reply from 192.168.1.1: bytes=32 time=616ms TTL=64
Reply from 192.168.1.1: bytes=32 time=249ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1590ms TTL=64
Reply from 192.168.1.1: bytes=32 time=307ms TTL=64
Reply from 192.168.1.1: bytes=32 time=261ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=464ms TTL=64
Reply from 192.168.1.1: bytes=32 time=158ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=250ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=294ms TTL=64
Reply from 192.168.1.1: bytes=32 time=4ms TTL=64
Reply from 192.168.1.1: bytes=32 time=124ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=231ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=248ms TTL=64
Reply from 192.168.1.1: bytes=32 time=32ms TTL=64
Reply from 192.168.1.1: bytes=32 time=276ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=135ms TTL=64
Reply from 192.168.1.1: bytes=32 time=93ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=309ms TTL=64
Reply from 192.168.1.1: bytes=32 time=67ms TTL=64
Reply from 192.168.1.1: bytes=32 time<1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=218ms TTL=64

[xboarder56]

Interesting discovery, this issue is not present on OPNsense 20.1. Could this have been caused by the recent net map changes or the 12.1 HardenBSD upgrade?

Booting a HardenBSD 13 build from the same VM config with the X710 passed through it pings fine. This is looking like something specifically done in 20.7 that I cannot pin down.