I'm having the same problem... except console stops responding most of the time too. Are you guys using a bridge in your config as well?
Wondering if this will help:
https://github.com/freebsd/freebsd/commit/db39757b73fd1287ce36fb334619703b76241a4f#diff-e350020ca3658fee0518b53dc6db0317
Wondering if this will help:
https://github.com/freebsd/freebsd/commit/db39757b73fd1287ce36fb334619703b76241a4f#diff-e350020ca3658fee0518b53dc6db0317