Messages

[A0369#############]

Hello all,

I was just wondering if anyone has attempted setting up OPNSense using real PCI Express passthrough of the NIC, to bypass the software virtual switch in Hyper-V environments. Which should treat OPNSense as if it was running on a metal box vs dealing with the Host Windows OS and Virtual Switch. Great for things like VLANs, intrusion detection and other plug-ins of that nature better suited with real NIC access.

I tried to do it this evening, but not without an error which may be driver related. However, I am not entirely sure and maybe someone can chime in with ideas.

FreeBSD 11.1 fully supports this type of PCI Passthrough/DDA on a Windows Server 2016+ Host OS w/Hyper-V:
https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/Supported-FreeBSD-virtual-machines-on-Hyper-V

"With Windows Server 2016 administrators can pass through PCI Express devices via the Discrete Device Assignment mechanism. Common devices are network cards, graphics cards, and special storage devices. The virtual machine will require the appropriate driver to use the exposed hardware. The hardware must be assigned to the virtual machine for it to be used."

I used this as a guide, so I don't take credit for the base script:
https://blogs.technet.microsoft.com/heyscriptingguy/2016/07/14/passing-through-devices-to-hyper-v-vms-by-using-discrete-device-assignment/

Using that guide as a base and making modifications...
The entered Windows Server 2016 PowerShell commands were the following:

$vmName = 'OPNSense Firewall'
$vm = Get-VM -Name $vmName
$dev = "PCI\VEN_8086&DEV_1521&SUBSYS_50018086&REV_01\A0369#############"
^^^ CAN BE FOUND IN DEVICE MANAGER - PROPERTIES - DETAILS - DEVICE INSTANCE PATH PROPERTY - # = omitted information for privacy (just in case)

Disable-PnpDevice -InstanceId $dev -Confirm:$false

$locationPath = (Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths -InstanceId $dev).Data[0]

Dismount-VmHostAssignableDevice -LocationPath $locationPath -Force -Verbose

Add-VMAssignableDevice -VM $vm -LocationPath $locationPath -Verbose


Once this was done, the NIC appeared in OPNSense Console immediately and on reboot. However, due to issues with either the NIC, FreeBSD, OPNSense, or Kernel Drivers. I was unable to utilize this Intel I350 NIC Port as a direct PCI Passthrough WAN port for my testing purposes.

I had the following console output errors on OPNSense:
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> at device 0.0 on pci0
igb0: Unable to map MSIX table
igb0: Using an MSI interrupt
igb0: Setup of Shared code failed
device_attach: igb0 attach returned 6

Has anyone seen these igb0 errors or have any information to resolve for the Intel I350 (with latest firmware)?
Not sure if this is Hyper-V pass-through related or OPNSense/FreeBSD Compatibility/Driver Issue.

FYI: Firmware being used on Intel I350 (Dell OEM) Version 18.5.18: https://www.dell.com/support/home/us/en/19/Drivers/DriversDetails?driverId=3XJH0

Thanks in advance for any assistance or ideas!
  - Dan

Same issue here... still exists on 18.7.5

Worked fine before updates without any changes.

opnsense: /services_dyndns_edit.php: Dynamic DNS (DOMAIN NAME HERE): ERROR - Zone ID was not found.

The hangs and error messages point to traffic being pushed over OpenVPN links that are not yet set up / active.

The line in question reads [84]: nat on ovpns3 inet proto {tcp udp} from (ovpns3:network) to {PRIVATE IP} port {53} -> ovpns3 port 1024:65535 #

Points to "ovpns3" causing the error, again, because it's being configured without being up/configured itself. I've added more () magic here in subsequent commits but testing is difficult and we need to give it some time on -devel.


Cheers,
Franco

If I switch from Production to Development build. Are those changes part of the update download or would another patch be needed to test?

Sent from my SM-N950U using Tapatalk

Ok, we're working on this issue here. There is a patch for your issue that should work already:

https://github.com/opnsense/core/issues/2625


Cheers,
Franco

I ran patch 71043f1

I still see "Configuring Firewall... failed" during the boot sequence. However, it passes right by, instead of hanging there for 5-10 minutes as it was doing before.

Also it may be unrelated but it also stays at "Configuring Dynamic DNS" for a few minutes, as well.

I still have 5 Notices in the GUI. Two of them are new after running the patch. (I acknowledged notices before redoing a reboot)

New:
There were error(s) loading the rules: /tmp/rules.debug:84: no translation address with matching address family found. - The line in question reads [84]: nat on ovpns3 inet proto {tcp udp} from (ovpns3:network) to {PRIVATE IP} port {53} -> ovpns3 port 1024:65535 #

The other notices are the same no IP address found for ovpns3, ovpns4, and ovpnc2:0

Can't be both oO

Let's widen the grep then...

# grep :network /tmp/rules.debug


Cheers,
Franco

I found the grep command issue... anyways here is the output if it helps resolve this issue or maybe I have rules that I do not need.


nat on ovpns3 inet proto {tcp udp} from ovpns3:network to {PRIVATE IP ADDRESS OMITTED} port {53} -> ovpns3 port 1024:65535 # Redirect All Unknown DNS Servers to Internal DNS

nat on ovpns3 inet proto {tcp udp} from ovpns3:network to {127.0.0.1} port {123} -> ovpns3 port 1024:65535 # Redirect All Time Inquires to This Firewall

pass in quick on hn0 inet from {PRIVATE IP ADDRESS OMITTED} to {(ovpns3:network)} keep state label "USER_RULE: Allow Clients Access to Domain Controller on Remo..." # e459891a011793b7e3cb4f5070b1082b

pass in quick on hn0 inet from {(ovpns3:network)} to {(hn0:network)} keep state label "USER_RULE" # aa280ba13da6a0070fc7dff60ae496da


nat on ovpns4 inet proto {tcp udp} from ovpns4:network to {PRIVATE IP ADDRESS OMITTED} port {53} -> ovpns4 port 1024:65535 # Redirect All Unknown DNS Servers to Internal DNS

nat on ovpns4 inet proto {tcp udp} from ovpns4:network to {127.0.0.1} port {123} -> ovpns4 port 1024:65535 # Redirect All Time Inquires to This Firewall

pass in quick on hn0 inet from {PRIVATE IP ADDRESS OMITTED} to {(ovpns4:network)} keep state label "USER_RULE: Allow Clients Access to Domain Controller on Site..." # 9dbe6a70da2a9a5461f6f3e4e532ccde

pass in quick on hn0 inet from {(ovpns4:network)} to {(hn0:network)} keep state label "USER_RULE" # ffa36a87892035d410807c749bd6a3b9


pass out log route-to ( ovpnc2 PRIVATE IP ADDRESS OMITTED ) from {ovpnc2} to {!(ovpnc2:network)} keep state allow-opts label "let out anything from firewall host itself"

pass in log quick on hn1 route-to ( ovpnc2 PRIVATE IP ADDRESS OMITTED ) inet from {(hn1:network)} to {any} keep state label "USER_RULE" # 0618b11751c2e8b2eb0f3d4807c00856

pass in quick on ovpnc2 route-to ( ovpnc2 PRIVATE IP ADDRESS OMITTED ) inet from {(ovpnc2:network)} to {any} keep state label "USER_RULE" # ce37ac04bdf4e549df26c96894cc3b60

When the firewall comes up but OpenVPN is not yet active this can happen.

Can you grep for me to be sure?

# grep 'ovpn[0-9].*:network' /tmp/rules.debug


Thanks,
Franco

Is it normal for that command to give no output (unless I am typing it wrong)?
18.7.1 - issue remains.

I see what you mean, you cannot enter WAN address there, only a static.


Hmm, see no reason why we cannot alter that a little to allow the selected WAN address, might need to make it an alias.


I'll look into it and see if it can be done.

Thank you :)

Sent from my SM-N950U using Tapatalk

Yes, that will work.

Yes, but do you happen to know the steps involved to assign a DHCP WAN address to a 1:1 NAT?  Last time I tried (on an older build of opnsense), I only saw options for manual entry of a static address.

Dan

1:1 is really for use if you have multiple WAN IPs. If  you only have one WAN IP then all WAN incoming traffic would be mapped to a specific internal IP, is that what you want?

I have TWO WAN IPs, just they are both DHCP.  I want to use one of them in a 1:1 setting.

Yup, all sorted in 18.7. Thanks for testing!


Cheers,
Franco

Could these two error messages be related to this change we made?

Whenever I reset start my OPNSense box... it hangs on loading firewall rules.

I see these alert notices in the OPNSense GUI once it loads after like 15 minutes from reboot.

There were error(s) loading the rules: no IP address fround for ovpn3:network
There were error(s) loading the rules: no IP address fround for ovpn4:network

Unless there is somewhere I should look to edit a file to resolve this?  Might be a migration glitch as well.

- Dan

I know OPNSense supports using 1:1 NAT from an external WAN IP to an internal LAN IP.

However, does it support doing this with a DHCP WAN IP Address?
This WAN IP Address will be updated using a DynamicDNS Service - this not worried about it being DHCP vs Static.

If this is possible, what configuration steps are needed to accomplish this?

Thanks!

Clear the log file once from the log file page and it should be back then. There were incompatible changes early in 18.1.x.


Cheers,
Franco

That worked! Thanks!

Sent from my SM-G892A using Tapatalk

Hi there,

Ah, thanks for testing and spotting this. The following should help: https://github.com/opnsense/core/commit/24f1d05c

# opnsense-patch 24f1d05c

Please confirm so we can fast-track this to RC2. :)


Thanks,
Franco

I reverted my // changes and tested your patch. All seems to be working for the moment! I will let you know if anything changes!

Now I just need to figure out why suricata.log hasn't been working in a long time...  always says operation is not supported by device or something like that.

- Dan

[I figured out a temporary fix...Made a OPNSense Configuration Backup File. I deleted my OpenVPN Client for Private Internet Access]

I figured out a temporary fix...
Made a OPNSense Configuration Backup File.
I deleted my OpenVPN Client for Private Internet Access

I commented out these lines in /usr/local/etc/inc/plugins.inc.d/openvpn.inc after chmod 777 of the file in the shell using the file edit feature in WinSCP. Then rebooted my OPNSense box. These lines started around line 620.

//        case 'p2p_tls':
//            // For non user auth types setup client specific overrides, user authenticated ones are commissioned
//            // using the auth script in option auth-user-pass-verify
//            $conf .= "client-connect \"/usr/local/etc/inc/plugins.inc.d/openvpn/ovpn_setup_cso.php {$mode_id}\"\n";
//            break;

Then reloaded my backup OpenVPN configuration. It then connected.

[client2.conf]

I decided to give the 18.7 r1 a try...

My OpenVPN servers are working perfectly....
However, my OpenVPN Client for Private Internet Access is not.

It will not connect, no configuration has changed and was working in 18.1.11.

The OpenVPN log shows the following:
Options error: --client-connect requires --mode server


client2.conf

dev ovpnc2
verb 4
dev-type tun
dev-node /dev/tun2
writepid /var/run/openvpn_client2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto tcp4-client
cipher AES-256-CBC
auth SHA256
up /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkup
down /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkdown
client-connect "/usr/local/etc/inc/plugins.inc.d/openvpn/ovpn_setup_cso.php client2"
tls-client
client
lport 0
management /var/etc/openvpn/client2.sock unix
remote us-newyorkcity.privateinternetaccess.com 501
auth-user-pass /var/etc/openvpn/client2.up
ca /var/etc/openvpn/client2.ca
comp-lzo no
passtos
route-nopull
resolv-retry infinite
reneg-sec 0
disable-occ
pull-filter ignore "auth-token"
topology net30


Any suggestions on how to resolve this?
- Dan