Topics

1

24.1 Legacy Series / Automatic outbound NAT rule not populating

« on: May 27, 2024, 03:42:04 pm »

I have a new OPNsense setup and have the lan and wan side configured. The lan side is providing dhcp configured correctly. The wan is fios that is pulling an ip and default route. The opnsense can ping Google and raw up addresses. An automatic gateway was setup and monitoring to 1.1.1.1 with great latency.

The issue is that it doesn’t matter what I have set in outbound nat no automatic entries are created. I’ve tried to find the logs for that but haven’t had any luck. I see a manual entry right now but would like to change it. I’ve done some searching and made sure options like upstream gateway are checked. The gateway says default in the rules.

I’m sure it’s something simple but happy to look through the logs if you can point me to the ones that would have relevant information on creating or not the automatic or hybrid rules.


Thanks

2

22.7 Legacy Series / HAProxy access from internal network

« on: August 31, 2022, 07:06:16 pm »

I'm using HAProxy + ACME on OPNsense to provide a reverse proxy to my internal services. After another small conditions issue, I now have it working as expected from the external internet on my phone (LTE connection).

I going to it using chrome and firefox by typing in the FQDN: https://server1.mydomain.com

However when I turn on wifi and am on the same network as the Real Server I get an ERR_TIMED_OUT. In HAProxy log I get a handshake failure error. I tried it using the same process with my laptop with a VPN to the internet and connecting in, the internal site loads as expected. When I disconnect the VPN an try it on the same subnet I get the same error as my phone.

Code: [Select]

2022-08-31T12:59:21-04:00 Error haproxy 173.66.23.118:2188 [31/Aug/2022:12:59:21.223] default_443/0.0.0.0:443: SSL handshake failure

3

22.7 Legacy Series / [SOLVED] HAProxy issue 403 error

« on: August 31, 2022, 01:16:48 am »

I recently resetup HAProxy with ACME let's encrypt using a known working setup guide.

I'm getting a 403 error with the message from the guide in the body of the site.

The haproxy log shows the following each time I try to get to the page from outside my network:

Code: [Select]

Error haproxy 154.89.5.203:44012 [30/Aug/2022:19:10:22.111] default_443~ default_443/<NOSRV> 0/-1/-1/-1/0 403 203 - - PR-- 1/1/0/0/0 0/0 "GET / HTTP/1.0"
I'm happy to provide additional logs to help troubleshoot the issue.

4

21.7 Legacy Series / Wireguard not switching from WAN2 back to WAN1

« on: September 06, 2021, 05:27:13 am »

I have WAN1 connected to my ISP which sometimes goes down for a minute or two. I have WAN2 connected to an LTE Modem so one is igb0 and the other is ibg1.

WAN1 latency is around 90ms, WAN2 is around 170ms.

I have 2 VPNs setup, one is OpenVPN and the other is Wireguard, they are used to support different services.

When I have a failure, I can see that both VPNs route through WAN2 and their latency jumps up. A few minutes after WAN1 comes back up the OpenVPN VPN will switch back to WAN1 and you can see the latency creep down. However, the Wireguard VPN never switches back and the latency stays high and I can see the traffic on my LTE modem.

I wanted to check if there is a setting that I'm missing or a way to force it to restart the connection, I know Wireguard isn't an always on type of connection but not sure what is happening there.

I'm not skilled with Monit but I guess I could use something like that to restart the service if the latency of the OpenVPN and Wireguard aren't close but I would like to see if there is a setting in the configs that I'm missing.

Thanks

5

21.7 Legacy Series / Nut Failure

« on: August 30, 2021, 04:00:25 pm »

I am running the latest OPNsense 21.7 and am trying to plug in my APC UPC into it. I have NUT running on a Debian 10 system and it works with the usbhid-ups driver.

I can't get it to work with OPNsense, the diagnostic page never loads. Is there a place in the logs, I can check.

Secondarily, my Debian 10 box is pushing the NUT info out on the network and I have another machine that can pull the info but not my OPNsense.

Thanks

6

20.7 Legacy Series / 20.7.7 Upgrade Broke Gateway Status and looking to fix it

« on: December 20, 2020, 09:20:18 am »

I just upgraded from 20.7.5 to 20.7.7 using the regular upgrade process, 20.7.5 was working perfect since installing it and survived several reboots. I made a backup configuration right before the upgrade.

After the upgrade I went from see the gateway status for my wan and multiple vpn clients but now only 1 of the vpns is showing an online status the other show offline (but are workign). I've included the syslog for when I try to reboot the dpingers that fail to start.

I have checked/deleted the /tmp/pppoe0*(WAN) files, same with ovpnc1*(EXPRESSVPN).

Code: [Select]

Dec 20 15:17:24 op opnsense-devel[81393]: /status_services.php: Choose to bind WAN on Array since we could not find a proper match.
Dec 20 15:17:24 op opnsense-devel[81393]: /status_services.php: The WAN IPv4 gateway address is invalid, skipping.
Dec 20 15:17:30 op opnsense-devel[81393]: /status_services.php: Choose to bind EXPRESSVPN_VPNV4 on Array since we could not find a proper match.
Dec 20 15:17:30 op opnsense-devel[81393]: /status_services.php: The EXPRESSVPN_VPNV4 IPv4 gateway address is invalid, skipping.

7

20.7 Legacy Series / Mellanox ConnectX-3 support

« on: November 21, 2020, 10:18:04 am »

I was able to follow mimugmail's instructions that were posted to the forum and to the site at the bottom of this post. I was able to get the driver to autoload on boot and updated the firmware.

These cards are able to do 56Gbps instead of just 40Gbps if being used with an FDR cable and capable switch such as the SX6036 I'm using. I have this card in some ESXi machines and everything is detected as 56Gbps as expected.

I would like to see if there is a way to set these cards at the faster speed in OPNsense, the interface settings only offer autoselect, 1G, 10G, and 40G right now.

I would also like to see if there are specific options to enable OPNsense to take "full" advantage of these cards such as hardware offloading. I am using VLANs so, the hardware offloading might not work.

Any help would be greatly appreciated.

https://www.routerperformance.net/opnsense/mellanox-connecx-management-in-opnsense/

8

20.7 Legacy Series / Intra VLAN performance issues when WAN has issues

« on: November 13, 2020, 03:12:10 pm »

I'll start by saying that my internet has high latency of around 150ms or more and that the ISP will randomly have slow downs and high packet loss. I have been able to verify this shouldn't be my opnsense setup by using the router provided by the ISP to test with.

My OPNsense hardware is running on an AMD Epyc 3000, 32gb ram, nvme drive, 1GBe interface to ISP ONT, and a 10GBe network to my 10GBe switch. I know it's overkill but the small machine was meant for something else that I never got around to doing.

My OPNsense configuration, has 3-4 VLANs for management, users, guests, iot like things and the OPNsense does the routing.

Normally this setup works pretty well without issues but when my ISP drops packets or it's connection, all of my intraVLAN traffic suffers. I was using Plex from one VLAN to another (waiting on another interface card for Plex server to avoid this) and it wouldn't play because of how much buffer it had to do. I did a couple of tests like manually down the WAN interface and unplug the WAN interface, both of these "fixed" the internal network issues and Plex streamed across the router without an issue. I have the same issue with SMB shares dropping connections if I am trying to transfer a file.

I wanted to see if anyone had some thoughts or ideas to help me troubleshoot.

Thanks

9

20.7 Legacy Series / [Solved] OpenVPN stopped working after latest upgrade

« on: September 17, 2020, 06:24:28 pm »

I just upgraded from 20.7.1 to 20.7.2, after a reboot my openvpn stopped working. I've included some of the logs that I can find. Willing to grab other logs as needed to help troubleshoot the issue. It looks like the connection is partially successful but then fails with error codes and shows as down in the gateways. I am using expressvpn and have change the endpoint and validated what configurations are needed in the client config.

I see a section that shows cannot assign address and I've confirmed that nothing in my network is in that range either.

Client Instance Status:
ExpressVPN UDP4    Unable to contact daemon    Service not running?

Code: [Select]

2020-09-17T23:23:50 openvpn[28094] Exiting due to fatal error
2020-09-17T23:23:50 openvpn[28094] TCP/UDP: Socket bind failed on local address [AF_INET]10.151.0.98:0: Can't assign requested address (errno=49)
2020-09-17T23:23:50 openvpn[28094] Socket Buffers: R=[42080->524288] S=[57344->524288]
2020-09-17T23:23:50 openvpn[28094] TCP/UDP: Preserving recently used remote address: [AF_INET]70.39.102.162:1195
2020-09-17T23:23:50 openvpn[28094] Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
2020-09-17T23:23:50 openvpn[28094] Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
2020-09-17T23:23:50 openvpn[28094] NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2020-09-17T23:23:50 openvpn[28094] MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client1.sock
2020-09-17T23:23:50 openvpn[74407] library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
2020-09-17T23:23:50 openvpn[74407] OpenVPN 2.4.9 amd64-portbld-freebsd12.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Jul 28 2020
2020-09-17T23:23:50 openvpn[74407] WARNING: file '/var/etc/openvpn/client1.up' is group or others accessible
2020-09-17T23:23:50 openvpn[96840] SIGTERM[hard,] received, process exiting
2020-09-17T23:23:50 openvpn[96840] /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkdown ovpnc1 1500 1557 10.151.0.98 10.151.0.97 init
2020-09-17T23:23:50 openvpn[96840] Closing TUN/TAP interface
2020-09-17T23:21:49 openvpn[96840] /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkup ovpnc1 1500 1557 10.151.0.98 10.151.0.97 init
2020-09-17T23:21:49 openvpn[96840] /sbin/ifconfig ovpnc1 10.151.0.98 10.151.0.97 mtu 1500 netmask 255.255.255.255 up
2020-09-17T23:21:49 openvpn[96840] TUN/TAP device /dev/tun1 opened
2020-09-17T23:21:49 openvpn[96840] TUN/TAP device ovpnc1 exists previously, keep at program end
2020-09-17T23:21:49 openvpn[96840] ROUTE_GATEWAY 101.108.0.1/255.255.255.255 IFACE=pppoe0 HWADDR=00:00:00:00:00:00
2020-09-17T23:21:49 openvpn[96840] Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2020-09-17T23:21:49 openvpn[96840] Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: data channel crypto options modified
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: adjusting link_mtu to 1629
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: peer-id set
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: route options modified
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: --ifconfig/up options modified
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: compression parms modified
2020-09-17T23:21:49 openvpn[96840] OPTIONS IMPORT: timers and/or timeouts modified
2020-09-17T23:21:49 openvpn[96840] PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 10.151.0.1,comp-lzo no,route 10.151.0.1,topology net30,ping 10,ping-restart 60,ifconfig 10.151.0.98 10.151.0.97,peer-id 25,cipher AES-256-GCM'
2020-09-17T23:21:49 openvpn[96840] SENT CONTROL [Server-6883-0a]: 'PUSH_REQUEST' (status=1)
2020-09-17T23:21:48 openvpn[96840] [Server-6883-0a] Peer Connection Initiated with [AF_INET]70.39.102.170:1195
2020-09-17T23:21:48 openvpn[96840] Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 2048 bit RSA
2020-09-17T23:21:48 openvpn[96840] WARNING: 'auth' is used inconsistently, local='auth [null-digest]', remote='auth SHA512'
2020-09-17T23:21:48 openvpn[96840] WARNING: 'cipher' is used inconsistently, local='cipher AES-256-GCM', remote='cipher AES-256-CBC'
2020-09-17T23:21:48 openvpn[96840] WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1554', remote='link-mtu 1606'
2020-09-17T23:21:48 openvpn[96840] VERIFY OK: depth=0, C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=Server-6883-0a, emailAddress=support@expressvpn.com
2020-09-17T23:21:48 openvpn[96840] VERIFY X509NAME OK: C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=Server-6883-0a, emailAddress=support@expressvpn.com
2020-09-17T23:21:48 openvpn[96840] VERIFY EKU OK
2020-09-17T23:21:48 openvpn[96840] ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
2020-09-17T23:21:48 openvpn[96840] Validating certificate extended key usage
2020-09-17T23:21:48 openvpn[96840] VERIFY KU OK
2020-09-17T23:21:48 openvpn[96840] VERIFY OK: depth=1, C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=ExpressVPN CA, emailAddress=support@expressvpn.com
2020-09-17T23:21:48 openvpn[96840] WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
2020-09-17T23:21:48 openvpn[96840] TLS: Initial packet from [AF_INET]70.39.102.170:1195, sid=f859e7f5 bcbb7064
2020-09-17T23:21:48 openvpn[96840] MANAGEMENT: Client disconnected
2020-09-17T23:21:48 openvpn[96840] MANAGEMENT: CMD 'state all'
2020-09-17T23:21:48 openvpn[96840] MANAGEMENT: Client connected from /var/etc/openvpn/client1.sock
2020-09-17T23:21:47 openvpn[96840] UDPv4 link remote: [AF_INET]70.39.102.170:1195
2020-09-17T23:21:47 openvpn[96840] UDPv4 link local: (not bound)
2020-09-17T23:21:47 openvpn[96840] Socket Buffers: R=[42080->524288] S=[57344->524288]
2020-09-17T23:21:47 openvpn[96840] TCP/UDP: Preserving recently used remote address: [AF_INET]70.39.102.170:1195
2020-09-17T23:21:47 openvpn[96840] Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
2020-09-17T23:21:47 openvpn[96840] Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
2020-09-17T23:21:47 openvpn[96840] NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2020-09-17T23:21:47 openvpn[96840] MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client1.sock
2020-09-17T23:21:47 openvpn[82953] library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
2020-09-17T23:21:47 openvpn[82953] OpenVPN 2.4.9 amd64-portbld-freebsd12.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Jul 28 2020
2020-09-17T23:21:47 openvpn[82953] WARNING: file '/var/etc/openvpn/client1.up' is group or others accessible

10

20.7 Legacy Series / DHCP doesn't seem to be started but is working

« on: August 03, 2020, 12:47:37 pm »

I have just recently moved over from pfSense to OPNsense and have been happy so far. I have a few questions that I'll break up into their own topics.

The first one is that I have several vlans that each have their own DHCP service running. They all seem to be working but the status icon isn't showing as such. I've included a picture of what I mean. Please let me know which logs would be best to check / upload to help with the troubleshooting.

Thanks