Messages

1

23.1 Legacy Series / Re: Multi-WAN: how to force traffic to one specific gateway?

« on: June 15, 2023, 11:52:37 pm »

Can you add some basic network schema (for example a NwDiag from Kroki) and firewall rules list?

If I got it right:

• You have to WAN connections, both in a "CABLE-DSL" gateway group:
  
  • Cable, using the "CABLE-DSL" gateway, is at tier 1
  • DSL, using the "DSL-GW" gateway, is at tier 2, OPNsense uses IP 192.168.177.1 to contact it
• Failover works: Internet access from your LAN uses Cable if available and continues to work if one of your 2 WAN is down (automatic switches)
• You managed (how?) to access 192.168.177.1 address (DSL-GW's)
• You have other devices on 192.168.177.0/24 network that you want to access but when you try to contact them using their 192.168.177.0/24 IP, OPNsense routes traffic to CABLE-DSL which it discards/reject

I guess each router has it's own network address and your LAN is also on a distinct network address.

Usually, to force a WAN for a given destination (host or network), you have to create firewall rule that sets the gateway to the one you want (in your case "DSL-GW" in lieu of "CABLE-DSL").

2

Virtual private networks / Re: WireGuard "Handshake did not complete after 5 seconds": Multi-WAN outgoing issue

« on: October 20, 2022, 10:57:17 am »

How can I "route" outgoing traffic from WireGuard daemon?

3

Virtual private networks / Re: WireGuard setup "Handshake did not complete after 5 seconds": Multi-WAN issue?

« on: October 05, 2022, 02:38:23 am »

I guess I am falling into the same issue as described in this topic with:

The only way is:

- All users use WAN1 as default
- Only if WAN1 fails they have to use WAN2
- When WAN1 is back, all users get kicked and should switch to WAN1

It wont work in a different way ...

I can live with all WireGuard users using WAN1 and then WAN2 if it fails but I does need the WAN1+WAN2 load balancing for other (non-WireGuard) traffic...

4

Virtual private networks / Re: WireGuard setup "Handshake did not complete after 5 seconds": Multi-WAN issue?

« on: October 04, 2022, 07:03:56 pm »

Instead of simple "Allow incoming UDP/51820" rules on both WAN, I tried using NAT Port Forward rules (on both WAN) to redirect incoming UDP/51820 to 127.0.0.1:51820.

And looking at the live view of firewall logs (Firewall: Log Files: Live View) I can see:

Action	Interface	Direction	Time	Source	Destination	Proto	Label
▸	WAN_2	←	…T18:25:33	WAN_2.PUBLIC.IP.ADDRESS:51820	CLI.ENT.IP.ADDRESS:55643	udp	let out anything from firewall host itself (force gw)
▸	WAN_1	→	…T18:25:33	CLI.ENT.IP.ADDRESS:55643	127.0.0.1:51820	udp	NAT to redirect UDP/51820 to 127.0.0.1:51820 (for WG_VPN)
⇄	WAN_1	→	…T18:25:33	CLI.ENT.IP.ADDRESS:55643	WAN_1.PUBLIC.IP.ADDRESS:51820	udp	rdr rule

It looks like the response is going via the other WAN (not the one the request came from).

So I tried adding a firewall rule on loopback interface to force a given interface:

Direction	Protocol	Source	Port	Destination	Port	Gateway	Description
out	UDP	*	51820	*	*	WAN_1	Set the gateway for WireGuard traffic (test)

But the outgoing WireGuard traffic stills uses WAN_2.

If I make the client connect to WireGuard tunnel using IP from WAN_2 as termination/endpoint it works (handshake is OK on server side and the client can access LAN's IP).

Currently on the "System: Gateways: Single" page I have :

Name	Interface	Protocol	Priority
GW_1	WAN_1	IPv4	255 (upstream)
GW_2 (active)	WAN_2	IPv4	255 (upstream)

And my "GW_LoadBalancing" gateway group is a simple group with both GW_1 and GW_2 as "Tier 1".

I guess the outgoing traffic is using GW_2 because of the "(active)" mention? which is lacking on WAN_1?

5

Virtual private networks / Re: WireGuard setup "Handshake did not complete after 5 seconds": Multi-WAN issue?

« on: October 04, 2022, 09:55:11 am »

My bad, I forgot to paste it, here it is:

Firewall: Rules: WAN:

Source	Port	Destination	Port	Gateway	Description
*	*	WAN address	51820	*	Allow incoming WAN traffic on WG_VPN port
10.0.0.0/24	*	10.3.0.0/26	*	GW_LoadBalancing	Set the gateway for LAN clients

6

Virtual private networks / Re: WireGuard setup "Handshake did not complete after 5 seconds": Multi-WAN issue?

« on: October 04, 2022, 02:01:12 am »

Where is that setting? I don't see anything in "Firewall: Settings: Advanced"

I do have an allow rule in "Firewall: Rules: WGVPN".

7

Virtual private networks / WireGuard "Handshake did not complete after 5 seconds": Multi-WAN outgoing issue

« on: October 03, 2022, 06:27:45 pm »

Hello,

I am failing to setup a WireGuard VPN tunnel on my OPNsense (v22.7.4 with "os-wireguard" plugin v1.12) and I begin to think multi-WAN configuration (load balancing, outbound rules, gateway groups, …) is causing issues.

I have, using the guide, already setup a "Road Warrior" WireGuard VPN server on another OPNsense box, but there was only a single WAN.

I've read (on this forum?) that WireGuard is not really (yet?) capable of multi-WAN, and I get it: But could I at least, get it to work over one of the WAN I have (without load balacing nor automatic failover)?

I am using the iOS client and logs shows:

Handshake did not complete after 5 seconds, retrying (try 3)

In the "VPN: WireGuard: List Configuration", the peer part does display "endpoint", "allowed ips" and some "transfer" values, but no "latest handshake" (which the "Handshakes" tab confirms: the timestamps for the peer is at "0").

Here is what I want to have in the end:

• Allow an user to connect to my LAN network from outside Internet
• It does not need to access Internet via the VPN (he'll use it's ISP for that)
• Restrict it to a fixed local IP address
• Apply some restrictions to what local IP addresses it can access (eg. only allow access to a printer having IP 10.0.0.78/24 (I'll try to set this up once I get the previous 3 points up and running)

Here is my setup:

(My LAN is on 10.0.0.0/24)

VPN: WireGuard:
* Local:
  * Enabled: ☑
  * Name: WG_VPN
  * Public Key: Mo_d…Vb1p
  * Listen Port: 51820
  * Tunnel Port: 10.3.0.1/26
  * Peers:
    * user1
* Endpoint:
  * Enabled: ☑
  * Name: user1
  * Public Key: Fx+p…Zw3d
  * Shared Secret: (empty)
  * Allowed IPs: 10.3.0.2/32
  * Endpoint Address: (empty)
  * Endpoint Port: (empty)
  * Keepalive: (empty)
 * List Configuration:
   
    interface: wg1
      public key: Mo_d…Vb1p
      private key: (hidden)
      listening port: 51820

    peer: Fx+p…Zw3d
      allowed ips: 10.3.0.2/32

Interfaces: [WG_VPN]:
* Enabled: ☑
* Lock: ☑
* Device: wg1
* IPv4 Configuration Type: None
* IPv6 Configuration Type: None

Firewall: NAT: Outbound:
* Mode: Hybrid outbound NAT rule generation
* Manual rules:

Interface	Source	NAT Address	NAT Port
WAN_1	10.0.0.0/24	Interface address	*
WAN_2	10.0.0.0/24	Interface address	*

Firewall: Rules: LAN:

Source	Port	Destination	Port	Gateway	Description
10.0.0.0/24	*	10.3.0.0/26	*	*	Allow LAN clients to contact WG_VPN clients
10.0.0.0/24	*	10.3.0.0/26	*	GW_LoadBalancing	Set the gateway for LAN clients

Firewall: Rules: WG_VPN:

Source	Port	Destination	Port	Gateway	Description
*	*	*	*	*	Allow all WG_VPN clients to contact LAN clients (for now)

I tried deleting all the WireGuard setup and starting fresh without much success.

Full logs from the iOS client:

[APP] startActivation: Entering (tunnel: MyCompany)
2022-10-03 17:22:06.517
[APP] startActivation: Starting tunnel
2022-10-03 17:22:06.518
[APP] startActivation: Success
2022-10-03 17:22:06.542
[APP] Tunnel 'MyCompany' connection status changed to 'connecting'
2022-10-03 17:22:06.672
[NET] App version: 1.0.15 (26)
2022-10-03 17:22:06.673
[NET] Starting tunnel from the app
2022-10-03 17:22:06.756
[NET] DNS64: mapped 1.2.3.4 to itself.
2022-10-03 17:22:06.756
[NET] Attaching to interface
2022-10-03 17:22:06.757
[NET] UAPI: Updating private key
2022-10-03 17:22:06.757
[NET] Routine: decryption worker 1 - started
2022-10-03 17:22:06.757
[NET] Routine: event worker - started
2022-10-03 17:22:06.757
[NET] Routine: encryption worker 1 - started
2022-10-03 17:22:06.757
[NET] Routine: handshake worker 1 - started
2022-10-03 17:22:06.757
[NET] Routine: decryption worker 2 - started
2022-10-03 17:22:06.757
[NET] Routine: handshake worker 2 - started
2022-10-03 17:22:06.758
[NET] Routine: encryption worker 2 - started
2022-10-03 17:22:06.758
[NET] Routine: TUN reader - started
2022-10-03 17:22:06.759
[NET] UAPI: Removing all peers
2022-10-03 17:22:06.761
[NET] peer(Mo_d…Vb1p) - UAPI: Created
2022-10-03 17:22:06.763
[NET] peer(Mo_d…Vb1p) - UAPI: Updating endpoint
2022-10-03 17:22:06.765
[NET] peer(Mo_d…Vb1p) - UAPI: Updating persistent keepalive interval
2022-10-03 17:22:06.767
[NET] peer(Mo_d…Vb1p) - UAPI: Removing all allowedips
2022-10-03 17:22:06.769
[NET] peer(Mo_d…Vb1p) - UAPI: Adding allowedip
2022-10-03 17:22:06.771
[NET] UDP bind has been updated
2022-10-03 17:22:06.771
[NET] Routine: receive incoming v4 - started
2022-10-03 17:22:06.773
[NET] peer(Mo_d…Vb1p) - Starting
2022-10-03 17:22:06.773
[NET] Routine: receive incoming v6 - started
2022-10-03 17:22:06.775
[NET] peer(Mo_d…Vb1p) - Sending keepalive packet
2022-10-03 17:22:06.775
[NET] peer(Mo_d…Vb1p) - Routine: sequential sender - started
2022-10-03 17:22:06.775
[NET] peer(Mo_d…Vb1p) - Routine: sequential receiver - started
2022-10-03 17:22:06.777
[NET] peer(Mo_d…Vb1p) - Sending handshake initiation
2022-10-03 17:22:06.780
[NET] Interface state was Down, requested Up, now Up
2022-10-03 17:22:06.781
[NET] Device started
2022-10-03 17:22:06.783
[NET] Tunnel interface is utun2
2022-10-03 17:22:06.786
[APP] Tunnel 'MyCompany' connection status changed to 'connected'
2022-10-03 17:22:06.787
[NET] Network change detected with satisfied route and interface order [pdp_ip0]
2022-10-03 17:22:06.788
[NET] DNS64: mapped 1.2.3.4 to itself.
2022-10-03 17:22:06.790
[NET] peer(Mo_d…Vb1p) - UAPI: Updating endpoint
2022-10-03 17:22:06.792
[NET] Routine: receive incoming v4 - stopped
2022-10-03 17:22:06.792
[NET] Routine: receive incoming v6 - stopped
2022-10-03 17:22:06.796
[NET] UDP bind has been updated
2022-10-03 17:22:06.796
[NET] Routine: receive incoming v6 - started
2022-10-03 17:22:06.796
[NET] Routine: receive incoming v4 - started
2022-10-03 17:22:07.044
[NET] Network change detected with satisfied route and interface order [pdp_ip0, utun2]
2022-10-03 17:22:07.045
[NET] DNS64: mapped 1.2.3.4 to itself.
2022-10-03 17:22:07.072
[NET] peer(Mo_d…Vb1p) - UAPI: Updating endpoint
2022-10-03 17:22:07.074
[NET] Routine: receive incoming v4 - stopped
2022-10-03 17:22:07.074
[NET] Routine: receive incoming v6 - stopped
2022-10-03 17:22:07.076
[NET] UDP bind has been updated
2022-10-03 17:22:07.078
[NET] Routine: receive incoming v6 - started
2022-10-03 17:22:07.078
[NET] Routine: receive incoming v4 - started
2022-10-03 17:22:11.509
[APP] Status update notification timeout for tunnel 'MyCompany'. Tunnel status is now 'connected'.
2022-10-03 17:22:12.044
[NET] peer(Mo_d…Vb1p) - Handshake did not complete after 5 seconds, retrying (try 2)
2022-10-03 17:22:12.045
[NET] peer(Mo_d…Vb1p) - Sending handshake initiation
2022-10-03 17:22:17.210
[NET] peer(Mo_d…Vb1p) - Handshake did not complete after 5 seconds, retrying (try 2)
2022-10-03 17:22:17.210
[NET] peer(Mo_d…Vb1p) - Sending handshake initiation
2022-10-03 17:22:22.497
[NET] peer(Mo_d…Vb1p) - Handshake did not complete after 5 seconds, retrying (try 2)
2022-10-03 17:22:22.497
[NET] peer(Mo_d…Vb1p) - Sending handshake initiation
2022-10-03 17:22:27.681
[NET] peer(Mo_d…Vb1p) - Handshake did not complete after 5 seconds, retrying (try 3)
2022-10-03 17:22:27.682
[NET] peer(Mo_d…Vb1p) - Sending handshake initiation
2022-10-03 17:22:32.936
[NET] peer(Mo_d…Vb1p) - Handshake did not complete after 5 seconds, retrying (try 4)

8

21.7 Legacy Series / FTP Proxy for LAN clients to connect to any external FTP server

« on: November 03, 2021, 03:44:40 pm »

Hello,

I want to provide an FTP proxy for my LAN clients so that their FTP traffic goes via the OPNsense router and via it's WAN.
I want them to be able to connect to any FTP server.

I got the "os-ftp-proxy" plugin installation part OK and successfully configured a proxy to connect to a fixed server (using "Reverse address" and "Reverse port" ports).
For the client (FileZilla), I set the FTP proxy IP and port (IP of OPNsense aand 8021) and the following custom auth sequence:

USER %u
PASS %p

But, for my real use case I need to be able to connect to any FTP server, so I emptied both "Reverse address" and "Reverse port" fields from the FTP proxy configuration and set FileZilla FTP proxy settings to the custom auth sequence:


OPEN %h
USER %u
PASS %p
ACCT %a

But connections attempts timeout.

Using Wireshark I don't see any mention of the real FTP server address in my outgoing traffic.

I could not find any tutorial about the client-side configuration for a FTP proxy setup: is it that software-dependent?

9

General Discussion / Re: [SOLVED] How to compute the PC Engines APU serial number from MAC ID?

« on: April 16, 2021, 06:28:39 pm »

Side note: I did got a strange serial number from kenv on a APU2C2 (OPNsense v20.1):

boot_serial="YES"
smbios.planar.serial="123456789"
smbios.system.serial="123456789"

(So I went for the binary computation, faster this time since I got the "last 6 digits of the MAC address" thing clear )

10

General Discussion / Re: How to compute the PC Engines APU serial number from MAC ID?

« on: April 16, 2021, 05:48:02 pm »

@pmhausen I was under the (false) impression it was the serial number of another part/chip of the PCB... But you are totally right... Way more simple... 

@Maurice True, I typed random hexadecimal digits for the example, not paying attention to the "multiple of 4" thing 

Thanks

11

General Discussion / [SOLVED] How to compute the PC Engines APU serial number from MAC ID?

« on: April 16, 2021, 02:16:35 am »

I want to get the serial number of my PC Engines APU2 card and the documentation (https://www.pcengines.ch/ht_macid.htm) says:

The MAC ID of the first NIC on all PC Engines boards is derived of its serial number. The following NICs have subsequent addresses.

This is the formula to convert from MAC ID to serial number and vice versa:

MAC ID = 00:0d:b9 (our OUI) : (serial + 64) * 4

serial = (MAC ID & 0x000000FFFFFF) / 4 - 64

As I am not really sure about my understanding of the formula (and didn't found more explanation/examples on the Internet not a shell command to get it) I would like to share my process of computing the serial number (so that it can be useful to others, and be corrected if wrong, in the reverse order )

So if my igb0 NIC's MAC address is 00:0d:b9:3e:ff:25.

First I convert the MAC ID from hexadecimal (00:0d:b9:3e:ff:25) to binary:

hex= 0    0   :0    d   :b    9   :3    e   :f    f   :2    5
bin= 0000 0000 0000 1101 1011 1001 0011 1110 1111 1111 0010 0101

Also the mask (000000ffffff) must be converted to binary:

hex= 0    0    0    0    0    0    f    f    f    f    f    f
bin= 0000 0000 0000 0000 0000 0000 1111 1111 1111 1111 1111 1111

Then I apply the MAC_ID & MASK AND bitmask operation and :

mac= 0000 0000 0000 1101 1011 1001 0011 1110 1111 1111 0010 0101
msk= 0000 0000 0000 0000 0000 0000 1111 1111 1111 1111 1111 1111
res= 0000 0000 0000 0000 0000 0000 0011 1110 1111 1111 0010 0101

obtained res is 4128549 in decimal notation.

My APU serial number would be: 4128549 / 4 - 64 = 1032073 ?

Thanks for any help you could provide.

12

Hardware and Performance / Re: OpnSense on Sophos SG series

« on: March 04, 2020, 01:02:17 am »

New update: v20.1 is running just fine on my Sophos SG 115 (for about 20 days now).

13

20.1 Legacy Series / Re: 1:1 NAT with an IPsec tunnel (in a multi-WAN setup)

« on: February 25, 2020, 01:03:43 pm »

Adding a firewall rule on LAN interface configured as follow did the trick: from=LAN_net to=172.31.254.254/32 gateway=default
(and placed before my firewall rule that sets gateway to the "primary/failover" gateway group for all traffic)

A tcpdump -n -i enc0 still don't show translation for outgoing traffic (id. 10.33.0.1) but answer arrives for translated address (id. 10.88.0.1):

12:39:44.593119 (authentic,confidential): SPI 0xbde8b870: IP 10.33.0.1 > 172.31.254.254: ICMP echo request, id 60353, seq 18, length 64
12:39:44.593503 (authentic,confidential): SPI 0xbde8b870: IP 172.31.254.254 > 10.88.0.1: ICMP echo reply, id 60353, seq 18, length 64

In the web GUI, at "Firewall: Log Files: Live View", I do have translation for outgoing traffic (id. 10.88.0.1):

Code: [Select]

IPsec Feb 25 12:39:44 10.88.0.1 172.31.254.254 icmp IPsec internal host to host
lan Feb 25 12:39:44 10.88.0.1 172.31.254.254 icmp Allow access to third-party server behind IPsec tunnel
So, to summarize, I needed (on my side):

• A firewall rule on all WAN interfaces to allow ESP ISAKMP and IPsec NAT-T (see guide)
• IPsec tunnel with "Apply Policy" on Phase 1 and Phase 2 where local network="10.88.0.0/16", remote network="172.31.254.254/32" and Manual SPD entry=10.33.0.0/16
• A NAT "One-to-One" rule where type=BINAT, external network="10.88.0.0/16", source network="10.33.0.0/16" and destination network="172.31.254.254/32"
• A firewall rule on LAN interface where source="LAN_net:*", direction="in", destination="172.31.254.254/32:*", gateway="default"
• A firewall rule on IPsec interface where source="172.31.254.254/32:*", direction="in", destination="LAN_net:*", gateway="default"

Guides :

• Setup IPsec site to site tunnel
• IPSec BINAT

14

20.1 Legacy Series / Re: 1:1 NAT with an IPsec tunnel

« on: February 24, 2020, 05:23:45 pm »

Just a thought: could my multi-wan setup (primary with failover) be responsible for the LAN clients' traffic not going through the tunnel?

15

20.1 Legacy Series / Re: 1:1 NAT with an IPsec tunnel

« on: February 24, 2020, 03:27:11 pm »

Also, if my LAN clients ping or traceroute the IP 172.31.254.254 it goes out to WAN not via IPsec tunnel (tcpdump -n -i enc0).
I don't understand why a ping to the same IP would follow different path if executed from OPNsense with LAN interface as source or executed from a computer on LAN (having the OPNsense as default gateway).

I naively tried to add a "Firewall: NAT: Outbound" rule on IPsec interface to replace traffic from 10.33.0.0/16 (LAN) to 172.31.254.254 (third party server) by 10.88.0.0/16 (natted network) with no luck.