Messages

16

22.7 Legacy Series / Re: Access website locally on different subnets

« on: October 14, 2022, 02:42:07 am »

This may not help you, but, I ran into this issue on my nascent OPNsense setup within my LAN as well.  A lot of tcpdumping later I ended up resolving the issue with a split DNS configuration.  Essentially, the OPNsense device was replying to packets with a different IP address on the IMAPS/HTTPS server than the client device was requesting them from.  I couldn't get the NAT reflection to work as I was expecting, and the split DNS solution was the most simple/elegant way to resolve it in my case.

Would strongly recommend tcpdump on the client and server - see what arrives, see what replies, and with what IPs, it may provide a clue.

17

22.7 Legacy Series / Re: AdGuard not updating

« on: October 05, 2022, 11:19:55 pm »

Solution picked from Adguard bug thread:


- SSH in your opnsense box
- Send this command, change USERNAME:PASSWORD to your Login/Pass of Adguard
- Specify port if you changed the default
 

Code: [Select]

curl -H 'Content-Type: application/json' -X POST -v 'http://USERNAME:PASSWORD@127.0.0.1/control/update'

Can confirm this worked - but, in my case, I had to use the IP address of my AGH box on my LAN, localhost didn't work.

Updated to .b17 successfully, no apparent loss of data.  Thank you!

18

Virtual private networks / Re: Permitting a host to use two different WireGuard tunnels

« on: September 27, 2022, 03:52:16 am »

Next up - load balancing across two simultaneous WireGuard links to Mullvad!

Mission accomplished.

I may write up a Tutorial on this, if there is interest.

I have now configured my OPNsense device to have two different VPN links running, with full load balancing between them. 

In my case I'm using Mullvad, but, I'll likely throw in another VPN provider I use.

It was relatively simple to sort out, once I wrapped my head around both the Multi-WAN and Multi-VPN concepts (NAT, routing rules, gateways).

19

22.7 Legacy Series / Re: Revoked certificate after upgrading to 22.7.4

« on: September 26, 2022, 08:59:26 am »

Hi
it is certainly better to wait for @franco, but if I understand the code correctly, the signature file is downloaded only if the checksum of the changelog.txz has changed (that is, if at some point the missigned changelog file was already downloaded, then after creating a new signature file it may not be downloaded if the checksum of the changelog.txz has not changed). you can try deleting changelog.txz and  changelog.txz.sig files  in /usr/local/opnsense/changelog dir and try checking for updates again

For what it is worth, this worked for me.

In my case, I blamed the fact that I sidegraded from the Business Edition to the Community Edition -- but that may not have been the reason for the error.  I made a backup of the changelog files just in case, then removed them, tried updates, and no longer have the error.

20

Virtual private networks / Re: Permitting a host to use two different WireGuard tunnels

« on: September 24, 2022, 06:58:09 am »

Thanks for the reminder, Bob.Dig - it was actually enough to help me solve this by thinking about the problem differently.

I was trying to solve it using only the tips in the Guide for selective routing on WireGuard/Mullvad, and, as you rightfully point out, simply looking at my problem as a "rules" problem helped.

So, the solution, for anyone who looks into this in the future, was for me to create an Alias for Networks which represented the networks in my wg2 AllowedIps.  Sadly, this does mean that if I change the IPs I have to change it in two places, but, that's not a common scenario.

Once the Alias existed, I made a new Firewall -> Rules -> LAN rule which permitted with a Quick Pass on In traffic from the "permitted" Alias to reach the new NetworkAlias which represents the AllowedIPs from the wg2.

As this rule is high enough in the list, it's matched first, and traffic goes out wg2 as needed.

For anything else, the rules processing proceeds to the wg1 interface for Mullvad, and everything works as I wanted.

Next up - load balancing across two simultaneous WireGuard links to Mullvad!

21

Virtual private networks / Permitting a host to use two different WireGuard tunnels

« on: September 23, 2022, 08:34:58 am »

Hi,

I have several WG interfaces defined on my OPNsense box.  One (wg1) is for Mullvad VPN, with 0.0.0.0/0 as the AllowedIPs.

On another WG interface (wg2), I have a series of IP addresses or networks (100+) defined as the Allowed IPs.

I would like a specific host on my LAN to be able to use either, depending upon what the destination is.

In other words:  If the destination network IP address is one of the AllowedIPs on wg2, send the traffic to that WG gateway.  If it is NOT one of those IPs, then, use the Mullvad VPN on wg1.

I can get either situation to work separately, but, not both simultaneously.

So, if I put the host in the Alias for the wg2 network, and only in that Alias, traffic will go out wg2 for those AllowedIPs, and out my WAN for anything else.  If I put it in the wg1 permitted alias, it goes out Mullvad just fine, but, can't reach any of the hosts in the AllowedIPs for wg2.

I've followed the guide at: https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html

And that's helped considerably, but, I'm missing something in creating the Firewall rules (probably) that would permit this.  I may also be mucking up the destination network definition of RFC1918 addresses as per that guide.

I have included ONLY my LAN IP network in that definition, as some of the IP address that are behind wg2 are in other RFC1918 space, so it seemed unwise to include those.  I also wasn't sure if I should include local WireGuard IP addresses or networks in this excluded RFC1918 alias definition.

My reading of the point of Step 8 was to "permit inbound traffic to the LAN interface from the hosts in the defined alias and NOT intended for the local RFC1918 used networks to go out through the defined WireGuard gateway".

I'm probably just getting a bit blind to the Interface and Floating rules, NAT outbound rules, and Aliases, to make this all work, but, would appreciate any insight into the best way to accomplish my goal.

Thank you!

22

Virtual private networks / Re: wireguard with shadowsocks

« on: September 19, 2022, 03:28:22 am »

Not a direct answer to your question, but most likely the success of this idea would depend upon the device your son is using and if it can run shadowsocks.

I have found (messy) solutions to this type of issue myself by using port forwarding invoked via apps like Termius on iOS -- works great for TCP, but, UDP requires more thought, and, as you probably know, WireGuard is UDP based.

There are solutions like "socat" which can (and does) work nicely, but, again, whether or not your son can run that locally on his device to tunnel the WireGuard UDP inside of a TCP tunnel back to your shadowsocks (or other) solution is more complicated.

Have you checked to see if it's all UDP that's blocked, or only certain UDP ports, from his school?

May not be a trivial one to solve without having extra software on his device. 

23

22.7 Legacy Series / Re: Yet another post about IPv6 connectivity loss after upgrading OPNsense

« on: September 19, 2022, 03:06:03 am »

I've only started using OPNsense in the last month or two, so, I do not have the historical experience that you do.

However, I experienced exactly the same symptoms that you describe.  I have a native IPv6 provider, and, previous network equipment all worked flawlessly.  When I started with the Deciso OPNsense appliance in-line on my fibre connection to The World, IPv4 worked, but, IPv6 failed miserably.

The first symptom of this was in getting OPNsense updated itself.  Updates were crawling, and there were hundreds to perform.  Eventually I found a suggestion in this forum about ticking the "prefer IPv4 even if IPv6 is available" option, enabled it, and within a few minutes everything wooshed along.

In my case, the solution for devices in my LAN was to make some tweaks to the tunables in OPNsense.

In specific, I had to:

• Set net.inet6.icmp6.nd6_onlink_ns_rfc4861 to "1"
• Set net.inet6.ip6.accept_rtadv to "0"

Ironically, this solution came from a pfSense forum, and was called out as recommended for my specific provider.

Once I did this, and restarted, IPv6 was working correctly and natively on my LAN/WAN setup.  I have not yet unticked the "prefer IPv4" box yet, but hope to in the next few days.

I don't think this will necessarily help you, but my symptoms were exactly the same as yours until I made these changes.

24

General Discussion / Re: NAT Reflection - Was working, not sure why it stopped.

« on: September 19, 2022, 02:57:59 am »

I am still trying to figure out why NAT reflection stopped working... but for now I've implemented a split DNS that is a workaround for the issue.

For what it is worth, I recently ran into a similar issue, and, this, too, was my solution.  I don't have a previous build of OPNsense to know if this was a recent change or not - but the split DNS was the only viable solution that seemed to avoid the problem!

25

General Discussion / Re: Dealing with port forwarding for a laptop which may be WAN or LAN based.

« on: September 18, 2022, 01:09:10 am »

Solved - sort of.

Gave up trying to get this to work as it used to on an ASUS home router; likely part of how FreeBSD does things under the bonnet is more proscriptive, in a good way, and I'm fine with that.

My solution was to resort to a split DNS solution so that local hosts resolve the LAN IP of the IMAP server, but, external will stick to public WAN IP.

26

General Discussion / Dealing with port forwarding for a laptop which may be WAN or LAN based.

« on: September 17, 2022, 08:36:59 am »

Hi,

I run an IMAP server on my LAN which I've got NAT -> port forwarding set up for on the WAN interface. Works fine, can reach the appropriate internal resource on the correct port as expected, whenever I'm attached to another network).

However, when I use my laptop on the LAN, my OPNsense is rewriting the response packets to the WAN IP to be from the LAN IP of the IMAP server.

This means that the laptop is trying to connect to 1.2.3.4:9003, but, the response packets are coming from 192.168.50.9:993.

This, rightfully, causes the LAN IPd laptop to reset the connection and try again.

To try to address this, I changed my NAT -> Port Forwarding rule so that it does NOT apply to the LAN Net (source invert).

Again, this continues to work just fine from outside of the network itself.  But when I'm attached to the LAN, it means that the packets to the WAN IP for the IMAP server are being completely dropped.

What is the right way to solve this?  I want my IMAP client to always be configured for the hostname of the WAN IP address, and the port I use (9003 in this example). 

I'm just not sure what forwarding/rules I need such that the OPNsense device will receive the traffic on it's LAN IP, with a destination of the WAN IP, and perform the port forwarding to the internal IP address and port, but NOT re-write the IP address of the IMAP server to be the LAN IP that it has.

I've tried just about everything I can think of for forwarding or rules, inbound and outbound, but, can't make this work when the laptop is on the same network as the IMAP server. 

Edit: I've tried with NAT reflection both enabled and disabled, doesn't seem to impact the situation I have.
Thanks!

27

Virtual private networks / WireGuard - AllowedIP limit on OPNsense?

« on: September 16, 2022, 03:03:27 am »

I have a need to have around 160 subnets defined in AllowedIPs for a WireGuard endpoint - this works just fine on a separate dedicated WireGuard device I have on my network, but, on the OPNsense, at some point in adding subnets to the list, the wg interface never comes up.

Whilst I can see the wg3.conf interface in /usr/local/etc/wireguard on the OPNsense system, the file empties out if I disable WG - so I'm not really sure where it's storing the data BEFORE it goes into this file (something that the UI is generating?).

I also tried modifying wg3.conf myself, and restarting WireGuard, but, the problem persists.

I'll try iteratively adding subnets till I find the breaking point, but that could take a bit. 

Anyone encountered this?

Is there another/better way to accomplish the same goal, such as by using Gateways and Routes (and maybe NAT?).

I was definitely able to add a few extra AllowedIPs to the list, and it worked fine, but, after a few, the interface never comes up and WireGuard never establishes a handshake.  Since I know WG itself can handle this, I'm guessing there's a glitch in processing the number of them that I need to process.

Open to other suggestions on how best to solve this!

28

General Discussion / Re: Firewall LAN rules not working as expected (Lan -> Lan blocked)

« on: September 13, 2022, 05:41:06 am »

There's nothing like staring at a problem and changing something that you think "this could NOT possibly have ANY impact to the problem" - but it does.

Fixed.

WHY it broke is still unclear to me, but the fix was for me to move the 50.2 server to the same switch that is on my LAN port of my OPNsense device.

It was attached to a different switch, which is actually my WiFi AP device. 

So the traffic WAS:

iPhone -> AP -> Server (physically attached to AP)

And is now:

iPhone -> AP -> Switch -> Server

I'm guessing that the Asus WiFi device in AP mode is doing something funky that is confusing the OPNsense firewall as to where the packet originated from.

29

General Discussion / Firewall LAN rules not working as expected (Lan -> Lan blocked)

« on: September 13, 2022, 01:21:42 am »

Hello all,

I've done as much debugging on this one as I can work out, so, would appreciate suggestions.

I have a LAN network defined as 192.168.50.0/23 (with the LAN IP being 192.168.50.1).

I've set up the DHCPv4 server so that 51.10 through 51.200 are for DHCP leases.

I have a static IP on a local Raspberry Pi on 50.2, and the https port on the Pi is reachable from anything in the 50.0/24 part of the /23, but, nothing in the DHCP lease pool can reach the Pi.

The Pi's eth0 is correct:

Code: [Select]

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.50.2  netmask 255.255.254.0  broadcast 192.168.51.255

Most of the devices in the 51.0/24 part of the network are WiFi, and, I've made sure the issue is not the WiFi access point; I've also set a WiFI device to have a static IP of 192.168.50.12 and using WiFi that can reach the 50.2 host no problem.  It's only when the DHCPv4 lease IP of 51.x is used that the Default Deny rule of the firewall in OPNsense kicks in.

I can see the packets logged as such:

LAN      2022-09-13T09:08:35   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:34   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:33   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:33   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:32   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:31   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule

And if you look at the details of one of the rules, it says:

__timestamp__   2022-09-13T09:08:35
ack   2551861249
action    [block]
anchorname   
datalen   0
dir    [in]
dst   192.168.51.10
dstport   53027
ecn   
id   0
interface   igb0
interface_name   LAN
ipflags   DF
ipversion   4
label   Default deny / state violation rule
length   60
offset   0
protoname   tcp
protonum   6
reason   match
rid   02f4bab031b57d1e30553ce08e0ec131
rulenr   14
seq   3121781003
src   192.168.50.2
srcport   443
subrulenr   
tcpflags   SAE
tcpopts   
tos   0x0
ttl   64
urp   65160

And the default rules which are in Firewall -> Rules -> LAN include:

Code: [Select]

 
IPv4 * LAN net * * * * * Default allow LAN to any rule    
IPv6 * LAN net * * * * * Default allow LAN IPv6 to any rule

Those rules are Pass, and In.

I have also tried adding rules which were IPv4+IPv6 in pass for any/any source destination, not specifically the LAN net, and, yes, I've also tried rules for OUT which were any/any.

The SYN packets are making it to the Pi, and the Pi responds:

Code: [Select]

09:14:50.815803 IP 192.168.51.10.53036 > 192.168.50.2.443: Flags [SEW], seq 2284235656, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 2341413872 ecr 0,sackOK,eol], length 0
09:14:50.815934 IP 192.168.50.2.443 > 192.168.51.10.53036: Flags [S.E], seq 2610416432, ack 2284235657, win 65160, options [mss 1460,sackOK,TS val 1298482186 ecr 2341413872,nop,wscale 7], length 0
09:14:51.818903 IP 192.168.50.2.443 > 192.168.51.10.53036: Flags [S.E], seq 2610416432, ack 2284235657, win 65160, options [mss 1460,sackOK,TS val 1298483189 ecr 2341413872,nop,wscale 7], length 0
But the firewall is blocking the returning packets from the Pi to the 51.10 IP (in this case, an iPhone).

Curiously, I CAN reach other 50.0/24 hosts from the iPhone, which is puzzling.

And anything at all with 50.0/24 can reach 50.2 - it's purely running into that default deny rule for packets returning to 51.0/24 within the overall /23.

And the LAN network IP is correct as well:

Code: [Select]

igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: LAN
options=4800028<VLAN_MTU,JUMBO_MTU,NOMAP>
inet 192.168.50.1 netmask 0xfffffe00 broadcast 192.168.51.255

Any obvious thoughts as to what I'm overlooking for this incredibly simple and stupid question? 

30

Virtual private networks / WireGuard - had to create outbound pass rule for DNS in Road Warrior setup

« on: September 12, 2022, 09:58:08 am »

I set up WireGuard on my OPNsense appliance (Deciso), and whilst I could see the firewall rule triggering for the inbound traffic, I could never get a successful handshake or traffic to route.

After trying many things for way longer than I should have, the "fix" was for me to set an outbound "pass" firewall rule for both In and Out in the WG_Mobile definition.  The guides only talk about "In".

Yes, I have created/assigned an Interface, so did not create a NAT rule (as per the various guides and their advice on this).

tcpdump on wg1 showed the packets coming in, DNS queries heading to my OPNsense IP address, and the responses coming back, but, the WireGuard client on iOS wouldn't work unless I either set this outbound rule to permit the responses to go back to the Peer.

Is this expected behaviour?  https://docs.opnsense.org/manual/how-tos/wireguard-client.html implies not.

It would behave as expected if I told the iOS WG client to use 1.1.1.1 as the DNS resolver, but if I used any of the IPs on the OPNsense box, the packets would never make it back to the iOS device.

Perhaps I have been staring at the problem for too long and am missing something obvious?