Topics

1

23.1 Production Series / [SOLVED] ACME plugin with Gandi LiveDNS

« on: April 10, 2023, 06:16:52 am »

Prior to 23.1, the ACME plugin seemed to work fine, and I had automatically renewed certificates for several months.

Somewhere around the change to 23.1, however, it no longer works via OPNSense, even though I can use Gandi's LiveDNS and API key from "letsencrypt" on a Pi just fine (so the issue is not Gandi, and not the API key).

My logs appear as such (with debug logging enabled for the ACME Settings):

Code: [Select]

2023-04-10T14:02:33 Error   opnsense    AcmeClient: validation for certificate failed: host.mydomain.com   
2023-04-10T14:02:33 Error   opnsense    AcmeClient: domain validation failed (dns01)   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: running acme.sh command: /usr/local/sbin/acme.sh --issue --syslog 7 --debug --server 'letsencrypt' --dns 'dns_gandi_livedns' --dnssleep '90' --home '/var/etc/acme-client/home' --certpath '/var/etc/acme-client/certs/whatever.07307279/cert.pem' --keypath '/var/etc/acme-client/keys/whatever.07307279/private.key' --capath '/var/etc/acme-client/certs/whatever.07307279/chain.pem' --fullchainpath '/var/etc/acme-client/certs/whatever.07307279/fullchain.pem' --domain 'host.mydomain.com' --days '1' --force --keylength '4096' --accountconf '/var/etc/acme-client/accounts/whatever.40506586_prod/account.conf'   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: using challenge type: GandiV5   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: account is registered: Let's Encrypt   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: using CA: letsencrypt   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: issue certificate: host.mydomain.com   
2023-04-10T14:02:25 Notice  opnsense    AcmeClient: certificate must be issued/renewed: host.mydomain.com
Obviously, this is in reverse chronological order.

I've obfuscated a few things, but, I do not think they are relevant to the issue.  The domain has the Gandi API enabled, the key works fine, etc etc.

What I do notice, however, is that the "dnssleep" option passed to the ACME shell script is being ignored.  I've tried various values here, 120 seconds, 240, 0 (default) - however, as you can see from the logs, within 2 seconds OPNSense records the attempt as a failure, and gives up.

Interestingly, even with "0" set as the value, the OPNSense plugin does not seem to re-try as per the on-screen note of:

The time in seconds to wait for all the TXT records to take effect after adding them to the DNS API. Defaults to 0 seconds, which causes Acme Client to check public DNS services every 10 seconds for up to 20 minutes. If set to a non-zero value, a fixed DNS sleep time will be used and the local DNS servers will be queried instead. A DNS sleep time of 120 seconds or more is recommended for some DNS APIs.

Does anyone have ACME working with 23.1 series and Gandi LiveDNS?

2

General Discussion / Static IPv6 for iOS devices in OPNsense network

« on: October 21, 2022, 04:39:46 am »

I realise there may be no answer to this, but, I have a few iOS devices on the LAN side of my OPNsense controlled network that I'd like to have a static IPv6 address for. 

I want this because I have some policy based routing that I'd like to apply to both IPv4 and IPv6. 

My rules all work fine, but, iOS devices tend to rotate IPv6 addresses, which means I frequently have to update the rules for the NEW IPv6 address.

Assuming it's even possible (it may not be), what's the best/right technology to use on my OPNsense box to help?

3

Virtual private networks / Permitting a host to use two different WireGuard tunnels

« on: September 23, 2022, 08:34:58 am »

Hi,

I have several WG interfaces defined on my OPNsense box.  One (wg1) is for Mullvad VPN, with 0.0.0.0/0 as the AllowedIPs.

On another WG interface (wg2), I have a series of IP addresses or networks (100+) defined as the Allowed IPs.

I would like a specific host on my LAN to be able to use either, depending upon what the destination is.

In other words:  If the destination network IP address is one of the AllowedIPs on wg2, send the traffic to that WG gateway.  If it is NOT one of those IPs, then, use the Mullvad VPN on wg1.

I can get either situation to work separately, but, not both simultaneously.

So, if I put the host in the Alias for the wg2 network, and only in that Alias, traffic will go out wg2 for those AllowedIPs, and out my WAN for anything else.  If I put it in the wg1 permitted alias, it goes out Mullvad just fine, but, can't reach any of the hosts in the AllowedIPs for wg2.

I've followed the guide at: https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html

And that's helped considerably, but, I'm missing something in creating the Firewall rules (probably) that would permit this.  I may also be mucking up the destination network definition of RFC1918 addresses as per that guide.

I have included ONLY my LAN IP network in that definition, as some of the IP address that are behind wg2 are in other RFC1918 space, so it seemed unwise to include those.  I also wasn't sure if I should include local WireGuard IP addresses or networks in this excluded RFC1918 alias definition.

My reading of the point of Step 8 was to "permit inbound traffic to the LAN interface from the hosts in the defined alias and NOT intended for the local RFC1918 used networks to go out through the defined WireGuard gateway".

I'm probably just getting a bit blind to the Interface and Floating rules, NAT outbound rules, and Aliases, to make this all work, but, would appreciate any insight into the best way to accomplish my goal.

Thank you!

4

General Discussion / Dealing with port forwarding for a laptop which may be WAN or LAN based.

« on: September 17, 2022, 08:36:59 am »

Hi,

I run an IMAP server on my LAN which I've got NAT -> port forwarding set up for on the WAN interface. Works fine, can reach the appropriate internal resource on the correct port as expected, whenever I'm attached to another network).

However, when I use my laptop on the LAN, my OPNsense is rewriting the response packets to the WAN IP to be from the LAN IP of the IMAP server.

This means that the laptop is trying to connect to 1.2.3.4:9003, but, the response packets are coming from 192.168.50.9:993.

This, rightfully, causes the LAN IPd laptop to reset the connection and try again.

To try to address this, I changed my NAT -> Port Forwarding rule so that it does NOT apply to the LAN Net (source invert).

Again, this continues to work just fine from outside of the network itself.  But when I'm attached to the LAN, it means that the packets to the WAN IP for the IMAP server are being completely dropped.

What is the right way to solve this?  I want my IMAP client to always be configured for the hostname of the WAN IP address, and the port I use (9003 in this example). 

I'm just not sure what forwarding/rules I need such that the OPNsense device will receive the traffic on it's LAN IP, with a destination of the WAN IP, and perform the port forwarding to the internal IP address and port, but NOT re-write the IP address of the IMAP server to be the LAN IP that it has.

I've tried just about everything I can think of for forwarding or rules, inbound and outbound, but, can't make this work when the laptop is on the same network as the IMAP server. 

Edit: I've tried with NAT reflection both enabled and disabled, doesn't seem to impact the situation I have.
Thanks!

5

Virtual private networks / WireGuard - AllowedIP limit on OPNsense?

« on: September 16, 2022, 03:03:27 am »

I have a need to have around 160 subnets defined in AllowedIPs for a WireGuard endpoint - this works just fine on a separate dedicated WireGuard device I have on my network, but, on the OPNsense, at some point in adding subnets to the list, the wg interface never comes up.

Whilst I can see the wg3.conf interface in /usr/local/etc/wireguard on the OPNsense system, the file empties out if I disable WG - so I'm not really sure where it's storing the data BEFORE it goes into this file (something that the UI is generating?).

I also tried modifying wg3.conf myself, and restarting WireGuard, but, the problem persists.

I'll try iteratively adding subnets till I find the breaking point, but that could take a bit. 

Anyone encountered this?

Is there another/better way to accomplish the same goal, such as by using Gateways and Routes (and maybe NAT?).

I was definitely able to add a few extra AllowedIPs to the list, and it worked fine, but, after a few, the interface never comes up and WireGuard never establishes a handshake.  Since I know WG itself can handle this, I'm guessing there's a glitch in processing the number of them that I need to process.

Open to other suggestions on how best to solve this!

6

General Discussion / Firewall LAN rules not working as expected (Lan -> Lan blocked)

« on: September 13, 2022, 01:21:42 am »

Hello all,

I've done as much debugging on this one as I can work out, so, would appreciate suggestions.

I have a LAN network defined as 192.168.50.0/23 (with the LAN IP being 192.168.50.1).

I've set up the DHCPv4 server so that 51.10 through 51.200 are for DHCP leases.

I have a static IP on a local Raspberry Pi on 50.2, and the https port on the Pi is reachable from anything in the 50.0/24 part of the /23, but, nothing in the DHCP lease pool can reach the Pi.

The Pi's eth0 is correct:

Code: [Select]

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.50.2  netmask 255.255.254.0  broadcast 192.168.51.255

Most of the devices in the 51.0/24 part of the network are WiFi, and, I've made sure the issue is not the WiFi access point; I've also set a WiFI device to have a static IP of 192.168.50.12 and using WiFi that can reach the 50.2 host no problem.  It's only when the DHCPv4 lease IP of 51.x is used that the Default Deny rule of the firewall in OPNsense kicks in.

I can see the packets logged as such:

LAN      2022-09-13T09:08:35   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:34   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:33   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:33   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:32   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule   
LAN      2022-09-13T09:08:31   192.168.50.2:443   192.168.51.10:53027   tcp   Default deny / state violation rule

And if you look at the details of one of the rules, it says:

__timestamp__   2022-09-13T09:08:35
ack   2551861249
action    [block]
anchorname   
datalen   0
dir    [in]
dst   192.168.51.10
dstport   53027
ecn   
id   0
interface   igb0
interface_name   LAN
ipflags   DF
ipversion   4
label   Default deny / state violation rule
length   60
offset   0
protoname   tcp
protonum   6
reason   match
rid   02f4bab031b57d1e30553ce08e0ec131
rulenr   14
seq   3121781003
src   192.168.50.2
srcport   443
subrulenr   
tcpflags   SAE
tcpopts   
tos   0x0
ttl   64
urp   65160

And the default rules which are in Firewall -> Rules -> LAN include:

Code: [Select]

 
IPv4 * LAN net * * * * * Default allow LAN to any rule    
IPv6 * LAN net * * * * * Default allow LAN IPv6 to any rule

Those rules are Pass, and In.

I have also tried adding rules which were IPv4+IPv6 in pass for any/any source destination, not specifically the LAN net, and, yes, I've also tried rules for OUT which were any/any.

The SYN packets are making it to the Pi, and the Pi responds:

Code: [Select]

09:14:50.815803 IP 192.168.51.10.53036 > 192.168.50.2.443: Flags [SEW], seq 2284235656, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 2341413872 ecr 0,sackOK,eol], length 0
09:14:50.815934 IP 192.168.50.2.443 > 192.168.51.10.53036: Flags [S.E], seq 2610416432, ack 2284235657, win 65160, options [mss 1460,sackOK,TS val 1298482186 ecr 2341413872,nop,wscale 7], length 0
09:14:51.818903 IP 192.168.50.2.443 > 192.168.51.10.53036: Flags [S.E], seq 2610416432, ack 2284235657, win 65160, options [mss 1460,sackOK,TS val 1298483189 ecr 2341413872,nop,wscale 7], length 0
But the firewall is blocking the returning packets from the Pi to the 51.10 IP (in this case, an iPhone).

Curiously, I CAN reach other 50.0/24 hosts from the iPhone, which is puzzling.

And anything at all with 50.0/24 can reach 50.2 - it's purely running into that default deny rule for packets returning to 51.0/24 within the overall /23.

And the LAN network IP is correct as well:

Code: [Select]

igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: LAN
options=4800028<VLAN_MTU,JUMBO_MTU,NOMAP>
inet 192.168.50.1 netmask 0xfffffe00 broadcast 192.168.51.255

Any obvious thoughts as to what I'm overlooking for this incredibly simple and stupid question? 

7

Virtual private networks / WireGuard - had to create outbound pass rule for DNS in Road Warrior setup

« on: September 12, 2022, 09:58:08 am »

I set up WireGuard on my OPNsense appliance (Deciso), and whilst I could see the firewall rule triggering for the inbound traffic, I could never get a successful handshake or traffic to route.

After trying many things for way longer than I should have, the "fix" was for me to set an outbound "pass" firewall rule for both In and Out in the WG_Mobile definition.  The guides only talk about "In".

Yes, I have created/assigned an Interface, so did not create a NAT rule (as per the various guides and their advice on this).

tcpdump on wg1 showed the packets coming in, DNS queries heading to my OPNsense IP address, and the responses coming back, but, the WireGuard client on iOS wouldn't work unless I either set this outbound rule to permit the responses to go back to the Peer.

Is this expected behaviour?  https://docs.opnsense.org/manual/how-tos/wireguard-client.html implies not.

It would behave as expected if I told the iOS WG client to use 1.1.1.1 as the DNS resolver, but if I used any of the IPs on the OPNsense box, the packets would never make it back to the iOS device.

Perhaps I have been staring at the problem for too long and am missing something obvious?

8

General Discussion / Port forwarding sanity check (blaming my VM setup)

« on: September 09, 2022, 05:06:35 am »

I set up a port forwarding test ahead of my migration to an OPNsense appliance next week, and made it to about 90% success - and I think the reason for my failure was my specific VM setup.  May not be anyone here who can confirm that, but, thought I'd try.

Issue:  tcpdump confirms packets arriving at intended destination on LAN from WAN, but, connections never get established (test was performed using netcat).

Setup:

• macOS Host
• Parallels virtualisation software
• OPNsense with WAN as Shared Network interface from Parallels, IP of 10.211.55.48
• OPNsense with LAN as static IP of 192.168.1.1/24
• Linux VM with static IP of 192.168.1.10/24 and gateway of 192.168.1.1

With this setup, and ensuring that the WAN interface permits RFC1918 and bogon traffic (disabled both tick boxes), the Linux VM correctly routes via the OPNsense LAN, can reach the Internet, and all is good.  I can manage the OPNsense from the Linux VM to the LAN IP of the OPNsense VM.

I set up Port Forwarding to permit IPv4, tcp, listening on WAN, for traffic going to the WAN address, and setting a destination IP of 192.168.1.10, listening for incoming port 9999 and sending it to 9999 on the Linux VM.  "Add associated rule" was also enabled.

I can see the rule, and the corresponding rule set up on the WAN interface in Firewall -> Rules. (Add associated rule did this)

When I use netcat on the Linux VM and listen on port 9999, I can reach it from Netcat on the OPNsense VM itself.

However, when I try to use my macOS Host, which has the IP of 10.211.55.2, I can see the packets arrive on 192.168.1.10:9999, but, netcat never connects.  I also see the NAT port forwarding rule in the firewall logs.  Nothing is being denied.

My assumption is that my weird setup is why the netcat from the macOS host never properly connects.  The packets, and responses, are absolutely visible on the Linux VM, but, something doesn't "click".

Probably not a lot of Parallels users here, and from what I can gather from various tutorials online, I've done everything right. 

Thanks for pointing out anything glaringly dumb in the above.

9

General Discussion / Planning for a migration to OPNsense (appliance)

« on: September 06, 2022, 01:02:13 am »

Good time of day all,

I recently took the plunge into OPNsense in a large way - bought one of the Deciso appliances which is en route to me now. 

I don't have a background in pfSense or m0n0wall, but, have worked with networks, firewalls, routing and the Internet for multiple decades (yeah, greybeard here).

As I'm sure most of you do, I like to plan out network changes as carefully as I can, and whilst I spent several days becoming familiar with OPNsense, plugins, configuration, this forum, and so on (software testing was via Parallels on my Mac), I realise that ACTUAL migration to the OPNsense appliance will come with a few more challenges.

The initial network that I'm testing this on is just going to be my home (rather I suffer than an employer suffer).  As I was documenting the various changes and tweaks I know I'll need to make, it occurred to me that since the Deciso appliance is headless, setting it up and configuring it may or may not be straightforward.

As I understand it, the default configuration will be that interface 0 (as labelled on the front panel of the DEC800 series) will be the WAN port, and interface 1 will be LAN.  Also, from what I gather, interface 1 will be 192.168.1.1/24.

I will need to make changes to the network addresses and mask to fit in with my home setup, but, I don't want to put the appliance "in line" to my gigabit WAN Internet service until I've got at least the basics set up right.

I'd appreciate any input from users of the Deciso appliances in particular about how they recommend initial setup.  I am sure I can dig up a USB C -> USB A -> USB Mini B -> Console port cable if needed, but, it was my understanding that whilst this would be useful for booting information, without explicitly enabling the serial console in OPNsense, I may not be able to use this for initial configuration.

No doubt I'm making some false assumptions, and there may even be documentation that will come with the unit when it arrives next week that makes this all much more obvious, but, I still like to plan through these things as much as possible in advance.

So if anyone has any advice on those very first initial steps with a headless appliance like the DEC850/DEC840, I'd really appreciate them!

My priority is simply getting the device to be accessible on my LAN so that I can configure it for the right number plan for existing devices.

My second priority is making sure that LAN to WAN connectivity works.  I've read through many posts on the forum and have conflicting views as to how automatic or manual this will be.  Since this is a residential ISP, and I have a static /56 for IPv6 and /32 for IPv4, my intention was actually to hope that the DHCP (and DHCPv6) on WAN would obtain the right details from my provider (as currently happens with my ASUS home router).  From there, I'm less clear.

Will I have to manually enable any sort of NAT from LAN to WAN?  Any firewall rules?

Essentially, I want to OPNsense to act like a home router whilst I become more familiar with it in situ, rather than a "lab" on virtual machines.

In my experiments on virtual machines, I did an excellent job of fubaring up the configuration (due to the vagaries of virtual machine "host only/shared/interface bound" interface configs) that more than once I blew away the VM and started over.  Trying to avoid that with the appliance!

Realise these sorts of newbie questions are often annoying to experts, but, if there is any input people could provide, I'd be grateful.  I've spent way too much time going down YouTube video rabbit holes on this topic as well, but still am not clear on whether or not I'll have to muck about with NAT and/or Firewall rules to Get Going.  Just want to have a sense of what I need to read or learn about to make my Deciso appliance act like a very simple home router as quickly as possible. 

Thank you!