Messages

1

23.7 Legacy Series / Re: TCP connections fail after ISP IP renewal

« on: October 01, 2023, 03:52:11 pm »

Actually, there is no modem. I have FTTH, terminating in an SFP module in my servers NIC.

Referring to CJs post: In the meantime I updated to OPNsense 23.7.5. I also rebooted my server due to BIOS and Proxmox OS updates. This lead to new IP addresses from ISP but all services remained reachable after IP change. DNS was also updated correctly by Dynamic DNS.

I have external monitoring up and running. As soon as I lose connection to my website again, I will dive into the logs and post them here.

2

23.7 Legacy Series / Re: TCP connections fail after ISP IP renewal

« on: September 30, 2023, 03:44:12 pm »

I have logging turned on for all nat rules. It can only be a matter of days till the next IP change.

Opnsense is a VM on a Proxmox host. NextCloud and Proxy are LXC containers.

Regards and I keep my fingers crossed ISP is changing the IP soon 

3

23.7 Legacy Series / Re: TCP connections fail after ISP IP renewal

« on: September 29, 2023, 04:22:35 pm »

Hi, thank you again for your patience and effort. I will first answer the new questions and afterwards give a detailed description of my hosting setup.

Some terminology clarity would be useful.
- "But all requests to my piblic IP based on TCP are dropped" . Are they being dropped by the firewall and can be seen from the OPN side?

To be verified as soon as the issue occurs again.

Or do you mean connections from another network (mobile phone for instance) time out, get rejected, what?

They time out.

And are they done to the new ip or to an url?

Both. The URL (resolving the correct IP via DNS) leads to a timeout as does the connection using the new IP.

- "Or at least my self hosted websites cant be resolved.". Which one is it, dns is updated or not?

https://nextcloud.baltic-hosting.de DNS is updated correctly

Done an external dns query? Resolves or not to the new ip?

yes, using dnschecker.org. It resolves to the new IP.

These websites, how are they being served through OPN, haproxy, port forwarding, nginx, etc?

I use port forwarding to my nginx proxy, running in a VLAN. The webserver hosting the service is in a different VLAN.

My hosting setup:

NextCloud | Proxy | Port Forwarding | Internet

The webserver, hosting the NextCloud service, is in VLAN "Management". There are no restrictive rules in this VLAN. Connections are allowed any to any.

The Proxy (Nginx) is in the VLAN "DMZ" with allow rules to reach my NextCloud webserver and the other services I am hosting. All other local addresses are denied. Internet access is available for updates.

Port forwarding is used to expose the Proxy and the necessary ports (80, 443).
Port forwarding is also exposing the Wireguard VPN Port to the internet. I am using the Wireguard plugin of OPNsense.
Please find screenshots of the settings attached.

4

23.7 Legacy Series / Re: TCP connections fail after ISP IP renewal

« on: September 27, 2023, 02:02:24 pm »

Thank you for your feedback. Please find all questionmarks and my answers below.

ISP lease expires and provides new lease.  WAN interface reports 198.51.100.15.  DNS lookup of dyn.example.com resolves to 198.51.100.15?

- yes. Tested with external service dnschecker.org, if the IP is resolved correctly.

Wireguard existing tunnel continues working?

no. The tunnel is terminated after ISP renewal.

Wireguard new connection works?

yes. peer to server connection. no site to site. WG connects to the new IP.

Local webserver receives no incoming connections?

correct. also using the new wan IP does not reach the web server.

OPNSense forwarding rule receives no incoming connections?

to be verified as soon as the issue occures again (mostly every couple of days)

I doubt that suddenly, OpnSense chooses to deny new TCP connections unless your firewall rules are weird.

me too, and thats why I am asking for help here. I need help to debug this. Again, reloading the interface untill I get a new IP, solves the issue without changes to any of my other settings.

After all, each incoming connection has to be a new one after a WAN IP renewal - all old connections must have been dropped already.

I am aware of that.

5

23.7 Legacy Series / Re: TCP connections fail after ISP IP renewal

« on: September 27, 2023, 08:45:40 am »

First of all, thank you for your time and effort.

I will try to clarify as good as possible:

Initial working situation:
WAN interface reports 198.51.100.12.  DNS lookup of dyn.example.com resolves to 198.51.100.12.  Wireguard connects and local webserver works correctly from externally.

correct, this is the working situation.

ISP lease expires and provides new lease.  WAN interface reports 198.51.100.15.  DNS lookup of dyn.example.com resolves to 198.51.100.15?  Wireguard existing tunnel continues working?  Wireguard new connection works?  Local webserver receives no incoming connections?  OPNSense forwarding rule receives no incoming connections?

Correct. Wireguard stops working at renewal and is working a couple of minutes later, when DNS is updated. Which is the expected behavior. Connections to my webserver are refused.
I will need to look into the connections, when the issue occures again. If I recall correctly, opnsense logged incoming connections but I need to verify this.

I am aware of the deprecation of ddclient, I had an issue with the OPNsense backend to not detect an IP change. It's in my backlog to look into this.

6

23.7 Legacy Series / Re: TCP connections fail after ISP IP renewal

« on: September 26, 2023, 08:52:43 am »

I use the ddclient plugin, it checks for a changed IP on the WAN interface every 5min. I checked the DNS resolving and wireguard via mobile broadband on my smartphone and from a different location.

Yes, UWP works. Wireguard VPN is not affected by the ISP side IP change. My website does not resolve though. Also, the DNS system points to the correct IP after changed by ISP. So dynamic dns works, too.

I reload the PPPoE Interface. Normally 3 times reload and the ISP provides a new IP. Afterwards all is fine and my website is loading.

7

23.7 Legacy Series / TCP connections fail after ISP IP renewal

« on: September 25, 2023, 08:58:28 am »

Hi, I have a weird problem:

I am using dyndns to host some services at home. If my ISP is providing me with a new public ip address, all is synced correctly vis ddclient. DNS is resolving the correct address and also wireshark VPN (UDP!) works fine. But all requests to my piblic IP based on TCP are dropped. Or at least my self hosted websites cant be resolved.
If I reload my PPPoE connection till I get a new IP from the ISP (3 times normally), TCP traffic to my public IP is resolved correctly again. This is a very inconvenient issue, because I need to monitor my service constantly and if it's down, I need to log in via VPN to reload the PPPoE Interface till I get a new IP.

Any ideas how to debug this further? I contacted the ISP and also tried a different router. No problems there. That's why I assume it is something buggy in OPNsense...