Messages

1

General Discussion / Re: Wireguard and assymetric traffic

« on: October 06, 2024, 05:11:49 pm »

I got it. I had to create an outbound NAT rule so that outgoing packets would be sent as coming from opnsense, and not from the remote server.

However, this has the problem that I won't get the logs of those who actually access the website - everything will have the address of the NAT server (opnsense). This is not ideal.

Is there a better way?

2

General Discussion / Wireguard and assymetric traffic

« on: October 06, 2024, 02:26:22 pm »

Hi,

I got a static ip address from a provider, which is basically a wireguard configuration so I get a public IP.
I have configured wireguard properly and all (I think), and I do a port forward so that traffic coming from the wireguard on, let's say, port 80, is redirected to my reverse proxy on my lan.

I see that the packets do arrive on my reverse proxy with tcpdump. I see that the server reply, but I don't see the reply sent back to opnsense - it seems my linux server (the reverse proxy) might be simply using another route to answer the request from the remote client.

Is there a way to configure nat in a way that traffic does go back through the same route it came from, or I have to add a route on my reverse proxy?

I tried to configure outbound NAT as described here: https://docs.opnsense.org/manual/how-tos/wireguard-selective-routing.html, but it doesn't seem to help.

Any clues?

3

23.7 Legacy Series / Re: Possible asymmetric routing? Lots of SYN_SENT:CLOSED

« on: January 09, 2024, 09:23:34 pm »

It was fail2ban that locked me out. 

4

23.7 Legacy Series / Possible asymmetric routing? Lots of SYN_SENT:CLOSED

« on: January 08, 2024, 07:07:36 pm »

Hi,

Bear with me, because this is a bit complicate:

I have two networks on two locations:

On location A, I have a network with a Unifi router, and port forwarding to some ports, including 80 and 443.
On location B, there's Opnsense and also port forwarding to some of the same ports as location A.

I have web servers and mail servers on both locations.

They are connected via a site-to-site Wireguard, which has worked flawlessly for almost two years.

Yesterday, out of the blue, I suddenly couldn't read the websites I host on location B, but _only from location B_! The world could still access them.

The webserver on location B also couldn't get any "curl" from the sites hosted at location A.

I did lots of tcpdump, and Wireshark  - as well as OPNSense logs, show me a lot of SYN_SENT between the two locations on the ports that are failing (80, 443, etc.).

This is what the "States" logs show me:

Code: [Select]

all tcp LOCATIONAIP:61025 LOCATIONBIP:443 LOCATIONBSERVER:443 CLOSED:SYN_SENT
all tcp LOCATIONAIP:61025 LOCATIONBSERVER:443 SYN_SENT:CLOSED let out anything from firewall host itself
all tcp LOCATIONAIP:61029 LOCATIONBIP:443 LOCATIONBSERVER:443 CLOSED:SYN_SENT
all tcp LOCATIONAIP:61029 LOCATIONBSERVER:443 SYN_SENT:CLOSED let out anything from firewall host itself
all tcp LOCATIONAIP:61031 LOCATIONBIP:443 LOCATIONBSERVER:443 CLOSED:SYN_SENT
all tcp LOCATIONAIP:61031 LOCATIONBSERVER:443 SYN_SENT:CLOSED let out anything from firewall host itself
all tcp LOCATIONAIP:46838 LOCATIONBIP:443 LOCATIONBSERVER:443 CLOSED:SYN_SENT
all tcp LOCATIONAIP:46838 LOCATIONBSERVER:443 SYN_SENT:CLOSED let out anything from firewall host itself
all tcp LOCATIONAIP:61040 LOCATIONBIP:443 LOCATIONBSERVER:443 CLOSED:SYN_SENT
all tcp LOCATIONAIP:61040 LOCATIONBSERVER:443 SYN_SENT:CLOSED let out anything from firewall host itself

LOCATIONAIP = WAN IP location A
LOCATIONBIP = WAN IP location B
LOCATIONBSERVER = webserver on location B

I see on tcpdump that traffic does goes back and forth, but I have no idea why it is getting this SYN_SENT:

Code: [Select]

19:03:31.056683 eth1  Out IP (tos 0x0, ttl 63, id 0, offset 0, flags [none], proto TCP (6), length 64)
    LOCATIONAIP.61153 > LOCATIONBIP.443: Flags \[S\], cksum 0xe3b4 (correct), seq 973490560, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 2633606817 ecr 0,sackOK,eol], length 0
19:03:31.106573 eth0  In  IP2 (invalid)
19:03:31.106573 eth0.21 In  IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto TCP (6), length 64)
    LAPTOPIP.61154 > LOCATIONBIP.443: Flags [S], cksum 0x4518 (correct), seq 697730383, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 552186860 ecr 0,sackOK,eol], length 0
19:03:31.106995 eth1  Out IP (tos 0x0, ttl 63, id 0, offset 0, flags [none], proto TCP (6), length 64)
    LOCATIONAIP.61154 > LOCATIONBIP.443: Flags [S], cksum 0x271a (correct), seq 697730383, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 552186860 ecr 0,sackOK,eol], length 0
19:03:31.137897 eth0  In  IP2 (invalid)
19:03:31.137897 eth0.21 In  IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto TCP (6), length 64)
    LAPTOPIP.61155 > LOCATIONBIP.443: Flags [S], cksum 0x4c52 (correct), seq 2061410625, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 3113723592 ecr 0,sackOK,eol], length 0
19:03:31.138285 eth1  Out IP (tos 0x0, ttl 63, id 0, offset 0, flags [none], proto TCP (6), length 64)
    LOCATIONAIP.61155 > LOCATIONBIP.443: Flags [S], cksum 0x2e54 (correct), seq 2061410625, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 3113723592 ecr 0,sackOK,eol], length 0

Nginx gives me no logs whatsoever.

I have Nat reflection. However, turning it off doesn't really help. Neither does it turning off wireguard.
Traffic also works normally if I connect a laptop behind location A to Location B via Wireguard (ie, directly, not via my site-to-site wireguard).

And, don't forget: outside my own net, everything works perfectly.

I fired up a python server on port 8000 on the webserver on location B (and the appropriate port forwarding), and I can't get traffic from this server on location A. However, doing the same on another server on location B - on the same vlan - works. This makes me think that the problem could be on the server, but I don't know.

How can I debug something like this? I'd appreciate any tip! So annoying that this just "happens" without any intervention whatsoever.

5

23.7 Legacy Series / Re: Half of requests block from internal host to itself - rdr rule

« on: November 20, 2023, 01:33:25 pm »

This is not a solution, but overriding the hostname internally via DNS seems to solve the issue. But I'd rather not do it since I'd have to do it for every domain name.

6

23.7 Legacy Series / Re: Half of requests block from internal host to itself - rdr rule

« on: November 20, 2023, 01:04:22 pm »

Things I tried so far:

- access from another host on the same vlan: same problem
- use another server instead of nginx, on another port: same problem
- disable nat reflector on the 443 port forwarding: no traffic from my vlan goes through, only external traffic.

7

23.7 Legacy Series / Re: Half of requests block from internal host to itself - rdr rule

« on: November 20, 2023, 12:32:46 pm »

It doesn't help - if I disable it, I get no traffic at all from my vlan that is sent to WAN at port 443.

8

23.7 Legacy Series / Re: Half of requests block from internal host to itself - rdr rule

« on: November 20, 2023, 11:30:46 am »

yes, it is disabled, and I use another port for the UI.

9

23.7 Legacy Series / Re: Half of requests block from internal host to itself - rdr rule

« on: November 20, 2023, 11:24:33 am »

I just realized now that this applies to all traffic to port 443 originating from the same vlan.

In short: when sending a request to port 443 of my WAN address from any host in the vlan where the port forwarding rule to port 443 sends traffic to, this happens. Half of requests goes through, half doesn't.

Is this a bug? Is there a way where I can override this behavior?

10

23.7 Legacy Series / Re: Half of requests block from internal host to itself - rdr rule

« on: November 20, 2023, 11:12:23 am »

nope!

host auth.mydomain.com
auth.med-lo.eu has address XX.XX.XX.XX <--- my wan address

11

23.7 Legacy Series / Half of requests block from internal host to itself - rdr rule

« on: November 20, 2023, 10:27:56 am »

Hi,

I have a setup where I have port forwarding setup for port 443 to my reverse proxy on one of my vlans.

The thing is that when my host send a request to the port 443 my WAN address, half of the time I get the request, half of the time I get connection refused.

It goes like this:

✘ francis@nginx $ curl https://auth.mydomain.com/realms/mydomain/account #works
 francis@nginx $ curl https://auth.mydomain.com/realms/mydomain/account
curl: (7) Failed to connect to auth.mydomain.com port 443: Connection refused
 ✘ francis@nginx $ curl https://auth.mydomain.com/realms/mydomain/account # works
 francis@nginx $ curl https://auth.mydomain.com/realms/mydomain/account
curl: (7) Failed to connect to auth.mydomain.com port 443: Connection refused
 ✘ francis@nginx $ curl https://auth.mydomain.com/realms/mydomain/account #works
 francis@nginx $ curl https://auth.mydomain.com/realms/mydomain/account
curl: (7) Failed to connect to auth.mydomain.com port 443: Connection refused
 ✘ francis@nginx $ curl https://auth.mydomain.com/realms/mydomain/account # works

Basically, every other request goes through.

When the connection is refused, I see a blue line log on firewall where the label says "rdr rule".

What can I do to fix this?

12

General Discussion / Port forwarding stopped working out of the blue!

« on: May 29, 2023, 03:37:02 pm »

Hi,

I have several port forward rules, including ports 80/443 to a web server.

Suddenly today I see the traffic through those ports do reach the server, but no data is sent back. If I do a curl from another server on some of the VLANs, it works fine. Doing it from an external client gets me a log on nginx (port 80), but no data is sent back.

All the ports that I forwarded are having the same behavior.

What could have suddenly stopped working? This has being working for years, and I haven't touched it for a while. Any hint on how to debug/fix this?

13

Virtual private networks / Wireguard not working - help!

« on: January 20, 2021, 07:29:58 pm »

Hi,

Wireguard stopped working for me. I use clients on mac and iPhone. It suddenly stopped working.
I don't know if the problem is on the client or on the server. I do see that access to the wireguard port is denied on the Firewall logs, despite that I have created a rule for that.

Any hints on this problem? What could I attach here to give an idea of the problem? Notice that I didn't touch anything, except that I updated OPNSense a few times.

Best,

Francis

14

High availability / Configuring WAN address

« on: December 21, 2020, 07:34:45 pm »

Hi,

I've seen some of the tutorials to configure CARP and HA, but one thing isn't clear to me:

On every tutorial, we need to configure private IP's for WAN. So the master gets like 10.1.0.20, the slave 10.1.0.30, and the CARP gets something like 10.1.0.1.

I have two questions:

- Where's the WAN IP assigned by the ISP configured, since the WAN interface is configured with those private IP's?
- I plan to use to VM's for the two OpnSense installs, on different machine. Is that ok to use vlan interface to connect both installs?

15

General Discussion / Re: Adding new network interface from ESXI reorders everything!

« on: September 30, 2020, 07:59:06 pm »

I see.
But isn't there a way to prevent OpnSense to change the interface name when adding another one? IOW, having em0 bound to mac address :aa instead of having to change the order of the adapters at ESXi...