Seemingly uncalled-for resets while port forwarding

[onnieoneone]

Hi,

I'm having an intermittent problem with (I think) my Opnsense 17.7.7 router/firewall. Apologies up front if I've missed some other post describing exactly this problem.

So, I have this Opnsense firewall (192.168.178.20) and a rpi3 (192.168.178.21) sitting on my flat network behind my ISP's CPE (192.168.178.1). I then have my real home network behind the Opnsense firewall.

I'm trying to terminal a little bit of HTTPS traffic from the outside world on the rpi3 and send it through my firewall as plain HTTP via a port forward from 192.168.178.20:1644/tcp to my Nextcloud server (10.1.6.44:80/tcp) in my real home network.

This works half the time. I'll show you what I mean.

Here's a successful wget from my docker instance on the rpi3:

Code: [Select]

root@b3f74cc5c974:/$ wget --header 'Host: myserver.com' http://192.168.178.20:1644/blah1.php
Connecting to 192.168.178.20:1644 (192.168.178.20:1644)
wget: server returned error: HTTP/1.1 404 Not Found
root@b3f74cc5c974:/$ netstat -tupa
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 localhost:9000          0.0.0.0:*               LISTEN      313/php-fpm.conf)
tcp        0      0 0.0.0.0:11211           0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:http            0.0.0.0:*               LISTEN      309/nginx.conf
tcp        0      0 0.0.0.0:https           0.0.0.0:*               LISTEN      309/nginx.conf
tcp        0      0 b3f74cc5c974:37972      myopnsensebox.com:1644  TIME_WAIT   -
tcp        0      0 :::11211                :::*                    LISTEN      -
udp        0      0 0.0.0.0:11211           0.0.0.0:*                           -
udp        0      0 :::11211                :::*                                -
And the Nextcloud server's access.log:

Code: [Select]

myserver.com 192.168.178.21 - - [30/Oct/2017:19:46:43 +0100] "GET /blah1.php HTTP/1.1" 404 0
So, the page isn't there, but they're at least they're on speaking terms.

Weird thing is that just a few seconds later I try again and get this:

Code: [Select]

root@b3f74cc5c974:/$ wget --header 'Host: myserver.com' http://192.168.178.20:1644/blah2.php
Connecting to 192.168.178.20:1644 (192.168.178.20:1644)
wget: can't connect to remote host (192.168.178.20): Connection refused
What's happening?

I did a traffic capture on the outside (192.168.178.20) and inside (10.1.6.1) interfaces of the firewall (simple text captures attached), but the crux of it is that the outside interface is resetting the second connection immediately like so:

Code: [Select]

19:47:40.630906 IP 192.168.178.21.37974 > 192.168.178.20.1644: Flags [S], seq 4153658341, win 29200, options [mss 1460,sackOK,TS val 10306995 ecr 0,nop,wscale 7], length 0
19:47:40.631670 IP 192.168.178.20.1644 > 192.168.178.21.37974: Flags [R.], seq 0, ack 4153658342, win 0, length 0
This is not showing up in any logs I can see, where can I look for what is doing this??

Many thanks,
Ben

[onnieoneone]

So, I gave up and used the HAProxy plugin instead. Nice to have learned something new.