reverse proxy not working as expected behind opnsense

[opnuser1]

I am very unsure what the problem is, I am asking for help, thanks.
Opnsense machine is in front of everything.  I changed the ssl address for the gui from 443 to 10443.  I also enable NAT reflection and also automatic outbount nat for reflection.
I have a DMZ interface and the normal Lan and WAN going.
I am forwarding 80/443 with an alias.  All of the interfaces are forwarding them to the correct webserver with the reverse proxy running in docker (swag).
I also have a NAT port forward rule.
I set these up using the guide here:
https://forum.opnsense.org/index.php?topic=6155.0
the dns service is cloudflare, which is configured inside swag.
This setup worked just fine when i had everything behind an asus router.  So the difference is there is now opnsense in front.

It appears everything works, but the sites wont reach the actual subdomain defined addresses.  It works with the ip addresses locally.  I figure it has something to do with DNS settings (do I need to setup split DNS?), or http redirect, but I have no clue.

[opnuser1]

HI everyone, I am new here, and before I pepper everyone with annoying questions, let me preface...
I am no expert with networking, but lets say I have dabbled for decades and have broad familiarity.  Thank you for any advice.
Please let me know if I am being annoying with my questions.

In order for me not to get distracted, I have a set of goals I am trying to achieve with this opnsense installation that has replaced (essentially) my asus router.  I use this list so I won't get distracted and also because network troubleshooting is tough, so good to keep track.  My goals are (with current status): 
all LAN computers have access to internet and local
currently good

all DMZ computers have access to internet and specified LAN resources
currently good

searching and visiting websites has no major delay. 
Currently, there is several seconds of delay I'm guessing due to some DNS configuration issues.
reverse proxy is working for the webservers

internal addresses resolve fine locally
currently working

external subdomains are working with cloudflare
currently not working

external subdomains are working with another external domain without cloudflare
currently not working

There is an Asus Router on the LAN.  This is used for wireless access on the LAN.  However, it is configured to be outside the LAN set of addresses on the DHCP because it has it's own DHCP.  So it is connected on 192.168.1.250 on the WAN side (the LAN DHCP of opnsense here doesnt go past 245).  On the LAN side of the router, it is on the 192.168.50.1 set of addresses.

Make sure wifi clients have access to the LAN
this is currently not working.  Opnsense does not see the 192.168.1.250 on its list of connected clients.  The wifi connected clients seem to have regular internet access, but opnsense does not have any knowledge of them it seems.  I don't get what is going on, and I'm pretty sure it is about DNS configuration.[/li][/list]

And to complete the picture for anyone offering help, this is my network visual (attached).

[Fright]

Hi
I'm not sure if the question is actually about the opnsense - rather the network in general?
it looks like your asus works in gateway ("router") mode, while (if you really want to) wireless clients to have access to the local network, the asus must either route this traffic (not translate, but route. dont use wan-port. configure routing between wireless-clients subnet and LAN-subnet) or just work in AP-mode

[opnuser1]

It is probably 50/50 opnsense/general network related. 

Yes, I am having trouble with the router config.  I tried initially in AP mode, and I had that working when it was plugged into the LAN in one of the switch ports there.  Then I moved it to it's own interface socket, in router mode.  NOw, you are right it doesn't really work.  But I tried in that same socket with the LAN mode and it didn't work either.  Let me continue trying.  The reason why I did it this way is I thought I came across an article that suggested doing that...connect with the WAN, then let the router deal with it's own network as if opnsense was the ISP basically.  Something like that.  ANyway, let me try some more things and I will report back.

[opnuser1]

OK, I did your suggestion and things have improved.  Here's what I did:
I changed the router to AP mode.  It's still on it's own interface, but I opened up rules between it and LAN.
It was very difficult getting into the router web gui.  It runs on 80 and 443 and interferes with the rest of the network on those ports.  I was able to change the ports to 8443 and remove http altogether.  It still seems like a shaky connection, it doesn't seem reliable and takes a long time to connect.  Also, wireless clients don't get the internet, while if i plug the same clients in with the ethernet cable it works.  Not sure.  I blame all this on DNS config.

I also check the DNS addresses, and I don't understand where it is getting them from.  I looked all around opnsense I don't see where the DNS addresses for these are configured.

So....AP mode is working.  Clients are shown connected in opnsense dhcp leases.  Internet is not working on the clients, so some more things to figure out.

EDIT...ok I see a problem.  the wireless clients are being added to different subnets than what I expected.  It's on a different subnet as the router, which is weird because it is in AP mode.  So the listings in DHCP leases in opnsense must be old.

[opnuser1]

UPDATE
wireless router is working!  I had manual tcp4 configs in the adapter on the laptop.  I changed it to auto and now it all works.  So wireless router on its own interface in AP mode is working.

[opnuser1]

This solved a lot of problems.  To recap, I'm going to now list what is and isn't working...

WORKING
all LAN computers have access to internet and local
all DMZ computers have access to internet and specified LAN resources
Make sure wifi clients have access to the LAN and internet on it's own interface

NOT WORKING
it takes several seconds to connect to most websites.
reverse proxy is not working either with cloudflare or without cloudflare

[opnuser1]

The slow website access is solved.  I had static ips configured on the client machines.  I changed it all back to auto, and now everything is fast.

WORKING
all LAN computers have access to internet and local
all DMZ computers have access to internet and specified LAN resources
Make sure wifi clients have access to the LAN and internet on it's own interface
websites load quickly

NOT WORKING
reverse proxy is not working either with cloudflare or without cloudflare

[opnuser1]

OK I do not understand the port forwarding lingo, and I've now read several articles about it.  I don't understand the logic, but I can copy the steps of the instructions.  So this is a question where I'm asking to have someone just explain the lingo and logic please, thanks. 

NAT port forward rules
what does the interface option do?  what if I don't choose WAN?
Who is the source?  Who is the destination?  What's the difference between destination and redirect address?

Firewall Rules
Why is the NAT rules in a different section than the firewall rules?  Going back to the first question....why is there an interface selector up there, when each interface has it's own set of rules anyway?
I understand the source and destination in the rules here.  I don't understand it in the NAT rules.

[Fright]

I'd like to suggest reading a bit about pf working and the actual syntax of the rules (https://www.freebsd.org/cgi/man.cgi?query=pf.conf)

what does the interface option do?

in order to perform some action on the packet (translate\drop\allow), pf need to match the packet's parameters with a set of rules. interface specification is the first input "filter" for this selection - the rule will be applied only if the packet originated on the specified interface. if no interface is specified, translation is applied to packets on   all interfaces

Who is the source?  Who is the destination?  What's the difference between destination and redirect address?

Let's say an external client from the 10.10.10.0 network wants to access port 143 of a server located behind firewall, on the local network (say, 192.168.11.11).
in this case Source is 10.10.10.0, Destination is the Destination address of the client request (one of the external addresses that the client is accessing), Redirect Address is the actual address of the server (192.168.11.11)

Why is the NAT rules in a different section than the firewall rules?

translation and packet filtering is the different actions. "..Translation rules modify either the source or destination address of the packets..." "..Translation occurs before filtering the filter engine will see packets as they look after any addresses and ports have been translated.."
(quotes from the docs)

why is there an interface selector up there, when each interface has it's own set of rules anyway?

for convenience. when creating, the value is set in accordance with the interface selected in the menu by default, but it may be necessary to change it

[opnuser1]

Thank you, I am chewing on all this.  That link was helpful for understanding what the NAT is doing.

Another question...in the system--Settings--General area:
There is a hostname and domain field.  What should I put there?  For hostname, I leave it as opnsense.  For domain, I put in this dynamic dns address I have from dynu so that the external ip always uses that.  Is that the correct domain?  Other domians I have are the local windows domain, which i don't think goes here.  and I have 2 purchased domains I use for web services, but those don't go here either I'm guessing since they are managed by cloudflare.  I can also just put some meaningless domain here as well, right?

and on the same page, there is the DNS server area.  And this is for the WAN right?  so I put 8.8.8.8 or another public dns like that right?  I also had my local windows domain controller address there at one point, but it didn't seem correct.

[Fright]

Hi. the choice of opnsense host domain really depends on preferences, network requirements, implementation of the internal DNS scheme etc. there are no explicit requirements or recommendations, and the hostname of published services is configurable at the application level.
the same applies to the configuration of the DNS (recursion, forwarding to internal servers or vice versa etc) - it really depends on internal requirements and approaches to implementation