Docker (registry-1.docker.io) get blocked

[ianwar]

I have OPNSENSE installed in machine with 2 NIC's. LAN & WAN in LAN I have docker server

my OPNSENSE installation includes using UNBOUND with TLS to 8.8.8.8 and 8.8.4.4

MY DHCP devices and docker server static IP points to my OPNSENSE

and in Rules>>LAN I have this rule
    IPv4 TCP/UDP    *    *    127.0.0.1    53 (DNS)    *    *       DNS Forward Rule

now I can't login to docker.io neither docker pull I got the messages

(Login did not succeed, error: Error response from daemon: Get "https://registry-1.docker.io/v2/": context deadline exceeded)

and this is if I use portainer pull

(Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers))

looks like the connection takes too long to this specific destination
note everything else is working I can access any website

Please I need your help

[Saarbremer]

Are you sure that this is your only LAN rule?

DNS just isn't enough

[ianwar]

YES the rule is in Firewall>>Rules>>LAN

[CJ]

Why do you have a DNS rule?  We need more information about your setup.

[ianwar]

Sure any information you need
so I plan to serve all my devices in the LAN with my UNBOUND, then after doing all UNBOUND TLS settings I needed to make sure that all DNS requests go through UNBOUND so I made this rule

so my physical installation is

Home router >Cable> OPNSENSE >Cable> SWITCH >Cable> WIFI AP
                                                                            >Cable> DOCKER Server

In my Home router OPNSENSE has DHCP IP and is set to DMZ so it has full access to internet

In my OPNSENSE I have two Network cards one to Home Router and is set to (WAN) and one to the Switch which set to LAN

then I installed some plugins (ACME Client for SSL cet, ddclient for DDNS, NGINX for revers proxy, Crowd source for Extra NGINX security)

then DHCPv4

Then I enabled UNBOUND with this settings
General:
 Listen port (53)
 Network interface (ALL)
 Enable DNSSEC (Marked)
 Register ISC DHCP4 Leases  (Marked)
 Register ISC DHCP Static Mappings (Marked)
 Flush DNS Cache during reload (Marked)
 Local Zone Type  (transparent)

Advanced
 Aggressive NSEC (Marked)
 Log Queries (Marked)
 
DNS over TLS
 8.8.8.8
 8.8.4.4

then Firewall>>Rules>>LAN
     IPv4 TCP/UDP    *    *    127.0.0.1    53 (DNS)    *    *       DNS Forward Rule

[CJ]

Sure any information you need
so I plan to serve all my devices in the LAN with my UNBOUND, then after doing all UNBOUND TLS settings I needed to make sure that all DNS requests go through UNBOUND so I made this rule

This sounds like an XY problem.  Remove the DNS rule and test the docker registry.  Does it work?

Once you've done that, post your Unbound changes.

[ianwar]

no not working yet
even ubuntu update and sync the time return with error
plus I found the FW blocks some IP address
52.213.60.25 owned by amazon
192.178.24.234 owned by google
172.64.149.149 owned by Cloudflare

it seems like blocking alot of secure domains that crazy

[meyergru]

This looks like the usual routing problem with such problematic double router setups. Actually, I believe, this does not work at all from the LAN side (like with your Ubuntu docker host).

Since OpnSense must have different subnets for "WAN" (DMZ in your speak) and "LAN" legs, did you either:

1. Use outbound NAT in order to hide the whole LAN behind one WAN (i.e. DMZ) IP (aka "double NAT") or
2. Have set up such that your front router knows how to reach your LAN subnet?

[ianwar]

Note: DMZ is set on home router (Huawei CPE) to pass all trafic from and to OPNSENSE without the need to open ports so that I can take control through OPNSENSE

I just tried to access LAN form WAN by connecting to the HOME router and I can't access any thing

and my OUTBOUND NAT set to AUTO

[meyergru]

Which outbound NAT is set to AUTO? That of the Huawei router? That is not the question.
In order to access your real WAN (not DMZ), you obviously need NAT on the Huawei, but depending on how you do it, you need another NAT on your OpnSense (double NAT).

Your Huawei expects only a specific RFC1918 subnet on its LAN leg (you called that DMZ). It does not know about the networks behind your OpnSense.

Your OpnSense's WAN IP must be in that DMZ subnet. Obviously, OpnSense's LAN subnet must be different from the DMZ subnet. So, you have two options to get this LAN IP range to communicate outside:

1. You can "hide" the LAN IP range via outbound NAT on your OpnSense. That way, these IPs all appear as the OpnSense's WAN IP to the Huawei (which in turn hides it via NAT to the internet, hence the term "double NAT").

2. You can set up a manual route on your Huawei to route the LAN subnet over the OpnSense WAN IP.

If you did not do one of those things, it will not work. Basic networking.

[ianwar]

meyergru 1st I would like to give a big thanks for taking time to reply

2nd - I did what you said turned outbound to manual and added a rule to translate all LAN to the WAN (internet) and it seems to work so far so another thanks for that I'll keep testing and if I face any issues I hope I can find you again

[CJ]

This looks like the usual routing problem with such problematic double router setups. Actually, I believe, this does not work at all from the LAN side (like with your Ubuntu docker host).

Doh!  Somehow I managed to skip completely over that whole section and not realize they were running double routers.