[SOLVED] I am losing my mind with transparent proxy

[jds]

This very well might be some Google problem, but getting help.from them is impossible, so am hoping someone here has fixed this before, or has some clues.

I wset up cache proxy, then transparent http proxy both according to the opnsense howtos,, and everything was working. I then set up SSL inspection, again according to the howto. I set up to bump certain sites, including the Google, googleapis and similar domains. I imported the certificate to my browser, and to the apps and vpn on my android tablet and phone. Everything worked fine, except the Google play store. It claimed that there was no internet connection. I double checked, triple checked everything. No joy. Rebooted everything, but no change. I gave up and restored my opnsense configuration to before even the cache proxy. No good. Rebooted the firewall. The router, the phone. Still no good. Removed the certificate, rebooted. No connection to Google. Even did a factory reset on the tablet after backing up everything to the micro sd. Seemed to work for a little while, but then started sending the same error again.the whole time, everything else on the interwebs was reachable.

It is the strange timelags the get me. Sometimes a setting takes time to.propagate, including on this issue. Rebooting isn't sufficient. But most urgently, is how do I fix this? I haven't found any clues in any logs, but I may not be looking in the right place.

UPDATE: Well, I solved the play store problem, but have not yet reimplemented the transparent proxy.  The key was finding out that it was NOT a firewall block, so OPNsense was not the issue, but rather a DNS filter.  One of the lists on pihole added the android.clients.google.com domain which blocked access. This happened at the same time that I set up transparent blocking, so that confused me.  Now I can reach the play store.  During this adventure, I discovered that (1) it is nearly impossible to reach anyone at Google, and (2) if that person you eventually reach does not have the information, they cannot ask someone else, or put you in contact with anyone else. This person could tell me what port the app uses, but could not find out the URL. I will try later to set up the transparent blocking again, but it will probably be necessary to add this domain to the bump list, too.  I found the list from here: https://community.arubanetworks.com/t5/Security/2017-Google-Play-Store-URL-whitelist/td-p/284663

Two asides:  I occasionally find (usually small) errors in the manuals or how-to lists.  Is there a place to send those correction, so that I can help keep them up to date?
Second aside, the web proxy mentions the yoyo ad blocking list, and references the squidblacklist, but says that it is only pay. However, they do provide a free list to block malicious domains.  This would seem to be a great minimum web filter to suggest in the how-to:
https://blog.squidblacklist.org/?p=1658

[fabian]

I cannot tell you what you have damaged afterwards but the reason for Android is a problem caused by certificate pinning (anti MITM thechnology).
You can solve this by adding some domains to the no bump hosts list. You cannot inspect that traffic.

[jds]

So, after your post, I did a little research on pinning. I am not sure about your proposed solution,
because I no longer have SSL inspection operational. I have gone back to the prior setup, so
effectively everything is bumped.  Your explanation makes sense for why things went awry to begin
with (if i missed some domain that should have been bumped), but I do not understand how that
could be a problem now that the firewall is not doing any transparent html proxying.  Am I missing
something?  I don't see now how I could "bump" any domains, with the whole transparent proxying
disabled.

[fabian]

you may have a firewall rule left which causes some issues.

[jds]

I can see no firewall rule that would block it. There is almost certainly no rule the has existed new since the problem began.

[jds]

Also, I see nothing in the firewall logs that looks like it is blocking it.

[jds]

I have found the problem, so marked my original post as solved.  I also put an update at the bottom explaining the solutions, plus a couple added bonuses.