Is policy routing supported (or planned) in tandem with the forward proxy?

[senseivita]

I think I know the answer to this already, so it's more like a feature request than a question.
However, if you have an answer/suggestion/advise, I'm all ears and open to anything. :)

I keep around an OPNsense instance ZeroTier and a few other light tasks. I've a few issues with OSPF in this system so I tried updating before getting down to vtysh.

It didn't but I was just advised to use GUI-less FRR, problem solved. With that done, I explored a little to see what was in the long list of changes and I found out that ASN-based aliases are now supported on OPNsense. If they're the same thing as using pfBlockerNG on pfSense that's amazing news.

There were a few reasons why OPNsense would never fully replace pfSense: ASN filters, HAProxy's GUI, log views, and (somewhat for) the forward proxy and VRF. VRF isn't available of pfSense either, ASNs are done, next was HAProxy's GUI's modularity nightmare.

I get that making it modular could in theory make it more practical, I do. Assembling the final proxy from dropdowns does feel sleek in its own right, however, getting there is so much non-repeatable/-usable/-cyclable disorienting work that's harder to diagnose. The level of modularity is only justifiable by needs that go beyond what an in-firewall reverse proxy should safely have, probably better served by a first-party-supported, dedicated Aloha appliance. HAProxy's rules, matches/fetchers, conditions, monitors are tightly focused in scope and hardly ever reused; all of these need to be defined, each with their own character-constrained definition name which translates into several more components per proxied service to keep track of and hunt around from screen to screen. A giant blank box without any instruction whatsoever (to paste in config files standard well-documented HAProxy syntax) would be much more helpful, easy, and clearer. Even without instructions. Fortunately you can just NAT it or if coming for straightforward-UI-HAProxied pfSense you can just put it inline in a transit network.

That leaves out the forward proxy — I've mixed feelings on this — on one hand OPNsense allows customization of the error pages. I love designing dumb stuff so this is a perfect fit for me. On the other I'm also pragmatic (see: HAProxy above) and good looking error pages, that are errors in the end, aren't nearly as useful as the live log view on pfSense. Additionally neither implementation has usable/maintained block lists which is kind of strange more so now than ever.

With DoH and DoT mainstream, they're readily used to maliciously bypass our network policies. Case in point, I use the Yahoo! website to test network connectivity (because otherwise I'd never visit it, so I know it isn't cached); the other day I found that the website embeds some sort of DoH client that was making requests for yahoodns.net or similar. This can be dealt with a proxy, except that few of them support multi-WAN, the ones that do, have basically no support for de/re-encryption of the traffic. I've been looking non-stop for a good proxy to help with this since I have six DNS server hosts, each has one or two DNS servers (Active Directory, Pi-hole, Dnsmasq, Unbound and BIND) running to filter, proxy, route the traffic to keep tight control over those rogue DoH clients, DNS (filtering not domain-related) is easily 60-70% of my network issues.

But given OPNsense for a long time has done some weird voodoo (shared forwarding) to support splitting traffic among captive portal and other of its otherwise-conflicting components, I'm wondering: has the proxy service been worked on as well? A proxy can be extremely memory hungry to deploy one each upstream.

UNSUBSTANTIATED
I've given it some thought to this not just on OPNsense but using a variety of systems, just using a single OPNsense system though; I have a theory that it could be done if: in transparent proxy mode, use NAT to force-policy-route the traffic running in a first/second instance using the jails subsystem on xBSD. It would still need a lot of memory but they can share kernel and storage and free memory and don't need to send traffic over the network since they'd be in-host. And, were VRF be supported, even NAT could be skipped as well. But, unfortunately none of this is even hinted in the GUI nor mentioned in the documentation other than some dev-level stuff. I wanted to try this but not being as familiar with FreeBSD as am with Linux, macOS; it'll be a while before I find the time to I learn jails to test it out. On paper it seems very plausible though, a good enough solution for cases where a dedicated proxy isn't needed, for beefy multi-WAN/multi-tunnel firewall with a small 100GB-ish disk cache.

[franco]

If I understand correctly (your post is pretty badly structured and coloured with opinions and irrelevant information) the issue is a pf(4) limitation in FreeBSD being lifted only recently https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268717 to be able to steer local traffic through policy rules.

It's likely going to break existing setups as well (for floating rules for sure as well as interface rules with outgoing policies).


Cheers,
Franco