Messages

Honestly, thank you for the "rubber ducking" because the more I think about it I just need a NAT port forward more than likely. I guess I just had a blind spot thinking "NAT=IPv4" and glanced over it thinking NPTv6 was where I needed to go.

I'll have to see when I stand up this new cluster and test and update this thread. Thanks for the help!

My diagrams.net expertise is sorely lacking but basically this.


I understand well enough the IPv4 side (essentially it's just a NAT port forward on the WAN to the service IP of the pod) and that works without issue but just trying to go dual stack here. Essentially it's the same thing only since IPv6 guarantees you GUA I assume that I should only listen on my K8s vlan GUA and then redirect that to the pod service IP. Which half makes me think that if I made a firewall rule that directs anything on the K8s vlan external IP to that pod IP I might be fine.

Image might not show so here it is- https://imgur.com/a/BcZuYwq

NPTv6 is for mapping whole prefixes, (e.g. the whole prefix 2003:0:0:0::/64 to fd01:0:0:0::/64, just a rough example)

If you want to use NAT overload (aka map a whole internal prefix to a single external IP address), just use the Outbound NAT menu and configure it just as if you would IPv4.

Boy you are quick @monviech :)

I also just thought- does it make sense then to just have some kind of dynamic IPv6 host alias (under Firewall-Aliases) and just a rule on that interface to direct all 80/443 to the ULA?

And I might be misunderstanding but I'm not trying to direct Outbound NAT- I am trying to get it so external devices can reach my ingress (ie. inbound via some combo of firewall rules to go to a certain pod.)

It does help though to read that NPTv6 is for mapping the entire prefix. I do not think this is what I would want then (unless that in combo with a firewall rule on the K8s vlan interface gets me the desired result, kind of like NAT Port Forwards.)

Trying to make this work with my on-prem K8s ingress.

The scenario is
- k8s cluster that peers with opnsense via BGP (and FRR of course)
- service subnet advertisement is done with a ULA subnet (and ingress pinned to a static LB IP)
- All hosted on a separate VLAN

Target setup
- Use NPTv6 to direct all external traffic to that specific ULA ipv6 address

Questions
- what are my settings for nptv6?

I've tried it and I *think* it should listen to the K8s vlan interface since that technically has an ipv6 address (courtesy of track interface), then internal should be...the entire ULA ipv6 address? And then lastly track interface being my K8s vlan interface yes?

When doing that it says "not listening on that interface" but selecting WAN as the interface works (maybe I misunderstand but I feel like that would be wrong but maybe it still is the right choice as providing all the actual external traffic.)

Then lastly I'd set up dynamic DNS to query the IP for traffic on the K8s vlan interface (like the rest I do, just with listening to that interface instead.) Or should that also be on WAN?

Apologies. I've looked but seems there is not a lot of info out there via blogs/videos and the opnsense docs have me confused just a bit (my lack of knowledge, not their lack of detail.)

Oddly enough, after another weird reboot (probably need to reinstall) it seems to be working somehow. Maybe it needed time (I gave it a day and no change but I guess it's been a few days at least by now) or something happened with that latest reboot.

I could be wrong because I honestly did not know of this before this version but it's not working for me. I'm unsure if it's specific to this version or not.

I made sure I have no errors in the new blocklist features and I've reloaded it since then but have NOT rebooted yet.

I see "Cannot read properties of undefined (reading 'total')" in the dev tools on the page. I have let it go a couple hours since fixing the issues with dnsbl of unbound.

One thing I will note is that I store /var/log in RAM to avoid excessive writes to my NVME. Will that at all affect the operation of the reporting? In Services-Unbound-Logging I see all the logs clearly. In Reporting-Unbound DNS I have nothing on Overview nor anything in the Details tab. I've enabled/disabled it and cleared the stats for it but no change over several hours.

Not a huge deal- just would prefer to have some nicer tools to look at what is going on vs. parsing through logs on Loki.

Edit- along with this, I get weird errors every few weeks it seems that generate crash reports. At first I thought it was something AMD related but maybe not. I debated trying opnsense-bootstrap which, to my understanding, wipes out all the opnsense stuff and reinstalls it from scratch (while somehow not wiping the config?) So I could run that, reboot, and it reinstalls all my plugins and keeps my config? I'm not above doing that or reinstalling if necessary but have never done so and have a lot of config done (multiple VLANs, multiple WG tunnels, policy tagging, etc. etc. that I absolutely do not want to set back up in case of issues.)

Hate to bump an old thread but just want to clear something up for anyone reading it. Services-Router Advertisement is *not* part of the ISC DHCP suite. Thus there is no *need* to migrate away from that (it is a separate service called radvd and actively maintained.)

So my migration became a lot easier. The steps (essentially) for me are like this https://docs.opnsense.org/manual/dnsmasq.html#dhcpv4-with-dns-registration and for a TLDR-

dnsmasq settings
- general- configure what you need here (including adding static mappings)
- hosts- export csv (tiny button in UI) from ISC DHCPv4 then import them into this panel
- domains- NA
- DHCP ranges- add all IPv4 ranges here (no need for dhcpv6)
- DHCP options- NA
- DHCP tags- NA

So basically just get your static mappings over, configure your ranges, then enable it (and add the unbound domain config as in the official docs.)

Hope this helps someone. If you still want to use nothing but dnsmasq and move away from radvd then I found this link helpful for understanding the RA modes in dnsmasq- https://rakhesh.com/linux-bsd/brief-note-on-ipv6-flags-and-dnsmasq-modes/

It seems to me (based on looking at the key usage specs) that CAs generated by opnsense are not designed to be used with revocation?

I ask this because I've had an old CA and am standing up a new one and see that my CAs are listed under revocation but are unable to export a CRL. Ones I've imported from a vendor or two are able to but clicking on the export button or anything else for mine does nothing.

Also, unsure but the docs say to click the "+" button to add one but I see no button like that on mine. Changed the theme back to the included stock "opnsense-dark" and still didn't see it.

Woof- I love the new (to me) docs with the tabs but man, I should've looked way closer and have seen that. Thank you.

For me I suppose ra-stateless would be the best for me. Thanks again.

Perhaps I should've read the opnsense config examples a bit more but now it's still a bit odd-

ra-stateless will generate ONLY a slaac address but slaac will receive a dhcpv6 address as well...

Honestly would volunteer to update the docs if I can figure out what option goes with what. I think it would make it a bit easier for people migrating. I just don't know what goes to what yet. :)

I am planning out my migrations from ISC DHCP to dnsmasq as my install shouldn't need all the bells and whistles of Kea.

The question I am running into is what the 1:1 match is for router advertisement modes from the old ISC implementation (Services-Router Advertisements) to dnsmasq (dnsmasq-DHCP Ranges-RA mode). Hoping this also serves as a clarifying post for people who come after me (I searched and didn't find what I needed.

dnsmasq

```
ra-only tells dnsmasq to offer Router Advertisement only on this subnet, and not DHCP.

slaac tells dnsmasq to offer Router Advertisement on this subnet and to set the A bit in the router advertisement, so that the client will use SLAAC addresses. When used with a DHCP range or static DHCP address this results in the client having both a DHCP-assigned and a SLAAC address.

ra-stateless sends router advertisements with the O and A bits set, and provides a stateless DHCP service. The client will use a SLAAC address, and use DHCP for other configuration information.

ra-names enables a mode which gives DNS names to dual-stack hosts which do SLAAC for IPv6. Dnsmasq uses the host's IPv4 lease to derive the name, network segment and MAC address and assumes that the host will also have an IPv6 address calculated using the SLAAC algorithm, on the same network segment. The address is pinged, and if a reply is received, an AAAA record is added to the DNS for this IPv6 address. Note that this is only happens for directly-connected networks, (not one doing DHCP via a relay) and it will not work if a host is using privacy extensions. ra-names can be combined with ra-stateless and slaac.

ra-advrouter enables a mode where router address(es) rather than prefix(es) are included in the advertisements. This is described in RFC-3775 section 7.2 and is used in mobile IPv6. In this mode the interval option is also included, as described in RFC-3775 section 7.3.

off-link tells dnsmasq to advertise the prefix without the on-link (aka L) bit set.
```

So I am wondering if my translation is right as I am using unmanaged currently on ISC-

(dnsmasq / ISC )
- ra-only = unmanaged (will basically serve DNS servers but clients use SLAAC for addressing)
- slaac = assisted
- ra-stateless = stateless DHCP
- ra-names = NA - seems an additional option you can use with slaac to resolve names for both v4 and v6 addresses on the same network
- ra-advrouter = router only?
- off-link = or this is router only?

I'm mainly interested for my use case in unmanaged. What this did for me in ISC was
- SLAAC addresses
- doled out DNS servers
- doled out a ULA prefix for clients

I didn't need dhcpv6 and it properly allowed clients to use ipv6 DNS which did ad blocking, etc. via unbound. Looking to replicate the same for dnsmasq. Apologies for formatting. I am trying to see about doing it a little cleaner.

I set my DNS in the WG config as well.  I don't bother with having a redirect rule.

What do you have set for Private DNS on the phone?

I did have my main LAN address as the DNS in the wireguard config on the phone. I changed that to the wireguard gateway and it still lets ads through.

I have nothing set for private DNS on the phone itself. I assume the wireguard app itself will direct all DNS to what I tell it.

I will note that I recently upgraded to 23.1.x (and still on that until 23.7 has another patch or two) and it now has the wireguard kernel implementation and no longer the go version. Did that introduce something different? Figured it'd be a seamless transition.

You have Private DNS set to Off or Automatic?  You should be using the os-wireguard plugin and not the go implementation, but that shouldn't be causing this.

What are you seeing in the unbound reporting?  What domains get queried when you start a game?  These are game apps, correct?

Can you have OPNSense grab a packet capture?  I'm wondering if the game is falling back to DoT or DoH after getting failures using the system DNS.  A lot of things will either hardcode additional DNS servers and/or use DoT/DoH.

Thank you both. I didn't have time for the longest time to troubleshoot further but finally got a sec.

Private DNS- *was* set to Automatic (by default I guess). Switched off and on. No change it seems.
Unbound- I do *not* see queries for it in my logs.
IPv6- I have configured this on a separate VLAN and tested- all works well at home. I can safely say ipv6 queries are successfully blocking it too.

But just sitting here, I will explain how I have it set up and how I figured out the issue.

1) Configure wireguard VPN per docs. https://docs.opnsense.org/manual/how-tos/wireguard-client.html
  a. Can configure ipv6 too and use this site to create a UL prefix for it- https://www.unique-local-ipv6.com/

2) *Optional* but I think it's prudent- assign the interfaces for your wireguard connection in opnsense (Step 5 from above guide). You can leave IPv4 set to "None" but I recommend taking one address (that isn't assigned to a wireguard client) and assigning it as a static IPv6 address on the interface. Reason why- unbound will now listen on this interface (if you allow it to listen on this or "All" interfaces) and can set this as a DNS server in your Android/mobile wireguard settings. IPv4 (for me) can be set to just your normal LAN IP of opnsense (assuming you allow traffic to your LAN, which is half of what I intend when setting this up, the other half blocking ads/scams/bs.) IPv6 (for my ISP) sometimes changes the prefix (no static here) so by leveraging a private address (instead of trying to track a randomly set "track" interface from your WAN on your WG or LAN interface) we can set this to always listen. It will thus be available without fail once you connect to the VPN.

3) Firewall-NAT-Port Forward- I set 4 rules here (NOTE the inverted destination with "!" and allow it to create filter rule association on the WG interface itself)
  a. Ipv4 tcp/udp to ! LAN_IP (for unbound) on DNS (53) and redirect to LAN IP on DNS (port 53)
  b. Ipv4 tcp/udp to ! LAN_IP (for unbound) on DoT (853) and redirect to LAN IP on DNS (port 53)
  c. Ipv6 tcp/udp to ! <our private IPv6 address set in step 2> on DNS (53) and redirect to <our private IPv6 address set in step 2> on DNS (port 53)
  d. Ipv6 tcp/udp to ! <our private IPv6 address set in step 2> on DoT (853) and redirect to <our private IPv6 address set in step 2> on DNS (port 53)
  e. I cannot guarantee this is absolutely necessary but...we're here so why not. Might help with apps that hardcode to google DNS, etc.

4) Firewall-WG interface - set up an IPv4/v6 pass * rule BELOW the automatically created rules (pass all traffic- if desired. Customize however you like)

5) Can leave Private DNS to Automatic (seems to work fine now that I've figured it out.)

Now...the actual crux of the issue that I had not even thought of is that the faulty app (among others) exist in my work profile. I run CalyxOS so all my "trusted" apps are basically FOSS and reside in the normal profile. All the google BS and stuff I like to freeze (with Shelter) or that I don't care to get any other data by whatever means they might go in the work profile.

Google, for good or bad reasons, doesn't want work profile traffic going over a VPN initiated in the main profile. That is why I was not seeing the queries.

This portion can be solved in one of two ways:
1) Install the offending app in the main profile.
2) Clone/install the VPN app (wireguard) into the work profile. Activate it from there, etc.

So once again, THANK YOU guys for helping me get this. Figuring out that ipv6 worked over wifi at home (for ad blocking) and all that helped me to really hone in on Android and then it just clicked when I thought of it being in the work profile.

Hopefully this helps someone in the future.

Edit- we don't listen on 853 with unbound (no infra set up for that- at least on my config) so I changed the 853 redirect to go to 53.

I set my DNS in the WG config as well.  I don't bother with having a redirect rule.

What do you have set for Private DNS on the phone?

I did have my main LAN address as the DNS in the wireguard config on the phone. I changed that to the wireguard gateway and it still lets ads through.

I have nothing set for private DNS on the phone itself. I assume the wireguard app itself will direct all DNS to what I tell it.

I will note that I recently upgraded to 23.1.x (and still on that until 23.7 has another patch or two) and it now has the wireguard kernel implementation and no longer the go version. Did that introduce something different? Figured it'd be a seamless transition.

I've reasoned about this a bit and can't get it.

I used the road warrior setup for wireguard. The idea being that I could use mainly mobile phones to connect back home and get all the benefits of ad blocking, LAN access, etc.

While at home, I can confirm that adblocking and such works just fine though I will note that I run only IPv4 on my main wifi VLAN. That DHCP server offers up my main LAN IP as the sole DNS provider.

For wireguard, I followed the guide all the way through 5 (so I have a WG interface.) I can see the rule automatically added in NAT outbound.

• 
  Firewall rules for the WG interface
  block ipv6
  (linked NAT port forward ---> IPv4 tcp/udp * * dst ! LAN_ADDRESS on 53 )
  IPv4 tcp/udp * * 127.0.0.3 53

That loopback is a new interface that I created and unbound listens on.

Now...in my wireguard configs I set my main LAN_ADDRESS as the sole dns server. DNS is NOT set on wireguard local (server) portion and disable routes is unchecked.

I can use termux and do nslookup on known domains that are redirected to 0.0.0.0.

nslookup 2no.co 8.8.8.8

Returns 0.0.0.0 and I can see in the unbound logs that it answers that itself. I can't seem to understand just how ads are getting in then.

If I open a game I can see via PCAPdroid that it's looking for certain domains and some are blocked, some aren't. It can be the same domain and one attempt is "error" and one is succeeded. Usually takes 15 seconds for the first ad to get through but then it's constant.

I feel like I'm missing something but can't see what. Anyone else have this going without issue?

Edit- apologies if it's a little scattered. I can provide any more info on request. Just not sure where to look at this moment.

I believe you would have to update your AdGuard Home yaml file itself on the host to mimic the same behavior in it as Unbound.