Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - randomwalk

#1
Quote from: Maurice on September 25, 2023, 12:39:43 PM
Quote from: randomwalk on September 24, 2023, 08:16:56 AM
So the LAN traffic goes like this:  LAN device (192.168.1.x) --> Ubuntu VM (192.168.1.10) --> out via Mullvad interface on the VM --> OPNsense gateway (192.168.1.1) --> internet.

Nope, it doesn't. If you use policy-based routing on OPNsense to achieve this, the LAN devices will keep using OPNsense as their default gateway, which then forwards the packets:

LAN device --> OPNsense --> Ubuntu VM --> OPNsense --> Internet

To avoid this, you can assign the Ubuntu VM as the default gateway via DHCP.

[/quote]

Making the VM multi-homed makes sense and it would be easy to add more virtual NICs.

I had not realized the above flow of traffic, which makes sense since if the traffic does not first hit OPNsense, then it can't do policy-based routing. I was thinking that because the VM and LAN devices are in the same subnet (192.168.1.1/24), then traffic between them wouldn't go through OPNsense.

I have two concerns about making the VM the default gateway (via DHCP or otherwise).

First, making the VM the default gateway would mean that none of the firewall rules or policy-based routing in OPNsense would have any effect, right? Since the traffic doesn't go to OPNsense first?

Second, since the VM currently forwards all incoming traffic on the LAN interface to wg-mullvad interface, LAN devices would not be able to access local subnets (devices in both the LAN and Guest nets) unless I somehow add iptable rules on the VM to direct "local" traffic to OPNsense. Effectively, I have to make the VM a router too?

It seems hard to decide whether making the VM the default gateway (so saving one hop to OPNsense for external traffic) is more efficient than the current setup where traffic across local subnets goes only through OPNsense without going through the VM, but external traffic requires an additional hop?  Just trying to understand the pros and cons of having the VM be the default gateway.
#2
Quote from: bartjsmit on September 24, 2023, 09:18:53 AM
Why not run another DNS server (such as pi-hole) and use that as a resolver for OPNsense, and therefore unbound.

The other networks could have an alternative default gateway as well. Can you add your Ubuntu to more VLAN's and set it as the DG there?

Bart...

Very interesting idea about adding a DNS server! It would add a hop in the lookup chain, but it should work. Thank you!

For the second point, how would I add the Ubuntu VM (192.168.1.10) to other subnets? I can certainly put that VM in its own VLAN (like 192.168.3.1/24), but I don't see how that would help. And I'm not aware of a way to have multiple IP addresses for the same computer (for example, I am not aware of a way for the VM being both 192.168.1.10 and 192.168.2.10 at the same time so that it can be the gateway for both 192.168.1.1/24 and 192.168.2.1/24).

#3
I have an interesting problem and would appreciate everyone's ideas.

I would like to use a separate computer (VM) on the network as a gateway and I have it mostly working, except two issues.  Specifically, I have a Ubuntu VM (192.168.1.10) on the LAN network (192.168.1.1/24) with Mullvad's app installed.  I have configured the VM to act as a gateway where all incoming traffic gets redirected out via Mullvad's Wireguard interface.  I prefer this over the built-in Wireguard plugin in OPNsense because the Mullvad app has additional features such as periodic rotation of the private key, rotation of the VPN server, quantum resistant encryption, etc.

In OPNsense, I manually add this VM (192.168.1.10) as a gateway.  Using firewall rules, I selectively direct some LAN traffic to use this VM as the gateway and can confirm that it works correctly.  For example, when using this gateway, the public IP address of LAN devices is the VPN server.

So the LAN traffic goes like this:  LAN device (192.168.1.x) --> Ubuntu VM (192.168.1.10) --> out via Mullvad interface on the VM --> OPNsense gateway (192.168.1.1) --> internet.

So this is all great, except there are at least two problems.

1)  I cannot figure out how to get Unbound to use this manually added gateway to send out DNS queries.  If I use the built-in OPNsense VPN functionality, then I can assign the VPN as an interface, which would allow me to configure Unbound to use the VPN as the outgoing interface.  However, my manually added VM (192.168.1.10) cannot be assigned an interface, so I'm not sure if there is some way around this problem.  Ideally, I would like to have DNS queries go through the VPN gateway so there is no DNS leak.

2)  I am not sure how to get devices on other subnets to use the VM as gateway.  For example, let's say I have a guest subnet (192.168.2.1/24).  If I use firewall rules to direct traffic on Guest Net to use 192.168.1.10 as the gateway, it does not work.  I think this might be because the default gateway on Guest Net is 192.168.2.1 and I don't think Guest Net devices can access 192.168.1.10.  I have tried adding a firewall rule to allow Guest Net devices pass traffic to 192.168.1.10, but that does not seem to solve the problem.

Any ideas would be greatly appreciated!
#4
I just want to say that the procedure by XeroX works!  To help folks who are interested in getting Windows Update to work with Squid SSL inspection, below are the certificates that you need to import into OPNsense. 

Go to OPNsense --> System --> Trust --> Authorities.  Click "Add," put in whatever descriptive name you want, method should be "Import and existing Certificate Authority," then paste in the below into the "Certificate Data" box.  You must include all of the text, including the part with BEGIN CERTIFICATE and END CERTIFICATE.

The first two certificates below can probably be found in your own Windows 10 installation.  Type "certmgr" into the Windows search box and that should find the certificate manager.  Under the folder "Trusted Root Certification Authorities," you should be able to find the first two certificates.  You can export them in the "Base-64 encoded X.509" format, then open the file in Notepad, which will show you what I pasted below.

The other two certificates I found online here:

https://censys.io/certificates/6139e2df97dc93bf7e90a303f75b3968fd06c57316b45e94dcff773707cf2754

https://censys.io/certificates/e39f93f3b2b40fd3c41de7dfa7d0b0cb6c4d8f97cbab2bb81c178f4b5f3c7eed

I don't know censys.io, but the certificate appears legit as when you open it, it tells you that they are issued by the other two root authorities.


Microsoft Root Certificate Authority 2011


-----BEGIN CERTIFICATE-----
MIIF7TCCA9WgAwIBAgIQP4vItfyfspZDtWnWbELhRDANBgkqhkiG9w0BAQsFADCB
iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1Jl
ZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEyMDAGA1UEAxMp
TWljcm9zb2Z0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIwMTEwHhcNMTEw
MzIyMjIwNTI4WhcNMzYwMzIyMjIxMzA0WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV
BAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jv
c29mdCBDb3Jwb3JhdGlvbjEyMDAGA1UEAxMpTWljcm9zb2Z0IFJvb3QgQ2VydGlm
aWNhdGUgQXV0aG9yaXR5IDIwMTEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK
AoICAQCygEGqNThNE3IyaCJNuLLx/9VSvGzH9dJKjDbu0cJcfoyKrq8TKG/Ac+M6
ztAlqFo6be+ouFmrEyNozQwph9FvgFyPRH9dkAFSWKxRxV8qh9zc2AodwQO5e7BW
6KPeZGHCnvjzfLnsDbVU/ky2ZU+I8JxImQxCCwl8MVkXeQZ4KI2JOkwDJb5xalwL
54RgpJki49KvhKSn+9GY7Qyp3pSJ4Q6g3MDOmT3qCFK7VnnkH4S6Hri0xElcTzFL
h93dBWcmmYDgcRGjuKVB4qRTufcyKYMME782XgSzS0NHL2vikR7TmE/dQgfI6B0S
/Jmpaz6SfsjWaTr8ZL22CZ3K/QwLopt3YEsDlKQwaRLWQi3BQUzK3Kr9j1uDRprZ
/LHR47PJf0h6zSTwQY9cdNCssBAgBkm3xy0hyFfj0IbzA2j70M5xwYmZSmQBbP3s
MJHPQTySx+W6hh1hhMdfgzlirrSSL0fzC/hV66AfWdC7dJse0Hbm8ukG1xDo+mTe
acY1logC8Ea4PyeZb8txiSk190gWAjWP1Xl8TQLPX+uKg09FcYj5qQ1OcunCnAfP
SRtOBA5jUYxe2ADBVSy2xuDCZU7JNDn1nLPEfuhhbhNfFcRf2X7tHc7uROzLLoax
7Dj2cO2rXBPB2Q8Nx4CyVe0096yb5MPa50c8prWPMd/FS6/r8QIDAQABo1EwTzAL
BgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUci06AjGQQ7kU
BU7h6qfHMdEjiTQwEAYJKwYBBAGCNxUBBAMCAQAwDQYJKoZIhvcNAQELBQADggIB
AH9yzw+3xRXbm8BJyiZb/p4T5tPw0tuXX/JLP02zrhmu7deXoKzvqTqjwkGw5biR
nhOBJAPmCf0/V0A5ISRW0RAvS0CpNoZLtFNXmvvxfomPEf4YbFGq6O0JlbXlccmh
6Yd1phV/yX43VF50k8XDZ8wNT2uoFwxtCJJ+i92Bqi1wIcM9BhS7vyRep4TXPw8h
Ir1LAAbblxzYXtTFC1yHblCk6MM4pPvLLMWSZpuFXst6bJN8gClYW1e1QGm6CHmm
ZGIVnYeWRbVmIyADixxzoNOieTPgUFmG2y/lAiXqcyqfABTINseSO+lOAOzYVgm5
M0kS0lQLAausR7aRKX1MtHWAUgHoyoL2n8ysnI8X6i8msKtyrAv+nlEex0NVZ09R
s1fWtuzuUrc66U7h14GIvE+OdbtLqPA1qibUZ2dJsnBMO5PcHd94kIZysjik0dyS
TclY6ysSXNQ7roxrsIPlAT/4CTL2kzU0Iq/dNw13CYArzUgA8YyZGUcFAenRv9FO
0OYoQzeZpApKCNmacXPSqs0xE2N2oTdvkjgefRI8ZjLny23h/FKJ3crWZgWalmG+
oijHHKOnNlA8OqTfSm7mhzvO6/DggTedEzxSjr25HTTGHdUKaj2YKXCMiSrRq4IQ
SB/c9O+lxbtVGjhjhE63bK2VVOxlIhBJF7jAHscPrFRH
-----END CERTIFICATE-----


Microsoft ECC Product Root Certificate Authority 2018


-----BEGIN CERTIFICATE-----
MIIDIzCCAqigAwIBAgIQFJgmZtx8zY9AU2d7uZnshTAKBggqhkjOPQQDAzCBlDEL
MAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1v
bmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjE+MDwGA1UEAxM1TWlj
cm9zb2Z0IEVDQyBQcm9kdWN0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIw
MTgwHhcNMTgwMjI3MjA0MjA4WhcNNDMwMjI3MjA1MDQ2WjCBlDELMAkGA1UEBhMC
VVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNV
BAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjE+MDwGA1UEAxM1TWljcm9zb2Z0IEVD
QyBQcm9kdWN0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIwMTgwdjAQBgcq
hkjOPQIBBgUrgQQAIgNiAATHERYqdh1Wjr65YmXUw8608MMw7I9t1245vMhJq6u4
40N41YEGXe/HfZ/O1rOQdd4MsJDeI7rI0T5n4BmpG4YxHl80Le4X/RX7fieKMqHq
yY/JfhjLLzssSHp9pvQBB6yjgbwwgbkwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB
/wQFMAMBAf8wHQYDVR0OBBYEFEPvcIe4nb/siBncxsRrdQ11NDMIMBAGCSsGAQQB
gjcVAQQDAgEAMGUGA1UdIAReMFwwBgYEVR0gADBSBgwrBgEEAYI3TIN9AQEwQjBA
BggrBgEFBQcCARY0aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9Eb2Nz
L1JlcG9zaXRvcnkuaHRtADAKBggqhkjOPQQDAwNpADBmAjEAocBJRF0yVSfMPpBu
JSKdJFubUTXHkUlJKqP5b08czd2c4bVXyZ7CIkWbBhVwHEW/AjEAxdMo63LHPrCs
Jwl/Yj1geeWS8UUquaUC5GC7/nornGCntZkU8rC+8LsFllZWj8Fo
-----END CERTIFICATE-----


Microsoft Update Secure Server CA 2.1


-----BEGIN CERTIFICATE-----
MIIF7TCCA9WgAwIBAgIQP4vItfyfspZDtWnWbELhRDANBgkqhkiG9w0BAQsFADCB
iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1Jl
ZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEyMDAGA1UEAxMp
TWljcm9zb2Z0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIwMTEwHhcNMTEw
MzIyMjIwNTI4WhcNMzYwMzIyMjIxMzA0WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV
BAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jv
c29mdCBDb3Jwb3JhdGlvbjEyMDAGA1UEAxMpTWljcm9zb2Z0IFJvb3QgQ2VydGlm
aWNhdGUgQXV0aG9yaXR5IDIwMTEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK
AoICAQCygEGqNThNE3IyaCJNuLLx/9VSvGzH9dJKjDbu0cJcfoyKrq8TKG/Ac+M6
ztAlqFo6be+ouFmrEyNozQwph9FvgFyPRH9dkAFSWKxRxV8qh9zc2AodwQO5e7BW
6KPeZGHCnvjzfLnsDbVU/ky2ZU+I8JxImQxCCwl8MVkXeQZ4KI2JOkwDJb5xalwL
54RgpJki49KvhKSn+9GY7Qyp3pSJ4Q6g3MDOmT3qCFK7VnnkH4S6Hri0xElcTzFL
h93dBWcmmYDgcRGjuKVB4qRTufcyKYMME782XgSzS0NHL2vikR7TmE/dQgfI6B0S
/Jmpaz6SfsjWaTr8ZL22CZ3K/QwLopt3YEsDlKQwaRLWQi3BQUzK3Kr9j1uDRprZ
/LHR47PJf0h6zSTwQY9cdNCssBAgBkm3xy0hyFfj0IbzA2j70M5xwYmZSmQBbP3s
MJHPQTySx+W6hh1hhMdfgzlirrSSL0fzC/hV66AfWdC7dJse0Hbm8ukG1xDo+mTe
acY1logC8Ea4PyeZb8txiSk190gWAjWP1Xl8TQLPX+uKg09FcYj5qQ1OcunCnAfP
SRtOBA5jUYxe2ADBVSy2xuDCZU7JNDn1nLPEfuhhbhNfFcRf2X7tHc7uROzLLoax
7Dj2cO2rXBPB2Q8Nx4CyVe0096yb5MPa50c8prWPMd/FS6/r8QIDAQABo1EwTzAL
BgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUci06AjGQQ7kU
BU7h6qfHMdEjiTQwEAYJKwYBBAGCNxUBBAMCAQAwDQYJKoZIhvcNAQELBQADggIB
AH9yzw+3xRXbm8BJyiZb/p4T5tPw0tuXX/JLP02zrhmu7deXoKzvqTqjwkGw5biR
nhOBJAPmCf0/V0A5ISRW0RAvS0CpNoZLtFNXmvvxfomPEf4YbFGq6O0JlbXlccmh
6Yd1phV/yX43VF50k8XDZ8wNT2uoFwxtCJJ+i92Bqi1wIcM9BhS7vyRep4TXPw8h
Ir1LAAbblxzYXtTFC1yHblCk6MM4pPvLLMWSZpuFXst6bJN8gClYW1e1QGm6CHmm
ZGIVnYeWRbVmIyADixxzoNOieTPgUFmG2y/lAiXqcyqfABTINseSO+lOAOzYVgm5
M0kS0lQLAausR7aRKX1MtHWAUgHoyoL2n8ysnI8X6i8msKtyrAv+nlEex0NVZ09R
s1fWtuzuUrc66U7h14GIvE+OdbtLqPA1qibUZ2dJsnBMO5PcHd94kIZysjik0dyS
TclY6ysSXNQ7roxrsIPlAT/4CTL2kzU0Iq/dNw13CYArzUgA8YyZGUcFAenRv9FO
0OYoQzeZpApKCNmacXPSqs0xE2N2oTdvkjgefRI8ZjLny23h/FKJ3crWZgWalmG+
oijHHKOnNlA8OqTfSm7mhzvO6/DggTedEzxSjr25HTTGHdUKaj2YKXCMiSrRq4IQ
SB/c9O+lxbtVGjhjhE63bK2VVOxlIhBJF7jAHscPrFRH
-----END CERTIFICATE-----


Microsoft ECC Content Distribution Secure Server CA 2.1


-----BEGIN CERTIFICATE-----
MIIEbzCCA/agAwIBAgITMwAAAAkGbLYB5EGOcwAAAAAACTAKBggqhkjOPQQDAzCB
lDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1Jl
ZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjE+MDwGA1UEAxM1
TWljcm9zb2Z0IEVDQyBQcm9kdWN0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5
IDIwMTgwHhcNMTgxMjA3MjAwNTM1WhcNMzMxMjA3MjAxNTM1WjCBljELMAkGA1UE
BhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAc
BgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjFAMD4GA1UEAxM3TWljcm9zb2Z0
IEVDQyBDb250ZW50IERpc3RyaWJ1dGlvbiBTZWN1cmUgU2VydmVyIENBIDIuMTB2
MBAGByqGSM49AgEGBSuBBAAiA2IABJZFwxWVPpORXxdExmMwhhRWYMWmvV8dA2gQ
KJOa6+MFlSq7AFchZZTP4ElW6B/w5cIz/7XYsQylwGwMWzB75vF2sgZEinv89FCC
Ee10iRmPAXMdypIyKGovmxy7hapay6OCAgQwggIAMA4GA1UdDwEB/wQEAwIBhjAQ
BgkrBgEEAYI3FQEEAwIBADAdBgNVHQ4EFgQURVR4uCOsreQqjLsBQVK0nI6Bke4w
VQYDVR0gBE4wTDBKBgRVHSAAMEIwQAYIKwYBBQUHAgEWNGh0dHA6Ly93d3cubWlj
cm9zb2Z0LmNvbS9wa2lvcHMvRG9jcy9SZXBvc2l0b3J5Lmh0bQAwEwYDVR0lBAww
CgYIKwYBBQUHAwEwGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwDwYDVR0TAQH/
BAUwAwEB/zAfBgNVHSMEGDAWgBRD73CHuJ2/7IgZ3MbEa3UNdTQzCDB6BgNVHR8E
czBxMG+gbaBrhmlodHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpb3BzL2NybC9N
aWNyb3NvZnQlMjBFQ0MlMjBQcm9kdWN0JTIwUm9vdCUyMENlcnRpZmljYXRlJTIw
QXV0aG9yaXR5JTIwMjAxOC5jcmwwgYcGCCsGAQUFBwEBBHsweTB3BggrBgEFBQcw
AoZraHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jZXJ0cy9NaWNyb3Nv
ZnQlMjBFQ0MlMjBQcm9kdWN0JTIwUm9vdCUyMENlcnRpZmljYXRlJTIwQXV0aG9y
aXR5JTIwMjAxOC5jcnQwCgYIKoZIzj0EAwMDZwAwZAIwGbO8ZwbR88S4RMjDqeR+
hG14h7sLuPMWicdmt6lPyx/q2CdJc4eXo39Aol7hR19eAjAIqH0bfMxzM8sxNI4M
W1oR30+Ncas1w44Wh9PoHf6+REAkEDuKjtpW9XqcBEmVh9I=
-----END CERTIFICATE-----
#5
Mullvad has a Bash script that appears to query their API to (1) get a list of their Wireguard server's public keys and locations, (2) generate your private key, (3) query their API to register your public key with Mullvad under your account, (4) save the API's response with the local interface address, and (5) write a Wireguard configuration file based on all of this info.  The script is here:  https://mullvad.net/media/files/mullvad-wg.sh

I am wondering if in OPNsense, one can change the Wireguard settings programmatically, and if so, how?  Is there an API to change the settings you see in the GUI?

I am interested in creating a cron job that can periodically (1) rotate the Mullvad endpoint, and (2) change the private key and other related settings.  Mullvad's own script seems like a helpful starting point.  If there is an API in OPNsense to change these settings, then this seems very doable.  Any insights would be greatly appreciated.

Also, does OPNsense have bash installed by default?
#6
Quote from: Maurice on March 06, 2021, 10:50:55 PM
Unbound default is 1232 bytes. If it works with a larger value, this might indicate that TCP fallback doesn't work through the tunnel for some reason.

I don't need to specifically open ports on my VPN interface to allow DNS to work over TCP, correct?  Otherwise, I am not sure what would be blocking when using VPN.
#7
Quote from: Maurice on March 06, 2021, 09:50:13 PM
Probably related to packet size. DNS packets are significantly larger if they contain DNSSEC records.
Keywords for further research: EDNS, MTU, fragmentation, PMTUD, DNS over TCP vs. UDP.

Ok, I think I solved it by adding this custom option in unbound settings:

edns-buffer-size: 4096

I had previously thought the problem might be fragmentation and looked into this EDNS setting.  But I incorrectly thought the way to solve fragmentation issues was to set the EDNS buffer size to be something small.  That obviously didn't work, which prompted me to post on this forum.  But actually, the solution was to set the buffer to be something high.  According to the internet, the default for this setting should be 4096, but that does not appear to be the case in OPNsense.  Once I manually specify this setting, it resolves fine.

Now when I run dig with unbound traffic through the VPN and DNSSEC enabled, here is the output:

root@OPNsense:~ # dig @127.0.0.1 workplace.schwab.com

; <<>> DiG 9.16.10 <<>> @127.0.0.1 workplace.schwab.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16740
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;workplace.schwab.com.          IN      A

;; ANSWER SECTION:
workplace.schwab.com.   300     IN      CNAME   workplace.gslb.schwab.com.
workplace.gslb.schwab.com. 20   IN      A       162.93.221.50

;; Query time: 112 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Mar 06 13:25:55 PST 2021
;; MSG SIZE  rcvd: 94


Notice that the EDNS UDP size is 4096, whereas in my previous posts, this size was 1232.
#8
If I set the timeout option to be a very long time, it comes back with a SERVFAIL when unbound traffic goes through the VPN and DNSSEC is enabled.

root@OPNsense:~ # dig @127.0.0.1 workplace.schwab.com +timeout=240

; <<>> DiG 9.16.10 <<>> @127.0.0.1 workplace.schwab.com +timeout=240
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 44880
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;workplace.schwab.com.          IN      A

;; Query time: 92514 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Mar 06 12:07:32 PST 2021
;; MSG SIZE  rcvd: 49


If I send unbound traffic through WAN, with DNSSEC still enabled, it resolves very quickly.

root@OPNsense:~ # dig @127.0.0.1 workplace.schwab.com +timeout=240

; <<>> DiG 9.16.10 <<>> @127.0.0.1 workplace.schwab.com +timeout=240
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5622
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;workplace.schwab.com.          IN      A

;; ANSWER SECTION:
workplace.schwab.com.   300     IN      CNAME   workplace.gslb.schwab.com.
workplace.gslb.schwab.com. 20   IN      A       162.93.233.50

;; Query time: 329 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Mar 06 12:11:47 PST 2021
;; MSG SIZE  rcvd: 94


If I send the unbound traffic through the VPN, but disable DNSSEC, it also resolves quickly.

root@OPNsense:~ # dig @127.0.0.1 workplace.schwab.com +timeout=240

; <<>> DiG 9.16.10 <<>> @127.0.0.1 workplace.schwab.com +timeout=240
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50717
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;workplace.schwab.com.          IN      A

;; ANSWER SECTION:
workplace.schwab.com.   294     IN      CNAME   workplace.gslb.schwab.com.
workplace.gslb.schwab.com. 14   IN      A       162.93.232.50

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Mar 06 12:13:52 PST 2021
;; MSG SIZE  rcvd: 94
#9
Quote from: schnipp on March 05, 2021, 01:21:01 PM
Oh sorry, wrong language in this forum  ???. Here is the translation...

Where do you exactly suspect the issue? The missing ip address in the output of "dig @127.0.0.1 www.schwab.com +trace" looks correct, because the parameter "+trace" only returns the delegation path (see manpage).

I am not sure what the issue is.  If I try to dig "workplace.schwab.com" without the +trace, it times out, but dig "www.schwab.com" works fine.  I don't understand what could cause this issue.

root@OPNsense:~ # dig @127.0.0.1 workplace.schwab.com

; <<>> DiG 9.16.10 <<>> @127.0.0.1 workplace.schwab.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached


root@OPNsense:~ # dig @127.0.0.1 www.schwab.com

; <<>> DiG 9.16.10 <<>> @127.0.0.1 www.schwab.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45826
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;www.schwab.com.                        IN      A

;; ANSWER SECTION:
www.schwab.com.         300     IN      CNAME   www.schwab.com.edgekey.net.
www.schwab.com.edgekey.net. 21600 IN    CNAME   e17738.x.akamaiedge.net.
e17738.x.akamaiedge.net. 20     IN      A       184.24.175.152

;; Query time: 406 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Mar 06 11:53:51 PST 2021
;; MSG SIZE  rcvd: 133
#10
I have a very weird DNS resolution problem that I cannot figure out.  I'm running OPNsense 20.7.8_4.  I'm using unbound in resolver mode with DNSSEC turned on and unbound traffic sent out via Mullvad OpenVPN (UDP) tunnel. 

The setup generally works great, but for some reason, unbound fails to resolve certain domains.  For example, it will not resolve "workplace.schwab.com."  There are likely other domains, but I don't have a list.  What I found is that unbound will resolve "workplace.schwab.com" if I either:

1) turn off DNSSEC (and continue to send unbound traffic via VPN); OR

2) send unbound traffic out via WAN (in this case, I do NOT have to turn off DNSSEC).

If I do not do either of the above, unbound does not resolve "workplace.schwab.com".  If I go to Interfaces --> Diagnostics --> DNS Lookup and put in "workplace.schwab.com," it would take about 10 seconds to run, and return the following:

Response
Type Address
CNAME workplace.gslb.schwab.com.
A 162.93.221.50

Resolution time per server
Server Query time
127.0.0.1 No response
1.1.1.1 45 msec
1.0.0.1 8 msec


As you can see above, in forward mode (to 1.1.1.1 or 1.0.0.1), DNS resolution works fine.  But unbound at 127.0.0.1 gets "No response."

If I SSH into OPNsense and run dig at the shell, nothing seems obviously wrong EXCEPT the dig takes like 2.5 minute to complete (it pauses for a super long time between the first block of output for the root-servers and the second block of output, then the remaining blocks of output follow very quickly).  Here is the output.

root@OPNsense:~ # dig @127.0.0.1 workplace.schwab.com +trace

; <<>> DiG 9.16.10 <<>> @127.0.0.1 workplace.schwab.com +trace
; (1 server found)
;; global options: +cmd
.                       80398   IN      NS      m.root-servers.net.
.                       80398   IN      NS      a.root-servers.net.
.                       80398   IN      NS      b.root-servers.net.
.                       80398   IN      NS      c.root-servers.net.
.                       80398   IN      NS      d.root-servers.net.
.                       80398   IN      NS      e.root-servers.net.
.                       80398   IN      NS      f.root-servers.net.
.                       80398   IN      NS      g.root-servers.net.
.                       80398   IN      NS      h.root-servers.net.
.                       80398   IN      NS      i.root-servers.net.
.                       80398   IN      NS      j.root-servers.net.
.                       80398   IN      NS      k.root-servers.net.
.                       80398   IN      NS      l.root-servers.net.
.                       80398   IN      RRSIG   NS 8 0 518400 20210318050000 20210305040000 42351 . RGrSTUNk4Ad41ITau7wzwMrm6Uk/ReeJlR/1cul8D1bs7qdYZOeICUvX CU+j9KipCbh0VUKvbcVWXFlpWoy9k/4ay0u1ZB5BbooERfyfGVyTe4ru pXrXymKeFLetZFhUr2KoO6ITyigRPPNvJFkRhwUn6nHqgCiHEvdG2cZW FmmvFpZ+0ejIB1h7lJYg+iaG8be2tI3aXp3CF/u8Cerjii5DddESAZrL bR9K6SeeQB9GxabnQJMvFY2FXsHBps9BQkx6D1vc5Vpn8E7R4e3uIcte Rt0c7fwvOyZE1lwHsvhxIaXugLJdlSX0bWT5XwGtGFm3xo6OHuL2cqXJ 9HbxVQ==
;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms

com.                    172800  IN      NS      l.gtld-servers.net.
com.                    172800  IN      NS      b.gtld-servers.net.
com.                    172800  IN      NS      c.gtld-servers.net.
com.                    172800  IN      NS      d.gtld-servers.net.
com.                    172800  IN      NS      e.gtld-servers.net.
com.                    172800  IN      NS      f.gtld-servers.net.
com.                    172800  IN      NS      g.gtld-servers.net.
com.                    172800  IN      NS      a.gtld-servers.net.
com.                    172800  IN      NS      h.gtld-servers.net.
com.                    172800  IN      NS      i.gtld-servers.net.
com.                    172800  IN      NS      j.gtld-servers.net.
com.                    172800  IN      NS      k.gtld-servers.net.
com.                    172800  IN      NS      m.gtld-servers.net.
com.                    86400   IN      DS      30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com.                    86400   IN      RRSIG   DS 8 1 86400 20210318050000 20210305040000 42351 . bVi/an3ya9VuX/O+2R5wTHP5+Ea7jmmQD+ZVs6rbmTpExiGl8Hsc8P+5 HSIbOcN9qcv/wnXoVwm8zLQojXWxJO4o4rkfAWI2fQ4ZvgEzZF5rxbmz DhOrXOexP7Yick8UqQpX8KADBrU6cH+jv1sYcc+pcDX0GzIq/LQV3bSa crTjtxBiqhYT8LD3d7bQ/kDbo6jyXMQTe77j2qFohW2+X3KBTpfFK6BZ iIrslY0OUYSCMqasCk9v5wSkM3qE0ebJlo71zcJVeGVaLEAEupS/HEzb ne+KSBIOMHJ3zSmZaFMXCZPSYmBAF2poNSh+L13Xpkf4Ib7w12PtWPUz BplviQ==
;; Received 1180 bytes from 192.5.5.241#53(f.root-servers.net) in 7 ms

schwab.com.             172800  IN      NS      ns1.schwab.com.
schwab.com.             172800  IN      NS      ns2.schwab.com.
schwab.com.             172800  IN      NS      ns3.schwab.com.
schwab.com.             172800  IN      NS      ns4.schwab.com.
schwab.com.             172800  IN      NS      a9-65.akam.net.
schwab.com.             172800  IN      NS      a8-64.akam.net.
schwab.com.             86400   IN      DS      3829 8 2 8B39D6D8CE4FA5D55DEB38CF05BB81E0CC087FA978AB9E0721411513 86CF2EA2
schwab.com.             86400   IN      RRSIG   DS 8 2 86400 20210309054915 20210302043915 58540 com. WCclyXLsxq4uaQpBB5WFJZvYbVNCra/EeN/AaBE+xVT0e+W9P0rJnWOM 1MdQ+FFdQDQndy9HQantJh7pOYsrroIrBDC84/MvvihnAzl0cSzUv8/1 zH95Rn0TGmyP1iGtUoBR9LTspXOy6vd6bsi3x8/J/KjzHco31YeBig1j nUSvSOG+w0gOx5XWq+1jkfh8rtIVTb8gDfDRc/muamDnNQ==
;; Received 476 bytes from 192.54.112.30#53(h.gtld-servers.net) in 22 ms

workplace.schwab.com.   300     IN      CNAME   workplace.gslb.schwab.com.
workplace.schwab.com.   300     IN      RRSIG   CNAME 8 3 300 20210313093720 20210211084427 43563 schwab.com. HMRYlzV44nhXrDntld7SwDAbk/zihLTrIwF+O6TnjdBjzwyAmYmT1BJA 9cAT7JAtQ8jKrkQDXvfrVdWZWiN/Pgrd1sjpprnasNaggYG/lg9hsfWU PawjDfTLfXs0jC/6PVHNcmJS1JoplkB8ccdzFMbFDw6qpxhx5ISP3MeX yl9yKrl7YJH69ufLv503ZU0tKKZ6oHJg60D07U9uxSuu6LZ6aDbYT0IA SHCEgVWq25uKBTS8eTekYalS0clyCYH9oeJ9JRN0GL84AoAlsZqOUeEj rde0yCzPk/aTCTZat8PgCP0Uz4gP/ooz6htu7TdCL7hDhqlRjbdowgIW Lq6CFg==
gslb.schwab.com.        900     IN      NS      gslb-anycast.schwab.com.
gslb.schwab.com.        86400   IN      DS      28456 8 2 D62CE9A0008171EE1F9DAC7A50AC167ADFCCF12A85C0314083F9CB86 8AC8C52F
gslb.schwab.com.        86400   IN      RRSIG   DS 8 3 86400 20210313094830 20210211090458 43563 schwab.com. ZaD1MLn/fOWaXgwZ6pyP2eKF5aG4t6fwjnRau/YF6zjigvfGHU+sNa26 qyzcFu2dnEUZsmnie2WDN4w7IhnkbzRUnzPN2Dkegj7gVvJ23UbkDOxP sQIxLWkog5okaUK9fv03Rh9pNk8pTEVUoSn/nnuPXrU57eJwscl2BJCc 6dzDuruTNE+wtmHe97tv3HZupWhyy4B5MpAKh6awWRBShpLmIE2NK0cR Hkwfo+Vb1cE2yfH6XTDQA/QeV1mBw32uvPQBT9Tp1ZGF6THjqZWyfaCV 1hsSN+KWavOgAjWxIt0OqJrfGewaQCQJDn5n0MrXQxB3ndoSxk/8/vYk wALTcw==
;; Received 1063 bytes from 162.93.253.171#53(ns3.schwab.com) in 43 ms


And if I dig "workplace.gslb.schwab.com" I get the correct IP address (162.93.221.50).  Again, the dig takes 2.5 minutes to complete, but the pause is only between the first block of output and the second block of output.  Here is the output.

root@OPNsense:~ # dig @127.0.0.1 workplace.gslb.schwab.com +trace

; <<>> DiG 9.16.10 <<>> @127.0.0.1 workplace.gslb.schwab.com +trace
; (1 server found)
;; global options: +cmd
.                       80069   IN      NS      i.root-servers.net.
.                       80069   IN      NS      j.root-servers.net.
.                       80069   IN      NS      k.root-servers.net.
.                       80069   IN      NS      l.root-servers.net.
.                       80069   IN      NS      m.root-servers.net.
.                       80069   IN      NS      a.root-servers.net.
.                       80069   IN      NS      b.root-servers.net.
.                       80069   IN      NS      c.root-servers.net.
.                       80069   IN      NS      d.root-servers.net.
.                       80069   IN      NS      e.root-servers.net.
.                       80069   IN      NS      f.root-servers.net.
.                       80069   IN      NS      g.root-servers.net.
.                       80069   IN      NS      h.root-servers.net.
.                       80069   IN      RRSIG   NS 8 0 518400 20210318050000 20210305040000 42351 . RGrSTUNk4Ad41ITau7wzwMrm6Uk/ReeJlR/1cul8D1bs7qdYZOeICUvX CU+j9KipCbh0VUKvbcVWXFlpWoy9k/4ay0u1ZB5BbooERfyfGVyTe4ru pXrXymKeFLetZFhUr2KoO6ITyigRPPNvJFkRhwUn6nHqgCiHEvdG2cZW FmmvFpZ+0ejIB1h7lJYg+iaG8be2tI3aXp3CF/u8Cerjii5DddESAZrL bR9K6SeeQB9GxabnQJMvFY2FXsHBps9BQkx6D1vc5Vpn8E7R4e3uIcte Rt0c7fwvOyZE1lwHsvhxIaXugLJdlSX0bWT5XwGtGFm3xo6OHuL2cqXJ 9HbxVQ==
;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms

com.                    172800  IN      NS      a.gtld-servers.net.
com.                    172800  IN      NS      b.gtld-servers.net.
com.                    172800  IN      NS      c.gtld-servers.net.
com.                    172800  IN      NS      d.gtld-servers.net.
com.                    172800  IN      NS      e.gtld-servers.net.
com.                    172800  IN      NS      f.gtld-servers.net.
com.                    172800  IN      NS      g.gtld-servers.net.
com.                    172800  IN      NS      h.gtld-servers.net.
com.                    172800  IN      NS      i.gtld-servers.net.
com.                    172800  IN      NS      j.gtld-servers.net.
com.                    172800  IN      NS      k.gtld-servers.net.
com.                    172800  IN      NS      l.gtld-servers.net.
com.                    172800  IN      NS      m.gtld-servers.net.
com.                    86400   IN      DS      30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com.                    86400   IN      RRSIG   DS 8 1 86400 20210318050000 20210305040000 42351 . bVi/an3ya9VuX/O+2R5wTHP5+Ea7jmmQD+ZVs6rbmTpExiGl8Hsc8P+5 HSIbOcN9qcv/wnXoVwm8zLQojXWxJO4o4rkfAWI2fQ4ZvgEzZF5rxbmz DhOrXOexP7Yick8UqQpX8KADBrU6cH+jv1sYcc+pcDX0GzIq/LQV3bSa crTjtxBiqhYT8LD3d7bQ/kDbo6jyXMQTe77j2qFohW2+X3KBTpfFK6BZ iIrslY0OUYSCMqasCk9v5wSkM3qE0ebJlo71zcJVeGVaLEAEupS/HEzb ne+KSBIOMHJ3zSmZaFMXCZPSYmBAF2poNSh+L13Xpkf4Ib7w12PtWPUz BplviQ==
;; Received 1185 bytes from 198.97.190.53#53(h.root-servers.net) in 23 ms

schwab.com.             172800  IN      NS      ns1.schwab.com.
schwab.com.             172800  IN      NS      ns2.schwab.com.
schwab.com.             172800  IN      NS      ns3.schwab.com.
schwab.com.             172800  IN      NS      ns4.schwab.com.
schwab.com.             172800  IN      NS      a9-65.akam.net.
schwab.com.             172800  IN      NS      a8-64.akam.net.
schwab.com.             86400   IN      DS      3829 8 2 8B39D6D8CE4FA5D55DEB38CF05BB81E0CC087FA978AB9E0721411513 86CF2EA2
schwab.com.             86400   IN      RRSIG   DS 8 2 86400 20210309054915 20210302043915 58540 com. WCclyXLsxq4uaQpBB5WFJZvYbVNCra/EeN/AaBE+xVT0e+W9P0rJnWOM 1MdQ+FFdQDQndy9HQantJh7pOYsrroIrBDC84/MvvihnAzl0cSzUv8/1 zH95Rn0TGmyP1iGtUoBR9LTspXOy6vd6bsi3x8/J/KjzHco31YeBig1j nUSvSOG+w0gOx5XWq+1jkfh8rtIVTb8gDfDRc/muamDnNQ==
;; Received 481 bytes from 192.43.172.30#53(i.gtld-servers.net) in 24 ms

gslb.schwab.com.        900     IN      NS      gslb-anycast.schwab.com.
gslb.schwab.com.        86400   IN      DS      28456 8 2 D62CE9A0008171EE1F9DAC7A50AC167ADFCCF12A85C0314083F9CB86 8AC8C52F
gslb.schwab.com.        86400   IN      RRSIG   DS 8 3 86400 20210313094830 20210211090458 43563 schwab.com. ZaD1MLn/fOWaXgwZ6pyP2eKF5aG4t6fwjnRau/YF6zjigvfGHU+sNa26 qyzcFu2dnEUZsmnie2WDN4w7IhnkbzRUnzPN2Dkegj7gVvJ23UbkDOxP sQIxLWkog5okaUK9fv03Rh9pNk8pTEVUoSn/nnuPXrU57eJwscl2BJCc 6dzDuruTNE+wtmHe97tv3HZupWhyy4B5MpAKh6awWRBShpLmIE2NK0cR Hkwfo+Vb1cE2yfH6XTDQA/QeV1mBw32uvPQBT9Tp1ZGF6THjqZWyfaCV 1hsSN+KWavOgAjWxIt0OqJrfGewaQCQJDn5n0MrXQxB3ndoSxk/8/vYk wALTcw==
;; Received 741 bytes from 162.93.195.171#53(ns4.schwab.com) in 44 ms

workplace.gslb.schwab.com. 20   IN      A       162.93.221.50
workplace.gslb.schwab.com. 20   IN      RRSIG   A 8 4 20 20210308200738 20210301200738 46146 gslb.schwab.com. rjkuOJx+2tBnwv3Hm3CJEhHSxx4+NMzFuw1iNnPUTxewzx8RaqKdqX3K vIhGDCGoVIWJLeL/QiKvXnpulAIg1y3Aha9DCnsPNPJY4kJ61D3+PkeP Ygx3bEQETt+EFd+CIDjhgYlmZLkt5pkSMhONaPK4cXUBYBbPsoYW5b/u TZtzGcVaqmoRGbJgiildwfeqgykH+dER/tZ2E3/yIxvZnVnorcQFYPw9 t7F88iSOnSLg3253CHxu6iU8d/0dZcBU/Ta5vH4Qbba8sm2RNLLeHe/T u4glfkZRRey8KbPxoozRUOhsl/kXKQ8slAIcpfPZHtmEWncfkmfVPt+n BYcDKA==
workplace.gslb.schwab.com. 20   IN      RRSIG   A 8 4 20 20210311004437 20210304004437 16098 gslb.schwab.com. hdltHg4v0iOH6idgOMxXXWUSbvKeZHP3igqcERU9pMCuZWaQweIc8XEX z5QOoMhujJI9o3AdFDnBT9JVN/AQs90GbLT/SbPP6OQt2fCtVPFI+xCh 4bVVidFfFvfuTP36W7RNXc3FrfLyPJwyWRBCOHg/3UjN8E2+goVoU/Uw Ft4xmPFHJ5tYL8v7o9v/paICpSQgk7RcjjIsZZiKzN+BF8coCJNtT8DN WEohKJNt9Du+LZq8F59HjTa3g0PopOOhxu5tEzSHbs+IKPc4x3lYL25W nquvnEfVexEw81KfQB3smdi3CEY0yz/zqG8nbMb6QkxC9XQxi6b2iBbf n+JO2w==
;; Received 676 bytes from 162.93.239.1#53(gslb-anycast.schwab.com) in 46 ms


So what might be causing this problem?  The dig output seems like it is working ok?  But dig takes like 2.5 minutes to run, which does not seem normal.  I am guessing this is why unbound fails to resolve this domain and there is "No response."

On the other hand, if I try to dig a domain that unbound DOES resolve, such as "www.schwab.com," it ALSO takes like 2.5 minutes to complete.  And "www.schwab.com" resolves fine using DNSSEC turned on through the VPN tunnel.  Here is the output of DNS Lookup:

Response
Type Address
CNAME www.schwab.com.edgekey.net.
CNAME e17738.x.akamaiedge.net.
A 104.125.55.112

Resolution time per server
Server Query time
127.0.0.1 51 msec
1.1.1.1 6 msec
1.0.0.1 7 msec


Here is the output of dig "www.schwab.com".

root@OPNsense:~ # dig @127.0.0.1 www.schwab.com +trace

; <<>> DiG 9.16.10 <<>> @127.0.0.1 www.schwab.com +trace
; (1 server found)
;; global options: +cmd
.                       79654   IN      NS      j.root-servers.net.
.                       79654   IN      NS      k.root-servers.net.
.                       79654   IN      NS      l.root-servers.net.
.                       79654   IN      NS      m.root-servers.net.
.                       79654   IN      NS      a.root-servers.net.
.                       79654   IN      NS      b.root-servers.net.
.                       79654   IN      NS      c.root-servers.net.
.                       79654   IN      NS      d.root-servers.net.
.                       79654   IN      NS      e.root-servers.net.
.                       79654   IN      NS      f.root-servers.net.
.                       79654   IN      NS      g.root-servers.net.
.                       79654   IN      NS      h.root-servers.net.
.                       79654   IN      NS      i.root-servers.net.
.                       79654   IN      RRSIG   NS 8 0 518400 20210318050000 20210305040000 42351 . RGrSTUNk4Ad41ITau7wzwMrm6Uk/ReeJlR/1cul8D1bs7qdYZOeICUvX CU+j9KipCbh0VUKvbcVWXFlpWoy9k/4ay0u1ZB5BbooERfyfGVyTe4ru pXrXymKeFLetZFhUr2KoO6ITyigRPPNvJFkRhwUn6nHqgCiHEvdG2cZW FmmvFpZ+0ejIB1h7lJYg+iaG8be2tI3aXp3CF/u8Cerjii5DddESAZrL bR9K6SeeQB9GxabnQJMvFY2FXsHBps9BQkx6D1vc5Vpn8E7R4e3uIcte Rt0c7fwvOyZE1lwHsvhxIaXugLJdlSX0bWT5XwGtGFm3xo6OHuL2cqXJ 9HbxVQ==
;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms

com.                    172800  IN      NS      l.gtld-servers.net.
com.                    172800  IN      NS      b.gtld-servers.net.
com.                    172800  IN      NS      c.gtld-servers.net.
com.                    172800  IN      NS      d.gtld-servers.net.
com.                    172800  IN      NS      e.gtld-servers.net.
com.                    172800  IN      NS      f.gtld-servers.net.
com.                    172800  IN      NS      g.gtld-servers.net.
com.                    172800  IN      NS      a.gtld-servers.net.
com.                    172800  IN      NS      h.gtld-servers.net.
com.                    172800  IN      NS      i.gtld-servers.net.
com.                    172800  IN      NS      j.gtld-servers.net.
com.                    172800  IN      NS      k.gtld-servers.net.
com.                    172800  IN      NS      m.gtld-servers.net.
com.                    86400   IN      DS      30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com.                    86400   IN      RRSIG   DS 8 1 86400 20210318050000 20210305040000 42351 . bVi/an3ya9VuX/O+2R5wTHP5+Ea7jmmQD+ZVs6rbmTpExiGl8Hsc8P+5 HSIbOcN9qcv/wnXoVwm8zLQojXWxJO4o4rkfAWI2fQ4ZvgEzZF5rxbmz DhOrXOexP7Yick8UqQpX8KADBrU6cH+jv1sYcc+pcDX0GzIq/LQV3bSa crTjtxBiqhYT8LD3d7bQ/kDbo6jyXMQTe77j2qFohW2+X3KBTpfFK6BZ iIrslY0OUYSCMqasCk9v5wSkM3qE0ebJlo71zcJVeGVaLEAEupS/HEzb ne+KSBIOMHJ3zSmZaFMXCZPSYmBAF2poNSh+L13Xpkf4Ib7w12PtWPUz BplviQ==
;; Received 1174 bytes from 192.203.230.10#53(e.root-servers.net) in 5 ms

schwab.com.             172800  IN      NS      ns1.schwab.com.
schwab.com.             172800  IN      NS      ns2.schwab.com.
schwab.com.             172800  IN      NS      ns3.schwab.com.
schwab.com.             172800  IN      NS      ns4.schwab.com.
schwab.com.             172800  IN      NS      a9-65.akam.net.
schwab.com.             172800  IN      NS      a8-64.akam.net.
schwab.com.             86400   IN      DS      3829 8 2 8B39D6D8CE4FA5D55DEB38CF05BB81E0CC087FA978AB9E0721411513 86CF2EA2
schwab.com.             86400   IN      RRSIG   DS 8 2 86400 20210309054915 20210302043915 58540 com. WCclyXLsxq4uaQpBB5WFJZvYbVNCra/EeN/AaBE+xVT0e+W9P0rJnWOM 1MdQ+FFdQDQndy9HQantJh7pOYsrroIrBDC84/MvvihnAzl0cSzUv8/1 zH95Rn0TGmyP1iGtUoBR9LTspXOy6vd6bsi3x8/J/KjzHco31YeBig1j nUSvSOG+w0gOx5XWq+1jkfh8rtIVTb8gDfDRc/muamDnNQ==
;; Received 470 bytes from 192.35.51.30#53(f.gtld-servers.net) in 21 ms

www.schwab.com.         300     IN      CNAME   www.schwab.com.edgekey.net.
www.schwab.com.         300     IN      RRSIG   CNAME 8 3 300 20210313110153 20210211103625 43563 schwab.com. eVem19JCDHIfAz3hu6smc3auF2TyWg7utEy+a43wF2Mo7cODhRsxqCvw hEffohd3bn3/INLkvuMWp7Ep4tIZD/EvQDSBzA0MYpXHUJZaCkY8j1iJ 3l2A3sO9f/ovDRAM4H0ZB6thgTErDDFpNPXVvqR2C8begFeL7M07/MZM M8eIc4tLpLDXFXKzkJk9h3Dg28xN5esKKIO7eEKS5IJEBom5YqUetHaz vwSDQQSltpHj3FR9kK6tz2AcuvtVIs/02Z0ZusbtVUNUDpozDFb3B/39 kVp87DUeFMMYaRETMAxK6lfAmlKZRpTT9cjia/qn2LkNmWzfS9qgpM4s n986XQ==
;; Received 381 bytes from 162.93.253.90#53(ns1.schwab.com) in 43 ms


ANY IDEAS? 
#11
Yes, I am coming to the conclusion that this cannot be done using HAProxy after additional searching.  The main issue is that Minecraft is not an HTTP protocol, and you cannot authenticate using a TCP proxy.  This seems strange to me as I would think you can authenticate via HTTP, then remember the IP address that authenticated, and then allow only that IP address to go through the TCP proxy.  It seems like this would be a desired function as not everything is HTTP.

I'm aware that this can be done with VPN and I can lock it down with firewall rules so that users can only access the Minecraft server.  I already have an OpenVPN server running for my own remote access.  But I do not want to give VPN access to my kids' school friends because (1) it's too complicated, I would have to issue them certificates and they would have to set it up on their end, (2) hard to cut people off if I want to; I'd have to revoke their certificates (vs. just deleting their user account).  This is why I would prefer a user / password scheme rather than a VPN that requires certificates and complicated set up. 

I'm not looking to give users fully secured / encrypted access to the internal network -- I just don't want the whole world to have access to the Minecraft server.

So are there alternatives to VPN?  Are there VPNs that don't require certificates?  Is there like a scheme where you have port forwarding conditional on user authentication? 


Quote from: cmdr.adama on December 09, 2020, 10:42:19 AM
So, First thing's first.

You won't be able to do this with HAProxy.

Really the only way you can achieve what you want is to sit the MC server behind a VPN.

So, you'll need to set up a VPN server, OpenVPN, WireGuard, etc.
Shift your MC server to sit in a DMZ, if you haven't already and then point all VPN connections to the DMZ.

If you want, you can also configure unbound to allow the users connecting to the VPN to use hostnames instead of IP addresses.
#12
After some searching, it seems that because Minecraft does not use HTTP, you cannot use Host_Minecraft condition.  That is, you can't depend on hostname = minecraft.domain.xyz to trigger the rule.

As a result, I have changed the rule Minecraft to equal IF Auth_User (i.e., the user successfully authenticates).

I think the solution must involving setting up a second frontend:

Public Service
Name:  Frontend_Minecraft
Listen Addresses:  0.0.0.0:25565
Type:  TCP
Default Backend Pool:  none
Rule:  Minecraft (this is basically just equal to IF Auth_User)

Unfortunately, the above does not seem to work.  I am able to connect to the Minecraft server from outside the network if I change the above public service to have Default Backend Pool = Minecraft.  But if I do that, then you are able to connect to the server regardless of whether you authenticated.

What I can't figure out is how to link the two Public Services.  I have one public service at port 12345 that just does HTTP authentication, and that works correctly and uses a proper SSL certificate.  I have another public service at port 25565 that connects to the Minecraft backend.  It works too, but I am not able to make the Minecraft backend conditional on authenticating with the first Public Service.

Any ideas on how to do this?
#13
One thing I noticed that is in my "Public Service" settings, I did not set anything under "Select Rules".  But when I try to select "Minecraft" rule inside that setting, HAProxy says there is a critical configuration error: 

http frontend 'Frontend' (/usr/local/etc/haproxy.conf:48) tries to use incompatible tcp backend 'Minecraft' (/usr/local/etc/haproxy.conf:81) in a 'use_backend' rule (see 'mode').

I am guessing this means that the Public Service "Type" should be set to something else, rather than "HTTP / HTTPS (SSL offloading) [default]"?  But I don't know  how to set this up.  I want to keep the HTTPS user authentication, but pass through TCP traffic.
#14
I have a personal Minecraft server on the LAN that I run for my kids.  They want to allow cousins / school friends to play on our server.  For security reasons, I don't want to just port forward and allow public access to the Minecraft server, nor do I want these people access to my network via VPN.  I want an in between solution where people can access the Minecraft server after HTTPS user / password authentication.  I thought HAProxy would work for this, but have not gotten it to work.  Any guidance would be greatly appreciated!

End Goal:  I want people to go to some web address (e.g., https://minecraft.domain.xyz:12345) and authenticate using a user / password that I give them.  After that, they are able to connect to my server by connecting to minecraft.domain.xyz inside Minecraft.  If people do not first go to the URL to authenticate, they should not be able to connect via Minecraft.  I understand Minecraft uses port 25565.

Here is my set up so far:

1)  I installed the Let's Encrypt plugin.  I purchased my own domain (domain.xyz) and have successfully issued a wildcard certificate for domain.xyz and *.domain.xyz.  In the Let's Encrypt plugin, I do NOT check "HAProxy Integration" because I understand that is only needed if I use HTTP-01 validation and I don't use that method.

2)  I use Dynamic DNS to set domain.xyz and minecraft.domain.xyz to equal my WAN IP address.

3)  Here are my HAProxy settings:

Real Server
Enabled:  Checked
Name:  Minecraft
IP:  192.168.1.90
Port:  25565
Mode:  active [default]
SSL:  Unchecked

Backend Pool
Enabled:  Checked
Name:  Minecraft
Mode:  TCP (Layer 4)  --> my understanding is that this should be set to TCP because Minecraft is not a webserver
Balancing Algorithm:  Source-IP Hash [default]
Servers:  Minecraft
Enable Health Checking:  Checked
Health Monitor:  None
Persistence Type:  Stick-table persistence [default]
Stick-table persistence table type:  Source-IP [default]

Users / Group
I created a single test user / password.
I added this single user to a test group.

Conditions
Name:  Host_Minecraft
Condition type:  Host matches
Host string:  minecraft.domain.xyz

Name:  Auth_User
Condition type:  HTTP Basic Auth:  username/password from client matches selected user/group
Parameters:  matches to my test group.

Rules
Name:  Minecraft
Test type:  IF [default]
Selected conditions:  Auth_User AND Host_Minecraft
Execute function:  Use specified Backend Pool
Use backend pool:  Minecraft

Public Service
Name:  Frontend
Listen Addresses:  0.0.0.0:12345  I don't know if 0.0.0.0 is the right address to use here
Type:  HTTP / HTTPS (SSL offloading) [default]
Default Backend Pool:  none
Enable SSL offloading:  Checked

SSL Offloading:
Certificates:  wildcard certificate from Let's Encrypt
Default certificate:  wildcard certificate from Let's Encrypt
Enable Advanced Settings:  Unchecked

HTTP(S) settings:
Enable HTTP/2:  Checked
HTTP/2 Without TLS:  Unchecked

Basic Authentication:
Enabled:  Checked
Allowed Groups:  my test group

Firewall rules
On the WAN, I allow IPv4 TCP/UDP protocol to pass at port 12345.

Here is what happens:

1)  Using my browser, I am able to go to https://minecraft.domain.xyz:12345, it gets a user/password prompt, and I able to "login" using my test user credentials.  The connection is properly secured using the Let's Encrypt certificate.  After login, the browser shows an error message because there is no webserver at that location.  But I don't care.  I just want to satisfy the Auth_User condition.

2)  I open Minecraft and add the server minecraft.domain.xyz, and I try to connect, but it does not work.  I thought this would work because I thought this would satisfy the Host_Minecraft condition.

So what am I doing wrong?  I am able to get the user authentication working, but HAProxy is not correctly passing traffic to my Minecraft server.  I am guessing something is wrong with my "Public Server" settings, but am not sure what.
#15
Yes, I have a couple of instances of OpenVPN server running for remote access to my network, one on TCP and one on UDP.  I connect using certificates, so it's pretty secure.

Quote from: Fright on September 10, 2020, 04:02:40 PM
QuoteBased on this, I'm not sure MSS is the fix for this problem.  Any other ideas?
agree.
but something strange in this strings:
162.142.125.35:47916 WARNING: Bad encapsulated packet length from peer (5635)
192.35.168.193:58604 TLS Error: tls-crypt unwrapping
its port-scanning bots knocking on your ports
what is your OpenVPN config? server enabled? which port it use?