resolving issue (?)

[opn_minded]

Hi,

I've a strange issue since 18.09.2021, 01:00CEST (right after the CRON for unbound-blocklist renewal was triggered). One of my apps would send notifications via telegram (so this app needs to connect to api.telegram.org), but since then it simply won't work.

I can't connect to api.telegram.org from any device/browser in my home-network, while from my mobile phone (via LTE) it resolves fine. Also, when I use my mobile phones' hot-spot it immediately works, even /wo dns-flush on my laptop.

• I can (successfully) PING api.telegram.com from my home-network.
• I can (successfully) TRACERT api.telegram.com from my home-network (it doesn't get stuck somewhere internally).
  
• When I try to open api.telegram.com from my home-network, I see in the firewall-log that it forward the DNS-query to unbound..
• ..and in unbound I see the info for A IN/AAAA IN

unbound:
The (only) blacklist I'm using is https://block.energized.pro/ultimate/formats/domains.txt, but it doesn't contain api.telegram.org, anyhow I white-listed api.telegram.org in unbounds' blocklist-settings.

I've also searched /usr/local/etc/unbound.opnsense.d/dnsbl.conf for "api.telegram.org", but it doesn't appear.

I use cloudflare's 1.0.0.2 and 1.1.1.2 for DNS over TLS.

Do you have any ideas how to solve this? Many thanks in advance for your time.

OPNsense 21.7.2_1-amd64 /

[abulafia]

Cloudflare could be blocking it.

But what's stage is that you seem to be able to ping it under its domain name, so it does get resolved?

Anyway, since I had my own issues with unbound: check whether strict qname minimisation is enabled and if so turn it off - it's a likely culprit.

[opn_minded]

hi abulafia - and many thanks for taking the time to reply.

But what's stage is that you seem to be able to ping it under its domain name, so it does get resolved?
yes, indeed. it get's "somehow" resolved - but i'm not able to connect.

what's even more strange is that if i use unbound to query cloudflare, it works:

• nslookup api.telegram.org returns "149.154.167.220"
• nslookup telegram.org returns "149.154.167.99"

if i use cloudflare to query directly..

• nslookup api.telegram.org 1.1.1.2 returns "149.154.167.220"
• nslookup telegram.org 1.1.1.2 returns "149.154.167.99"

if i use google's dns server (just for testing)..

• nslookup api.telegram.org 8.8.8.8 returns "149.154.167.220"
• nslookup telegram.org 8.8.8.8 returns "149.154.167.99"

so, it's not a cloudflare issue.
at this point, "connecting" to api.telegram.org doesn't work on any on my machines (neither win, nor *nix) - only after i e.g. connect a laptop to my mobile phone's LTE hot-spot, so the issue must be somewhere in my network :(. i didn't make any changes whatsoever at the point where the errors started to occur.

check whether strict qname minimisation is enabled and if so turn it off
Strict QNAME minimisation - was always OFF.

[karlson2k]

If you can ping or tracert host by DNS name then you don't have any DNS resolution problems.

Use 'nslookup' or 'drill' to check whether DNS name is resolved correctly to IP address.

If IP address is correct, use 'curl -v https://your_url' or even 'curl --trace - https://your_url' to find what's wrong (Telegram API is HTTP-based).

Your provider may block access to specific IP. You will not notice single IP blocking in Telegram client, because it uses anti-blocking features, but access to specific hosts could be blocked.

[opn_minded]

hi karlson2k - also thx to you for replying here!

i already did nslookups in my second post, IP is correct and cURLing would return;
== Info: Recv failure: Connection was reset
== Info: Closing connection 0
curl: (56) Recv failure: Connection was reset

i also wifi-ed in my modem to see if it's actually my ISP or opnSense causing that issue => from my modems' wifi, i can access api.telegram.org, so the rootcause must be something within opnSense. and i have the strong feeling that something in unbound messes up.

EDIT:
- in the fw-log, i can see (outbound) traffic to 149.154.167.220:443 - that is api.telegram.org
- if i put https://149.154.167.220 in the browser, it connects (fwd to https://core.telegram.org/)

it's still not solved, but we're narrowing it down :)

[abulafia]

Not an unbound issue - it only does DNS. And your resolves work apparently.

Do you use IP blocklists?

[karlson2k]

Enable all firewall logging on opnSense.

"Connection was reset" means that connection is actively blocked otherwise you would just get a timeout.
It's still possible that connection is blocked in outer net.

You can also try to SSH to your opnSense host and try curl from it.

[opn_minded]

hi,

@abulafia - yes, as stated in my initial post, i am using blocklist - but only one: https://block.energized.pro/ultimate/formats/domains.txt. it doesn't contain "api.telegram.org", nor does the compiled /usr/local/etc/unbound.opnsense.d/dnsbl.conf.

@karlson2k - thx for mentioning, firewall logging is active.
this is the result of curl-ing directly from opnsense:

Code Select Expand

curl --trace - api.telegram.org
== Info:   Trying 149.154.167.220:80...
== Info: Connected to api.telegram.org (149.154.167.220) port 80 (#0)
=> Send header, 80 bytes (0x50)
0000: 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a GET / HTTP/1.1..
0010: 48 6f 73 74 3a 20 61 70 69 2e 74 65 6c 65 67 72 Host: api.telegr
0020: 61 6d 2e 6f 72 67 0d 0a 55 73 65 72 2d 41 67 65 am.org..User-Age
0030: 6e 74 3a 20 63 75 72 6c 2f 37 2e 37 38 2e 30 0d nt: curl/7.78.0.
0040: 0a 41 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a 0d 0a .Accept: */*....
== Info: Recv failure: Operation timed out
== Info: Closing connection 0
curl: (56) Recv failure: Operation timed out

[Fright]

so name resolution actually works. but the connection does not (connection to api.telegram.org:80 should result with http code 301 and redirect to https)

[sja1440]

Just a few thoughts.

(1) Do you have Intrusion Protection (i.e. Suricata) configured? My understanding is that on OPNSense Suricata acts after the firewall (pfilter) on outgoiing connections.  If Suricata was configured to drop your particular connection then I guess you would see "== Info: Recv failure: Operation timed out" at the curl level and indeed the firewall level.

(2) Does a wireshark analysis of the traffic show anything odd? Maybe comparing a capture from the WAN side with one of the lan side and one of the wifi side (since you say that devices connected to your wifi don't have the problem)

[opn_minded]

hi there @sja1440 - that's a bingo!

suricata blocked, exactly as you stated. there's a SID #2033967 (ET INFO Observed Telegram API Domain (api .telegram .org in TLS SNI) that would identify packets as "harmful".

it seems this sid has been created 2021_09_16 and suricata loaded it the first time (on my opnsense) on the 18th.

i created an exception via policy > rule adjustments (alerting only), now it works again as intended.

to all of you who took the time and helped me out here, a big thank you guys.
have a great day!