Issues with android app

[Minskaya2]

Hello every one,

I have installed my opnsense box a few time ago and since the beginning i have an issue with the android devices on my network.
They are all connected to wifi by an access point connected by ethernet on my network.
If we use them to browse with http/https sites, all is fine . Youtube App is also working.
But a lot other apps are not working. It seems theses apps are having some time out : bank app, Deezer app, Microsoft authenticator, ...
As soon i deconnect the device from wifi they are working properly.
All the phones and tablet are concerned.

What confuses me it's the lack of logs to make a diagnostic: i don't see in the firewall logs any blocked queries for these devices when i made some test, neither in the dns logs.

Anybody could give me some advice to investigate my problem as i am searching from several months without any success.
I have already checked my firewal rules, dns configuration, ipV6 configuration.
I have made a lot of tries, all have failed. Thanks to the backup configuration functionality.

I thank in advance everyone who agrees to help me because I have exhausted all my ideas.

Mins

[Minskaya2]

No one has an idea of what i can do to log more accurately what's happen between a specific device and my router while i make some tests to diagnose my problem ?

Mins

[Minskaya2]

Ping

[cookiemonster]

The problem is that there are too many variables to consider and nobody knows your setup. You need to narrow down the problem, post your relevant setup. if you don't know what is relevant, imagine how everyone else in the forum is unknowing of it :)
I'd start with DNS. What's your DNS setup? The whole of it: what provides DNS to your network (is it OPN dhcp, unbound, dnsmasq, pi-hole in another device), what is your network infra like, VLANs, switches, etc.
And yes, there are separate logs for separate services. So my advice is grab a pc/laptop into the wifi and start diagnosing from it. Ideally not Windows (why I hear you ask? because I for one don't know how to use diagnostics with commands from it).

[Minskaya2]

Hi,

I'm aware my problem is not easy to investigate.
That's why my last question is what i can activate as logs in my opnsense box to see where the trouble begins because currently i see no request failed neither in dns or firewall logs.

My setup is this one :
- my opnsense box is a little vault protectli like this one : https://eu.protectli.com/vault-4-port/ My mistake was to think opnsense is an all in box system and the dns was provided.
- the Dns is UnboundDns running on the vault
- the vault is linked to an access point netgear WAX 214 like this one https://www.netgear.com/business/wifi/access-points/wax214/
- i don't us any vlan

What i noticed is the problem seems to affect all the android devices using wifi.
For example when i connect a smartphone to my pc to diagnose the problem using adb, i used to share the pc network connection to the device and the problem suddenly vanished.

All the devices acquire an ip v4 and ip v6 from the dhcp running on the vault but I haven't been able to determine if the issue is related to ipv6 or not but ping -4 and ping -6 to google are ok.


I know this is a difficult question so i thank you very much for any help you can provide.

Mins

[cookiemonster]

Different services each has its own log. They are mostly in /var/log/. For instance Unbound is /var/log/resolver/latest.log. This one has a UI to look at too. Then there are settings for the service to increase the verbosity and include additional messages: Services > Unbound DNS > Advanced. There you can dial up the logging whilst diagnosing. Uses more storage so be sure to reduce it later.
So if you were to diagnose DNS, you could increase this logging for failures like NXDOMAIN. If you can see the name resolution when using the app, then you know the name resolution part is OK.

Then you can move to the firewall side. Similarly you can look in the UI for the incoming requests.
Additional logging which is default off is in System > Settings > General. Again, careful with storage. Go back to defaults afterwards.Tooltips will help.

Finally, the problem could be with IPv6 which I don't use, so can't advise on that.

[Minskaya2]

Could the DNS silently drop the queries ?
I observe the clients waiting a long time before displaying an error as if they were waiting the name resolution without never getting it.
In case there is an active functionality dropping the queries (i think a kind of adblock), Would this cause this kind of issue ?

Mins

[Minskaya2]

I have increased the loglevel and i am trying to forward them to another host with some sort of kibana to analyse them more easily

Work still in progress ...
Mins

[cookiemonster]

> Could the DNS silently drop the queries ?
Not by default.
You should describe your full infrastructure setup. There might be other elements in play.
Everybody else can just plug a wireless AP into a switch that is connected to OPN by ethernet and has no problems:
Internet -> WAN - OPN - LAN-> switch -> AP -> wifi clients
                                                 |
                                                 ---------------> wired clients

[Minskaya2]

Indeed that's exactly my setup :

Internet -> WAN - OPN - LAN-> switch -> AP -> wifi clients
                              |                  |
                              |                  ---------------> wired clients
                              - OPT1-> Synology

[MarieSophieSG]

Same here !
set-up:
LAN1 192.168.101.101/24 => Switch1
LAN2 192.168.102.101/24 => WiFi AP
LAN3 192.168.103.101/24 => Swicth2

Fpor some reason each LAN can't communicate with the others (but that's another thread)

Devices on WiFi AP are all Android or IoT, and all have full Internet access.
But I have 2 app that can't connect to Internet, but can't see/find any packet dropped, any blocked traffic (or I don't look in the right place)

FW rules are all defaults
IDS/IPS on/off doesn't change anything
FW disable leads to no Internet access, so I couldn't try without it :/

[cookiemonster]

> Fpor some reason each LAN can't communicate with the others (but that's another thread)
Only first LAN interface has an allow all out rule. New interfaces and networks need it creating explicitly.

> Devices on WiFi AP are all Android or IoT, and all have full Internet access.
> But I have 2 app that can't connect to Internet, but can't see/find any packet dropped, any blocked traffic (or I don't look in the right place)
If all else works on the Android device except the app and no other services on the firewall enabled, then it suggests the problem not on OPN, no ?
Zenarmor enabled ?

[MarieSophieSG]

> Fpor some reason each LAN can't communicate with the others (but that's another thread)
Only first LAN interface has an allow all out rule. New interfaces and networks need it creating explicitly.

Yes, they have it, the two extra rules "Allow all" on LAN1 cloned to each other LAN

> Devices on WiFi AP are all Android or IoT, and all have full Internet access.
> But I have 2 app that can't connect to Internet, but can't see/find any packet dropped, any blocked traffic (or I don't look in the right place)
If all else works on the Android device except the app and no other services on the firewall enabled, then it suggests the problem not on OPN, no ?
Zenarmor enabled ?

No Zenarmor, no extra application, it's pretty much all stock,
The only mod I did is to anable IDS/IPS, but as said in previous, disabling it didn't change anything :/

I'm pretty sure (as it is a secured app, like banking or gouvernement or crypto or such) it's a matter of port and layer used and being bloked, but I can't see which

[cookiemonster]

Ah!. Unlikely unless you hare doing something wacky with certificates and breaking TLS.
Some of those use certificate pinning.
To me the next step in diagnostic is to do a packet capture and analysis.
You are using Unbound, right ?
And do they (the apps) give some error or some indication of the problem?

[MarieSophieSG]

Ah!. Unlikely unless you hare doing something wacky with certificates and breaking TLS.
Some of those use certificate pinning.

If it is so, then it's completely out of my wish/controle, I just use the tablet the same way I did while connected to my previous router, which was much of a strainer, letting most going through, hence me on OPNsense now

To me the next step in diagnostic is to do a packet capture and analysis.

I would be happy to oblige, using the search I found "packet capture" and set it up to interface LAN2, IP 192.168.102.103 (Tablet)
And will post it below

You are using Unbound, right ?
And do they (the apps) give some error or some indication of the problem?

Unbound, yes
with or without blocklist (AdWare, ...) doesn't change anything
The app asks for passphrase, then spin for about a minute and then drop saying: -"Sorry, there was an issue processing your request, please try again later" kinda standard msg