Messages

Suppose you have a schools community with 10 schools at different locations, with more than 1000 persons. It is not prohibited to tie the data to a location, it is prohibited to tie the data to individual persons.

Which highlights the sort of assumption that gets people into trouble.  If data collection assumes that linking data to a location is safe then it may tie data to individuals in homes, small business, or people working late or on weekends.

More obscure issues arise when product X identifies data by location and so does product Y, and if an attacker has access to supposedly anonymised data from both it can link the locations and draw conclusions not available otherwise.  If you follow the various security blogs you can read about researchers identifying individuals by linking several supposedly anonymous data sources.

Of course services may need to collect some data, but the security and privacy issues are not simple, and they are ongoing.

The only problem is if you tie such data to a person. If it is anonymised, it is not a privacy matter.

But it is.  The effectiveness of security measures changes over time and attacks only ever get better.  What was thought to be effective anonymisation today may turn out to be not-so-effective tomorrow, perhaps because someone has been very clever in their analysis of the metadata, or because an attack was made on the data in transit to anonymisation, or because mission-creep has introduced additional metadata that allows more dots to be joined.  Security and privacy are an ongoing concern.

First, I thought explaining the connection situation might let someone who knows jump straight to what I suspect might be conclusion: you cannot reliably shape this.

I operate from a remote location via a 4G wireless connection.  In simple terms:

   WiFi----\
          OPNsense---4G---ISP---Internet
   LAN-----/

The quality and speed of the 4G connection varies significantly with the weather and time of day.  Download: rarely I might see 25Mbps, most of the time I see 8Mbps..15Mbps (it really does vary that much even over a brief period), but sometimes it will drop to 3Mbps or even lower.  Upload: tends to be more consistent, anything from 7Mbps to 20Mbps; it regularly tests faster than download.

How do you define shaper pipes to encompass such variability?  I don't want to constrain the traffic unnecessarily, which would seem to exclude the possibility of dedicated pipes for certain traffic (unless I want to constrain the total traffic down to a very low value).

Without reliable pipe definitions, I have trouble seeing how the shaper can make appropriate choices, most especially as it relates to download traffic (which is where I see the worst of my problems).


What I wanted to achieve:

My mobile phone currently uses a VoIP app for a business line, and WiFi Calling for mobile calls.  (The office 4G wireless uses an antenna on an 11m mast, so it gets a much better connection than the mobile phone.)  During problem times (Internet being used as well as calls) the download side of the call in particular seems to experience lengthy transmission delays (up to 5 seconds) and sometimes whole words or sentences get dropped.  VoIP is worse than WiFi Calling.  So I was hoping to use the traffic shaper to prioritise calls over other traffic.

The VoIP traffic is easy to identify, I have a list of IPs used by the service provider.  Watching the traffic graph shows it using around 120kbps each way, and both ways are constantly active while a call is active, so utilising 240kbps even if no one speaks (which is higher than I was expecting).

WiFi Calling uses IPSec, but is not the only IPSec service being passed through this firewall, however, setting up shaper rules based the ports (4500 and 500) and the mobile IP address appears to do the job in this case.  WiFi Calling seems more efficient than VoIP, sending traffic only while speaking, but this makes it harder to get a direct measure of its bandwidth requirement.


What I have tried:

I tried setting up a dedicated pipe for VoIP but that seemed to be the worst for download delays - I think because I had oversized the lower priority pipe (so Shaper tried to give it bandwidth it assumed must exist but didn't at the time).  Using queues appeared to make some difference, but often not enough to make for reliable phone calls.

I started off using separate upload and download pipes but was still seeing problems.  When I experimented with running multiple network speed-tests it seems that upload does significantly influence download (at least in those tests, I could not find a test that does proper full duplex speed testing).

So then I moved to a single pipe and just prioritise queues inside that single pipe (up and down).  While it's unclear how much this helped, at least it was simpler.  It made it easier, for example, to test reducing this pipe down to lower speeds.  When I cut the pipe to 5Mbps and ran a speed test while talking, I saw very little delay and break-up in the voice call, but - obviously - the total bandwidth was reduced to under 5Mbps (the speed-test I ran while on the call showed 4.6Mbps).


Possibilities?

* Operate using a shaper pipe defined to the lowest commonly available speed.  Not really an option, but it would be possible.

* If I'm sitting at my desk when a call comes in I could bring up the Shaper and set the pipe according to current connection quality.  I'm not sure how dynamically the shaper responds, but I'm sitting at my desk enough of the day that this is probably feasible if not attractive.

* Maybe there are some advanced options in the Shaper that apply to this situation?


Suggestions welcome.

This turns out to be my mistake/misunderstanding.  :-[   (Surprise, surprise, surprise.  ;))

After much research, including reading some docs from pfSense, which cover packet capture in more detail (like where it actually happens) than the OPNsense docs, I now know more than I did.

Yes, while ever I'm looking at the LAN interface it looks like there are active connections. (And looking at the LAN interface is what ZenArmor is looking at in my set up; what I used for my previous packet capture; and what I see when look at connections in the VM.)  But apparently these connections are lies told by Captive Portal: it is spoofing the connections to try and make redirections happen smoothly in the browser.

I thought I'd seen corresponding traffic on the WAN interface but probably saw something resulting from other hosts on the network.  Tonight, while the network was quiet, I captured LAN and WAN simultaneously and then stepped through the results in WireShark and was able to verify that none of the Internet hosts that are reported on the LAN side as having connections when starting Firefox or Vivaldi actually appear on the WAN side at all.  If there are more connections now than what I've noticed before, it could be because things changed on the browser (certainly Firefox seems to have something more elaborate going on than the last time I used it).

So my apologies to anyone whose time I have wasted.

Tracking things across the NAT is where I always stumble.  I've yet to find a tool that makes this easy/obvious.

I am indeed using Routed Mode (with Cloud Threats turned off) at the moment, at least while I wait to hear back from sy, because the confusion in Passive Mode makes using the product difficult.

I started with Passive Mode because that's all I wanted and I thought it might have less overhead (but don't know if that's true).  I have a very small network, just a few users and occasional guests (they're not encouraged  ;) ), and for now at least I am content with the Unbound DNS blocklist feature for blocking stuff.

What I find myself looking for most regularly is "what is (or was) causing that traffic?"  It seems to be the Sessions Explorer in ZA that most easily answers that question.  The normal reporting options in OPNs and ntopng show parts that you have to join together yourself (unless I'm missing something obvious, it wouldn't be the first time).  What I'd really like was a product that took me directly from "bump in traffic graph" to "sessions active at that moment".  ZA comes close.

Thanks for your input BNaCl.  It sounds like there maybe overlap between what you see and what I see, but probably more than one issue.  However, your mention of bridges made me think to add (just in case it turns out to be relevant):

The WAN interface in my set up is connected to a 4GB modem set in a transparent bridge mode (its default) rather than router mode.  This has caused me some issues getting the WAN address updated when modem reboots or whatever, mostly solved with a cron script now I hope, but I doubt if this is related to what I'm seeing in ZenArmor.

Thanks for the input, I had not thought to look in the client app config.  Turning the captive portal Enabled option to false prevents that button bar from popping up asking to log in to the network, and I think means there are one (or maybe a few) less Internet connections established before log in, but there are still some connections established.

So I have a Windows 10 VM.  It does not have an exception in Captive Portal and is not currently logged in.

I start a packet capture on the LAN interface, specifying that VM's IP address and port 443 (most of these connections appear to be happening on 443).

Then I start Firefox and I get the little bar at the top asking me to log in to the network, but I ignore it.  I wait a few seconds (during which I look at the TinyWall "connections" display and see a number of Internet connections showing as "Established").  Then I close Firefox.

Here is a small sample from the start of that process showing requests going out from 10.xxx.yyy.151 and responses coming back from 34.117.237.239 (Google apparently).

Code Select Expand

11:55:28.109210    ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 128, id 5544, offset 0, flags [DF], proto TCP (6), length 52)
    10.xxx.yyy.151.55784 > 34.117.237.239.443: Flags [S], cksum 0x30c9 (correct), seq 3417154090, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
11:55:28.109275    ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
    34.117.237.239.443 > 10.xxx.yyy.151.55784: Flags [S.], cksum 0x8a81 (correct), seq 4183075335, ack 3417154091, win 65228, options [mss 1460,nop,wscale 7,sackOK,eol], length 0
11:55:28.109576    ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 5545, offset 0, flags [DF], proto TCP (6), length 40)
    10.xxx.yyy.151.55784 > 34.117.237.239.443: Flags [.], cksum 0xa90b (correct), seq 1, ack 1, win 8212, length 0
11:55:28.110918    ethertype IPv4 (0x0800), length 571: (tos 0x0, ttl 128, id 5546, offset 0, flags [DF], proto TCP (6), length 557)
    10.xxx.yyy.151.55784 > 34.117.237.239.443: Flags [P.], cksum 0x68e3 (correct), seq 1:518, ack 1, win 8212, length 517
11:55:28.110958    ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    34.117.237.239.443 > 10.xxx.yyy.151.55784: Flags [.], cksum 0xc51c (correct), seq 1, ack 518, win 510, length 0
11:55:28.112754    ethertype IPv4 (0x0800), length 1514: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 1500)
    34.117.237.239.443 > 10.xxx.yyy.151.55784: Flags [.], cksum 0xf5c3 (correct), seq 1:1461, ack 518, win 514, length 1460
11:55:28.112782    ethertype IPv4 (0x0800), length 1514: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 1500)
    34.117.237.239.443 > 10.xxx.yyy.151.55784: Flags [.], cksum 0x6ea4 (correct), seq 1461:2921, ack 518, win 514, length 1460
11:55:28.112800    ethertype IPv4 (0x0800), length 675: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 661)
    34.117.237.239.443 > 10.xxx.yyy.151.55784: Flags [P.], cksum 0x5a89 (correct), seq 2921:3542, ack 518, win 514, length 621

Certainly looks like a conversation happening to me.

If I put an actual Firewall rule in to block 10.xxx.yyy.151 then Firefox never displays the bar prompting me to log in to he network and the TinyWall connections display shows Firefox attempting to make connections but the state never gets past "SynSent".

Much the same sort of thing happens when I use Vivaldi (Chromium).

I saw something similar on a system running an Apache server, where it made some Internet connections despite that host being blocked by Captive Portal.  But I am having trouble replicating this.

Presumably what is getting through is deliberate.  The fact that the login prompts never appear if the device is completely blocked even suggests that some of the connections are necessary, but it wasn't what I expected ... and I'd be interested to know what the rules are: what is allowed through without authorisation?

Thanks for the suggestion.  I had seen that Cron job but wasn't clear on what it did, and posts here indicated people were using it once-a-day at 3 or 4 am to reset PPPoE connections.  It didn't seem likely it was appropriate to run it every 10 minutes on the off-chance the WAN was down.

However, and maybe this was your point, I found the script that "Periodic interface reset" calls and tested calling it from inside my script - and it does seem to do what I need.  Thank you very much.

So now my script pings a few different addresses and if they all fail, or if the WAN address as reverted to the NetGear default (192.168.5.*) then I call "/usr/local/etc/rc.configure_interface wan" and it does its thing and the OPNsense interface reports consistently throughout, which is great.

I know there is an option to block accepting that NetGear address, but then it also blocks access to the modem management interface until a full connection comes back.  I like that it might revert to that and let me still see the modem even if the outer network is still down.

In the user interface, under Interfaces, Overview, you can expand out the WAN interface and get offed "Reload" and "Renew" buttons that do what I need.   How do I do this in a script?

I've been searching all night.  dhclient in OPNsense doesn't have a -r option, and anyway, running that appears to result in multiple instance issues; sometimes it looks like it has worked (when you do an ifconfig) but the change never appears on the user interface, so something gets missed, or maybe it's because it's running a separate instance.


Yes, I'm talking the years old problem, known both here and on pfSense, of some modems (in this case NetGear) that don't appropriately notify the DHCP client (whose address comes over the bridge provided by the modem) when the address has changed as the result of a reboot of the modem, or other disruption on their side.  The connection is there but OPNsense doesn't know and never gets around to asking.

It appears we're stuck with running a cron job to check whether the network has fallen over and then take action accordingly - and that much I've found examples of here, but the examples I've found resort to rebooting OPNsense, which seems a bit extreme.

I've tried using ifconfig to take the interface down and back up, but that doesn't appear to result in a DHCP renewal ... or is there a hidden option somewhere?

Do I have any other choices, or is a full reboot really it?


(Actually, I suspect there may be another solution: change the modem from bridge to router mode and let it take care of the WAN address change while giving OPNsense a static address.  But that adds another layer, so if I can avoid it I will.)

(Edited to say this was my mistake, see my later post https://forum.opnsense.org/index.php?topic=31769.msg154049#msg154049.)

I notice that devices that have not yet logged in to the Captive Portal - and do not have an exception listed - are still making some connections to the Internet.  I did not expect this.  ( I don't recall it happening on the pfSense firewall I was running up to now - but I guess it's possible it happened without me noticing it. )

I can confirm the connections on the device (eg: a Windows 10 VM running TinyWall lets me see the established connections), but it was ZenArmor showing the sessions that first brought it to my attention.

For example, when I start Firefox on a Windows 10 VM it brings up the expected "You must log in to the network before you can access the Internet".  Meanwhile, in the background the Internet is already being accessed.  At the very least "detectportal.firefox.com" and "safebrowsing.googleapis.com" are getting through, but I also use a speed-dial set up and it seems that at least some of those are updating too.

With Vivaldi (a Chromium based browser) I see much the same thing, with connections to "update.vivaldi.com" and "update.googleapis.com" amongst others.


Is this supposed to happen, or is this a bug in 22.7 that I should be reporting?

So I went back to a previous snapshot and reinstalled, again using Passive Mode, again turned off the Cloud Reputation, but this time selecting only LAN (vmx1).  Same problems.

Then I tried changing the mode (on the above install) from Passive Mode to Routed Mode (native netmap).  Same problems.

Then I did an uninstall and reinstall, but this time I chose Routed Mode (native map) right at install, again turned off the Cloud Reputation and still selecting only LAN (vmx1).  And this time it's looking much better.  Remote hosts are showing under remote hosts and local under local ... the only exception seems to be local host name for the ntp server showing up in the remote stuff. (My devices point to this name for time sync rather than directly to the gateway address - long story.)

For the next test, I went into configuration and switched back to Passive Mode and added DMZ (vmx2).  It seemed to take a few minutes for the impact to show, but it looks like everything is moving back to being the wrong way around as described in the OP.  So it looks like I took too many steps in this test, I don't know which broke it.

I changed the config back to Routed Mode (native map), and left DMZ (vmx2) in the list.  So far it is looking okay.  Both LAN and DMZ hosts are showing in local hosts (and no stray internet stuff).  The remote hosts lists are mostly remote, but do also show the gateway addresses (so "LAN address" and "DMZ address", if you see what I mean); this may be expected and okay, since the gateway is acting as ntp server.

( One oddity is that it seems to show an internal media server as having some internet connections - which I confirmed by a monitor on the host itself.  This host does not have an exception in the Captive Portal, so it seems that I have more issues to run down yet. )


Summary:  It appears to be Passive Mode that causes the problem.  And to avoid it you MUST install in a Routed Mode, simply switching away from Passive Mode doesn't work unless the initial install was in Routed Mode.


P.S. I still cannot get it to filter by Interface name.

Having just solved a problem that's been bugging me half the day I thought I would share.  Maybe the behaviour should have been expected and obvious, but it wasn't to me.

OPNsense 22.7 with WAN, LAN and DMZ interfaces.  Also two separate Captive Portal zone definitions, one for LAN and one for DMZ, and each defined a few addresses (that being the only option) that could access the network without seeing the login screen.

I could have used just one captive portal zone, but since I find the interface for managing allowed devices to be a bit cramped and awkward, I thought it would be easier to use separate zones ... and therein lies the problem.

I later introduced a firewall rule to let LAN devices access a HTTP server on the DMZ and it didn't seem to be working.  What I found was that a connection would make to the server, but the responses never got back.  It appears Captive Portal was blocking it.

The device in question did have its address in the LAN Captive Portal zone, but did not have it in the DMZ Captive Portal zone.  As soon as I added it there too, the connection started working.

All good, I have merged the two Captive Portal zones into one, so I have just one list of device exceptions, and now I can move on.  In my situation this arrangement is not going matter very much, but I can imagine it could be inconvenient in some more complex networks.

I figure I have nothing nothing much to lose on this new installation, so I changed the configuration to remove DMZ (vmx2) from being monitored, then under Reporting & Data in the configuration I chose toe Erase Reporting Data (all of it).

Then I visited a few sites and checked the reports.  I'm still seeing remote names in the "Top Local Hosts" (plus IP of the local device doing the browsing), and seeing LAN IPs in the "Top Remote Hosts".  (And so on for the ports etc.)

There's probably nothing stopping me from doing an uninstall and reinstall of Zenarmor if that might make a difference.

Thanks for your reply.

I'm pretty sure I must be doing something wrong, because I cannot get any interface filter to work as expected.

As I noted in the OP, Zenarmor insists on using the nic device name, not the other possibilities (DMZ or opt1), so in this case vmx2.  But just in case I've tried all variations.  And just to be sure, I tried setting up a positive (equals) test using LAN, "LAN", 'LAN', lan, "lan", 'lan', vmx1, "vmx1", 'vmx1'

All the "NOT EQUALS" tests I've tried make no difference to the output.  All the EQUALS tests I've tried produce empty results.

I notice that the filter display, after I hit refresh, changes from showing something like: Interface: LAN

to showing: Interface: Lan

And (so on with quotes or not according to the variation I tested).  Is there any chance this is actually doing a case-sensitive match and so getting it wrong on my all upper or all lowercase interface names?  Or is this just a filter display error?

It really seems like this must be a user error, but I cannot work out what I'm doing wrong.  Removing the interface filter brings everything back to what I've described.