Messages

1

22.1 Legacy Series / Re: With "Less Secure" depreciated for gmail, how are you getting notifications?

« on: July 21, 2022, 06:02:54 pm »

Yep, just tested - seems to work with app specific password.

I use it with Monit, to forward via email to PushOver for notifications.

2

22.1 Legacy Series / Re: With "Less Secure" depreciated for gmail, how are you getting notifications?

« on: July 21, 2022, 05:57:30 pm »

Can't you just setup an 'app specific password'?

https://support.google.com/mail/answer/185833?hl=en-GB

I assume mine is still working, port 465, SSL enabled, etc, as of 5 days ago:

3

22.1 Legacy Series / Re: VPN + WAN failover

« on: July 18, 2022, 03:20:07 pm »

Whilst I haven't used it for a while, I ended up hacking together a script that ran from cron every minute:

https://github.com/opnsense/core/issues/3516#issuecomment-620415211

Basically if the default route is via the primary WAN, and there is more than 1 state (gateway monitoring) on the Secondary (4G in my case) kill all states to force it to reconnect.

Obviously you could also fairly easily modify it to restart the tunnel instead, or both...or not.

There is also the following, although I've not tested this:

- Firewall, Settings, Advanced, Dynamic state reset

...not sure if this works on fail over/IP change

4

22.1 Legacy Series / Re: WAN interface flapping with 22.1.2

« on: July 15, 2022, 07:12:08 pm »

Been seeing the same on 22.1.10, somehow it seems worse since I upgraded to .10 - although I've certainly been fighting this for a while - but might be a coincidence.

....

- This morning, I have now loaded the Intel updated drivers (1.12.35)

Whilst I'm still running the updated drivers, I'm pretty sure my issue was 4 dodgy wall ports where my cable modem connects - 2 ports per 1 gang box - so when I swapped the port, even to a different box, I still saw the problem...on all 4 ports.  In the end, I ran a 15m cable direct from the opnense box to the cable modem (and got moaned at with the cable going through the house/hallway) which fixed the problem.

Long story short...I replaced the wall modules and re-terminated the cabling and I think this has resolved my problem.  Over the next week, I'll start rolling back the various changes such as the updated drivers.

I've re-enabled gateway monitoring, this was in some cases causing the problem when the port flapped - although interestingly often the flaps weren't shown in the switch logs, so whilst in duration they were short they were long/frequent enough to cause disruption when gateway monitoring was enabled.

5

22.1 Legacy Series / Re: WAN interface flapping with 22.1.2

« on: July 12, 2022, 08:15:55 am »

Been seeing the same on 22.1.10, somehow it seems worse since I upgraded to .10 - although I've certainly been fighting this for a while - but might be a coincidence.

WAN interface: Intel(R) Ethernet Controller X710 for 10GbE SFP+

- Disabled MAC spoofing, did not fix things
- Disabled gateway monitoring, seemed to improve things but did not resolve
- Upgraded the 710 firmware, using stock drivers, did not fix things
- This morning, I have now loaded the Intel updated drivers (1.12.35)

I am also using RSS, so this is maybe the next thing for me to rule out.

I do also note, that the latest Intel drivers are not iflib, so the various tunings have now changed (tx/rx ring buffer, queues).  There are details in the readme.txt.

I did also try the updated IGB driver (out of curiosity) as I also have the below, although these ports are NOT on the WAN and did not see the problem.

Intel(R) I211 (Copper)

But this lead to 'weird' things.  For example, the HAproxy instance running on opnsense could not health check my Home Assistant server (to provide SSL externally) TCP port.  The Home Assistant physical port on opnsense, is on the I211.  Although traffic could pass from LAN -> Home Assistant through the firewall...the firewall itself could not reach the Home Assistant TCP port.

I could see the SYN from opnsense HAProxy -> Home Assistant on the server port, and the SYN,ACK reply reach the firewall, but for some reason it was being dropped.  I did not have time to look into this further, so rolled it back leaving just the WAN X710 using the Intel drivers.  This instantly resolved the HAproxy/Home Assistant issue.

6

22.1 Legacy Series / Re: OPNsense router keeps crashing / becoming unresponsive

« on: February 09, 2022, 09:14:45 am »

I also had issues with PowerD on a recent i7 CPU (2019, same year as yours) - disabling it resolved a similar-ish problem for me, relying on the CPU scaling instead,

Had no issues with a much older Celeron J1900 using PowerD, until I upgraded to the above.

7

Hardware and Performance / Re: Deciso DEC850 - CPU speed goes up only to 1500MHz instead of 3100MHz?

« on: February 01, 2022, 11:48:52 am »

My Intel i7-9700 does as well:

Code: [Select]

dev.cpu.7.freq_levels: 3000/-1
dev.cpu.7.freq: 4497
dev.cpu.6.freq_levels: 3000/-1
dev.cpu.6.freq: 4497
dev.cpu.5.freq_levels: 3000/-1
dev.cpu.5.freq: 4497
dev.cpu.4.freq_levels: 3000/-1
dev.cpu.4.freq: 4497
dev.cpu.3.freq_levels: 3000/-1
dev.cpu.3.freq: 4497
dev.cpu.2.freq_levels: 3000/-1
dev.cpu.2.freq: 4497
dev.cpu.1.freq_levels: 3000/-1
dev.cpu.1.freq: 4497
dev.cpu.0.freq_levels: 3000/-1
dev.cpu.0.freq: 4497
I'm just using hwpstate_intel, rather than PowerD - I had issues with that with this CPU, in 21.7 yet to try PowerD again on 22.1

8

Tutorials and FAQs / Re: Check_MK Agent setup

« on: January 24, 2022, 02:28:19 pm »

Wonderful, many thanks @NilsS

9

General Discussion / Re: Virgin Media UK - 'Super' Hub 4 - Modem Mode

« on: January 22, 2022, 10:30:50 pm »

Replacement UPS should arrive later, so I'll have to restart the Hub again. 

Well I swapped in the new UPS, I left the Virgin Hub 4 powered off for about 25 minutes as it was last night when the previous UPS failed, poweedr it up...

... opnsense got an IP address first time using the settings above. 

Fingers crossed.  Guess time will tell, or if anyone else who has/is having problems can test.

10

22.1 Legacy Series / Re: RSS Support Yet?

« on: January 22, 2022, 07:06:22 pm »

Works for me.

root@fw00:~ # grep "isr\|rss" /boot/loader.conf.local
net.isr.bindthreads=1
net.isr.maxthreads=8
net.inet.rss.enabled=1
net.inet.rss.bits=3

Sticky post at the top of this very forum?

11

General Discussion / Re: Virgin Media UK - 'Super' Hub 4 - Modem Mode

« on: January 22, 2022, 01:11:59 pm »

From: https://www.freebsd.org/cgi/man.cgi?query=dhclient.conf&sektion=5&n=1

opnsense defaults:

- Timeout: 60
- Retry: 15
- Select-timeout: 0
- Reboot: 1
- backoff cutoff: empty
- initial interval: 1

So, for as much my benefit to work through this as potentially anyone else, this means:

- 60 seconds must pass, before the firewall decides it's not going to be able to contact a server

- After the 60 second timeout has passed, if there are static leases (which we shouldn't have) or any leases not yet expired (possibly/likely), client will loop through them trying to validate them

- After the 60 second timeout has passed, if there are no valid static leases or expired leases, client will restart the protocol after retry (15) interval

- 15 seconds must pass after opnsense has determined there is no DHCP server present, before it tries again to contact a DHCP server (default is 5 minutes in FreeBSD)

- select-timeout only seems to be relevant if there is more than one DHCP server on a network, not sure this is relevant in this case, especially as we are excluding the local requests from 192.168.100.254 so we want the first 'other' offer we receive (but get it with a zero setting anyway)

- When the client is restarted, 1 second must pass before it gives up trying to get its old address again (default 10 seconds)

- To try to prevent many clients making requests in lockstep, opnsense uses a backoff algorithm with some
randomness, default for this is 2 minutes.  This is called backoff cutoff.

- There is 1 second between the first attempt to reach a server and the second attempt, each message sent the interval is incremented by twice the current interval multiplied by a random number between 0 and 1.  If it is greater than the backoff cut off (2 minutes) it is set to that amount (2 minutes)

If we accept as 'true' that a device connecting in modem mode must make a DHCP request within a 15 second window, which perhaps could be 30 seconds to a few minutes after the Hub 4 starts up, we need much more consistent DHCP requests over this period to ensure we make at least 1 request in that 15 second window.

I believe it is this continually sliding and partially random interval that causes problems and perhaps adds some weight to why Asus also call theirs 'continuous' - in that it makes requests in a metronomic type fashion, to ensure a request is made within that 15 second window.

So if we assume that 'Asus 12Hz Continous DHCP' is not 12Hz, so not 12 requests a second (!!!) but more likely 12 per minute/every 5 seconds, perhaps reasonable options would be:

- Timeout: 4
- Retry: 1
- Select-timeout: 0
- Reboot: 1
- backoff cutoff: 1
- initial interval: 1
- Reject Leases From: 192.168.100.254

.. this should also potentially help as/when the DHCP IP address offer changes, due to reboot being set to 1.   

12

General Discussion / Re: Virgin Media UK - 'Super' Hub 4 - Modem Mode

« on: January 22, 2022, 11:50:57 am »

After doing a bit more digging, some seem to suggest that the Asus 12Hz is 12 per minute - every 5 seconds.... which means it isn't 12Hz at all...

But I don't have an Asus router to check.  Replacement UPS should arrive later, so I'll have to restart the Hub again. 

Will perhaps try modifying my DHCP options to replicate every 5 seconds, rather than every 1, I think the opnsense default is 15 seconds (but then extends the re-try if no response, so you have to time it correctly - repeatedly every X seconds would be better potentially for the Hub 4).

.. especially if it is true that there is a 15 second window when the Hub 4 will hand out an IP in modem mode and then ignores all others.

13

General Discussion / Re: Google Fiber 2GBps

« on: January 22, 2022, 09:50:29 am »

Whilst I cannot say for sure, I'm still waiting for a 2.5Gbps port cable modem... somethings to perhaps consider:

You'd need an SFP+ module in the card for anything over 1, but I believe some are picky about 2.5Gbps.

Additionally, possibly a firmware upgrade to the X710.  This can be done 'online' from FreeBSD/opnsense using the 'intel-nvmupdate' package from ports:

https://www.freshports.org/sysutils/intel-nvmupdate/

Code: [Select]

/usr/local/sbin/nvmupdate

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.35.33.4
Copyright (C) 2013 - 2020 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [.....|****]


Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Converged         8.00(8.00)   1572 00:001 Up to date
    Network Adapter X710
There does appear to be a later version on the Intel site, although when I recently got my box with an X710 card I used the ports version:

https://www.intel.com/content/www/us/en/download/18636/non-volatile-memory-nvm-update-utility-for-intel-ethernet-adapters-700-series-freebsd.html

Link state can be lost, requiring a reboot after completed, so if it is only your WAN on the card it shouldn't be too much of a problem - as you could still access the firewall - if not, completing over a local/console connection is probably advisable, so you can tell when it has actually finished if you need to reboot.

14

General Discussion / Re: SMTP server on Monit

« on: January 22, 2022, 09:36:25 am »

Hi there,

I use a gmail account for mine - configured with:

https://support.google.com/mail/answer/7126229?hl=en-GB#zippy=%2Cstep-change-smtp-other-settings-in-your-email-client

... port 465, SSL enabled, etc.

However, if you just put your regular gmail password in, you'll see the below in the Monit logs:

Code: [Select]

SMTP: Mailserver response error -- 534 5.7.9 https://support.google.com/mail/?p=InvalidSecondFactor o18sm2679679ejb.111 - gsmtp
Once you configure an App specific password, and use this in Monit instead of your main/actual gmail password, then it works:

https://support.google.com/mail/search?q=application+specific+password&from_promoted_search=true

15

General Discussion / Virgin Media UK - 'Super' Hub 4 - Modem Mode

« on: January 22, 2022, 09:00:03 am »

Recently upgraded my Virgin Media connection to Gig1 and with it (unfortunately) I was provided with a Hub 4 - from day 1, it has been a PITA.  It took me hours and hours to finally get an IP address from modem mode to opnsense.

Running the device in Modem Mode, it is extremely picky about when/if it will let opnsense have an IP - or the DHCP servers are - certainly with the default DHCP settings, even though the opnsense defaults request more frequently than the FreeBSD defaults.

There are various reports of problems, with people using opnsense, pfSense, Asus, you name it.  It seems to be 'pot luck' whether the device behind the modem gets an IP address or not, the suggested steps of:

- Power off the modem
- Restart your main firewall/router
- Give it a few minutes
- Power on the modem

.. seems to work in some cases, but not others.  I do firmly believe there is still an element of luck to whether this works or not. 

I read somewhere that if the DHCP request doesn't complete in a 15 second window, after the modem has booted, it basically ignores all other requests after.  This was quoted from a Virgin engineer, so who knows....

I thought I had cracked it last time I had problems, when I filtered 192.168.100.254 in 'Reject Leases From' - otherwise opnsense ends up getting a 192.168.100.x address instead.  Note, previous modems you needed to filter 192.168.100.1 (I believe).

Running tcpdump, filtering for DHCP requests/replies, the requests are being made, but just no response or not a completed response:

Code: [Select]

tcpdump -i eth0 port 67 or port 68 -e -n -vv
Last night the UPS on my cable modem died, opnsense failed over beautifully to 4G and until I heard a whining noise on my way to bed coming from the garage I didn't realise it had failed (Muted PushOver notifications after 10PM).  So I removed the UPS, powered on the cable modem and went to bed - this morning, it still hadn't got an IP and was still running on 4G.

I tried restarting the modem, and opnsense, using the basic procedure above that is reported to (sometimes) work.  But it didn't.

In the end, I discovered that Asus now has some '12Hz DHCP' option - to supposedly fix similar issues, where the device doesn't get an IP - which requests an IP 12 times a second (12Hz)?!?!

https://www.asus.com/my/support/FAQ/1043591/

Frustrated, I ended up putting '1' in every box in the DHCP options for the WAN interface - along with filtering 192.168.100.254 from DHCP replies - to try to make DHCP requests as frequent as possible.  Rebooted the Hub 4....and it got an IP address first time.

... I don't really want to test this again, just yet, but will update this post with any further developments next time I come across it.

Aside from this, does anyone have any idea how to replicate the Asus 12Hz option?  I do see some mention that others with Starlink and other cable modems also need to use this feature, so it doesn't seem to be Virgin specific.