Topics

1

19.1 Legacy Series / Upgrade from 19.1 to 18.7.10

« on: March 10, 2019, 02:03:45 pm »

I clicked on the unlock to perform the upgrade from 18.7.10 to 19.1 and got the following message.

***GOT REQUEST TO UPGRADE: maj***
Fetching packages-19.1-OpenSSL-amd64.tar: .................................. done
Fetching base-19.1-amd64.txz: ............ done
Fetching kernel-19.1-amd64.txz: .............................................. failed, no signature found
***DONE***

At this point it seemed like there was no way to upgrade and so searching the forum I found:

• https://forum.opnsense.org/index.php?topic=11199.0

Rather than remove the files suggested in the final post there, I mv'ed them out of the way and things looked promising. I had an upgrade button again and so I proceeded with the upgrade. All looked good, a few reboots, etc and the system was back up and functional.

Checking the versions now, though, things do not look right. The lobby dashboard now shows

Versions   OPNsense 18.7.10_4-amd64
FreeBSD    11.2-RELEASE-p8-HBSD
OpenSSL    1.0.2q 20 Nov 2018

And my upgrade screen shows what it did previously (That I am on 19.7.10) but only very briefly. After s second or two, It changes its mind and instead offers me the following upgrade options.

There are 2 updates available, total download size is 98.7MiB. This update requires a reboot.

Package Name   Current Version   New Version   Required Action
base           19.1              18.7.10       upgrade
kernel         19.1              18.7.10       upgrade

Given that things seem to be working fine but the state is looks confused, I am not sure what the safest path forward would be.

2

19.1 Legacy Series / The next major release is: 19.1

« on: March 09, 2019, 02:31:39 pm »

I am currently running 18.7.10 and now see vulnerabilities being reported. To upgrade I have to unlock the next release. I am wanting to remain on stable releases and avoid development releases or release candidates. Initially, when 19.1 was the only choice I thought it was a release candidate and and that the unlock would be removed when it became stable, however it looks like I mis-understood as there are now multiple releases queued up (19.1, 19.1.1, 19.1.2, 19.1.3) and the forums label it as production.

Looking at https://opnsense.org/about/road-map/, I see:

• NEXT RELEASE 19.1 - Januari 2019
• LATEST RELEASE 18.7 - July 31, 2018

So I feel 18.7 is latest prod.

From the "book link" by 19.1.3 on the UI, I read "recommend running 18.7 until this is taken care of" so again I think 18.7 is current prod.

The forum however labels 18.7 as legacy and 19.1 as production.

So, will the unlock go away when 19.1+ is as stable as it will get or will it always require a conscious click?
Since 18.7 is EOL, I guess vulnerabilities will never get fixed, so is the 19.1 path going to lead to better security with a little upgrade risk?

3

18.7 Legacy Series / Fix insight < 7 Days and keep data

« on: November 23, 2018, 04:01:17 pm »

I'b de grateful for any tips of fixing or troubleshooting the most granular Insight graphs.

Insight worked fine for a while, but broke for me a while back after an upgrade through the UI. I am now running 18.7.8 and had hoped my problem would go away with subsequent updates, but it has persisted. I have read numerous articles, but everything I have seen that may help looked like it would loose my current history. I am hoping to be able to fix the granular graphs and preserve my history.

My Insight works partially as follows

Interface totals (bits/sec) - Last 2 hours - No Data Available
Interface totals (bits/sec) - Last 8 hours - No Data Available
Interface totals (bits/sec) - Last 24 hours - No Data Available
Interface totals (bits/sec) - All the rest seem to work fine.

Top usage ports / sources (bytes) - Last 2 hours - No Data Available
Top usage ports / sources (bytes) - Last 8 hours - No Data Available
Top usage ports / sources (bytes) - All the rest seem fine.

I have rebooted the system.
I have reinstalled the package flowd.
I ran Reporting --> Settings --> Repair Netflow Data
I checked /var/log/flowd.log and see current data using flowd-reader flowd.log

The flow logs seem to be updating fine and rotating.

root@OPNsense:/var/log # ls -lh flowd.log*
-rw-------  1 root  wheel   1.4M Nov 23 09:34 flowd.log
-rw-------  1 root  wheel    11M Nov 23 08:17 flowd.log.000001
-rw-------  1 root  wheel    11M Nov 22 23:09 flowd.log.000002
-rw-------  1 root  wheel    11M Nov 22 18:45 flowd.log.000003
-rw-------  1 root  wheel    11M Nov 22 08:45 flowd.log.000004
-rw-------  1 root  wheel    11M Nov 21 22:08 flowd.log.000005
-rw-------  1 root  wheel    11M Nov 21 13:38 flowd.log.000006
-rw-------  1 root  wheel    11M Nov 20 22:34 flowd.log.000007
-rw-------  1 root  wheel    11M Nov 20 11:28 flowd.log.000008
-rw-------  1 root  wheel    11M Nov 19 22:27 flowd.log.000009
-rw-------  1 root  wheel    11M Nov 19 07:06 flowd.log.000010

I checked unique flowd type messages in system.log (One issue starting flowd_aggregate a while ago)

root@OPNsense:/var/log # grep flowd system.log | cut -d' ' -f 5- | sort -u
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/metadata.sqlite
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/metadata.sqlite [done]
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/src_addr_000300.sqlite
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/src_addr_000300.sqlite [done]
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/src_addr_003600.sqlite
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/src_addr_003600.sqlite [done]
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/src_addr_086400.sqlite
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/src_addr_details_086400.sqlite
OPNsense flowd_aggregate.py: sqlite3 repair /var/netflow/src_addr_details_086400.sqlite [done]
OPNsense flowd_aggregate.py: start watching flowd
OPNsense flowd_aggregate.py: startup, check database.
OPNsense flowd_aggregate.py: vacuum /var/netflow/dst_port_000300.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/dst_port_003600.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/dst_port_086400.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/interface_000030.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/interface_000300.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/interface_003600.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/interface_086400.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/src_addr_000300.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/src_addr_003600.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/src_addr_086400.sqlite
OPNsense flowd_aggregate.py: vacuum /var/netflow/src_addr_details_086400.sqlite
OPNsense flowd_aggregate.py: vacuum done
OPNsense pkg-static: flowd reinstalled: 0.9.1_3 -> 0.9.1_3
OPNsense root: /usr/local/etc/rc.d/flowd_aggregate: WARNING: failed to start flowd_aggregate
flowd_aggregate.py: start watching flowd
flowd_aggregate.py: startup, check database.
flowd_aggregate.py: vacuum /var/netflow/dst_port_000300.sqlite
flowd_aggregate.py: vacuum /var/netflow/dst_port_003600.sqlite
flowd_aggregate.py: vacuum /var/netflow/dst_port_086400.sqlite
flowd_aggregate.py: vacuum /var/netflow/interface_000030.sqlite
flowd_aggregate.py: vacuum /var/netflow/interface_000300.sqlite
flowd_aggregate.py: vacuum /var/netflow/interface_003600.sqlite
flowd_aggregate.py: vacuum /var/netflow/interface_086400.sqlite
flowd_aggregate.py: vacuum /var/netflow/src_addr_000300.sqlite
flowd_aggregate.py: vacuum /var/netflow/src_addr_003600.sqlite
flowd_aggregate.py: vacuum /var/netflow/src_addr_086400.sqlite
flowd_aggregate.py: vacuum /var/netflow/src_addr_details_086400.sqlite
flowd_aggregate.py: vacuum done
root@OPNsense:/var/log #
root@OPNsense:/var/log # grep 'failed to start flowd_aggregate' system.log
Nov  4 09:01:15 OPNsense root: /usr/local/etc/rc.d/flowd_aggregate: WARNING: failed to start flowd_aggregate
Nov  4 09:01:24 OPNsense root: /usr/local/etc/rc.d/flowd_aggregate: WARNING: failed to start flowd_aggregate
Nov  4 09:01:31 OPNsense root: /usr/local/etc/rc.d/flowd_aggregate: WARNING: failed to start flowd_aggregate
Nov  4 09:06:46 OPNsense root: /usr/local/etc/rc.d/flowd_aggregate: WARNING: failed to start flowd_aggregate
Nov  4 09:15:22 OPNsense root: /usr/local/etc/rc.d/flowd_aggregate: WARNING: failed to start flowd_aggregate

I took a shot at making sure databases were valid.

# pwd                                                                                                                                                                             
/var/netflow
# for sqlite in `ls *sqlite`
do
echo $sqlite; sqlite3 ${sqlite} 'pragma integrity_check;' '.exit'
done
dst_port_000300.sqlite
ok
dst_port_003600.sqlite
ok
dst_port_086400.sqlite
ok
interface_000030.sqlite
ok
interface_000300.sqlite
ok
interface_003600.sqlite
ok
interface_086400.sqlite
ok
metadata.sqlite
ok
src_addr_000300.sqlite
ok
src_addr_003600.sqlite
ok
src_addr_086400.sqlite
ok
src_addr_details_086400.sqlite
ok


I did not do I ran Reporting --> Settings --> Reset Netflow Data
I have been avoiding this as I did not want to loose what it has collected.