root@freebsd:/home/vagrant # sh opnsense-bootstrap.sh -r 27.4 -y
This utility will attempt to turn this installation into the latest
OPNsense 27.4 release. All packages will be deleted, the base
system and kernel will be replaced, and if all went well the system
will automatically reboot.
fetch: https://github.com/opnsense/core/archive/stable/27.4.tar.gz: Not Found
Did anything change in that regard?
24.7 ;)
*patsch* (german onomatopoeia for my hand hitting my forehead) :)
Thanks.
But that was only the manual try to debug why my Vagrant file doesn't work and the Vagrant file got it correctly. I'll probably follow up with a different issue.
Yep ... that matches my Vagrant output:
root@freebsd:/home/vagrant # sh ./opnsense-bootstrap.sh -r 24.7 -y
This utility will attempt to turn this installation into the latest
OPNsense 24.7 release. All packages will be deleted, the base
system and kernel will be replaced, and if all went well the system
will automatically reboot.
/tmp/opnsense-bootstrap/core.tar.gz 7683 kB 8348 kBps 00s
pkg: 12 packages installed
bash-5.2.26_1: already unlocked
When executed with `sh -x` the final lines are:
+ pkg -N
pkg: 12 packages installed
+ pkg unlock -ya
bash-5.2.26_1: already unlocked
https://github.com/opnsense/update/issues/93#issuecomment-2242714677
IMHO you should simply not `set -e`, so the script will continue to run in case an irrelevant command exists with a status != 0.
The patch you linked does not change opnsense-bootstrap.sh.in - that tool won't work on any system that already has any package installed ...
set -e is good practice. It found an upstream bug after all. :)
Besides if you want reliable firmware scripting there's no way around aborting on issues early rather than late.
Cheers,
Franco
Will you adapt opnsense-bootstrap.sh.in?
I now patch opnsense-bootstrap.sh.in in my Vagrant file and it fails in line 205:
rm -f /var/db/pkg/*
Error message:
rm: /var/db/pkg/repos: is a directory
And well it is:
root@freebsd:/var/db # ls -lR pkg/
total 4
drwxr-xr-x 3 root wheel 512 Jun 15 14:40 repos
pkg/repos:
total 4
drwxr-xr-x 2 root wheel 512 Jun 15 14:40 FreeBSD
pkg/repos/FreeBSD:
total 62760
-rw-r--r-- 1 root wheel 64212992 Jun 15 03:40 db
-rw-r--r-- 1 root wheel 0 Jun 15 14:40 db-journal
-rw-r--r-- 1 root wheel 2 Jun 15 14:40 groups.ucl
-rw-r--r-- 1 root wheel 0 Jun 15 14:41 lock
-rw-r--r-- 1 root wheel 172 Jun 15 14:40 meta
Going to patch that too for now, but the bootstrap script needs an overhaul if it is supposed to work on a FreeBSD 14.1 system that has any packages installed, already.
The patch for pkg will be released sooner or later which fixes this just the way it broke it.
https://github.com/freebsd/pkg/issues/2278
I think the bigger issue is that this went unnoticed for a couple of months.
We're still at pkg 1.19 where it isn't broken.
Cheers,
Franco
Looks like the repos subdir was added since:
https://github.com/freebsd/freebsd-ports/commit/c923e0d7d
Can fix that, no problem.
Cheers,
Franco
https://github.com/opnsense/update/commit/3881cbf56
That's from April ... I need to deploy OPNsense in Vagrant now.
*sigh*
Thanks for fixing the subdir problem. Why didn't you just nuke /var/db/pkg from orbit? Only way to be sure ;) I mean rm -rf that thing?
I'll keep the code to remove the pkg unlock line in my Vagrant project. After all I know that my base boxes don't have any locked packages.
Kind regards,
Patrick
Quote from: franco on July 24, 2024, 10:11:30 PM
https://github.com/opnsense/update/commit/3881cbf56
That won't remove anything in there starting with a dot. I try to avoid shell glob patterns in scripts like the proverbial plague. Why not rm -rf /var/db/pkg?
Plus your fix is not in the tagged 24.7 release.
I load bootstrap.sh.in from
https://raw.githubusercontent.com/opnsense/update/#{$opnsense_release}/src/bootstrap/opnsense-bootstrap.sh.in
Because I want to be able to set the FreeBSD base version (13.2/3 or 14.1) and the matching OPNsense version (24.1 or 24.7). Because of the explicit FreeBSD version checks that you have in place that prohibit just fetching from master.
That's dangerous when pkg is a link for whatever reason or a mount point which can't be removed.
A bit of ancient cruft of /var MFS functionality... https://github.com/opnsense/core/commit/aa2bc6599e
Cheers,
Franco
> Because I want to be able to set the FreeBSD base version (13.2/3 or 14.1) and the matching OPNsense version (24.1 or 24.7). Because of the explicit FreeBSD version checks that you have in place that prohibit just fetching from master.
I've avoided stable branches for update tools because backporting deprecated versions is not very elegant and dragging one or two line changes through seems silly enough. FreeBSD EoL versions quickly deteriorate as well as you know.
Most of the noise of update.git is when a new major is released which very briefly breaks expectations for 90% upstream reasons. I'm not sure how I can make it better in the bulk of the issues to be honest. It all works 99,9% of the time in all OPNsense versions it lives in.
Feel free to raise a ticket there and we can see what major pains we can fix maybe.
Cheers,
Franco
Quote from: franco on July 24, 2024, 10:18:17 PM
That's dangerous when pkg is a link for whatever reason or a mount point which can't be removed.
Understood. Maybe I don't have the full picture of the intended use of the bootstrap script. I thought its purpose is to turn a stock FreeBSD into an OPNsense
once and then never again.
So people can ... like ... install in otherwise inaccessible cloud providers. [1] Or with unusual ZFS pool topologies or whatever.
[1] E.g. boot Linux rescue system, use depenguinator (https://depenguin.me) to install FreeBSD, bootstrap OPNsense.
The README does note its main use and the additional benefits:
https://github.com/opnsense/update?tab=readme-ov-file#opnsense-bootstrap
You are not wrong. It's simply a lot harder to control FreeBSD itself where we don't ship an opnsense-boostrap port that could be fixed timely (and with the right chain of bug reporting and user exposure).
The script bundled with any OPNsense version should fulfil its expectations without issues (yes there have been a few but not too many over the years).
Cheers,
Franco
Quote from: franco on July 24, 2024, 10:23:14 PM
I've avoided stable branches for update tools because backporting deprecated versions is not very elegant and dragging one or two line changes through seems silly enough. FreeBSD EoL versions quickly deteriorate as well as you know.
But if I use master, you cannot bootstrap 24.1 on FreeBSD 13.3 - for example to try to debug some problem in the lab with the version you have in production. Most installations will be running 24.1 for at least a couple of weeks to come. You/we just broke 24.1 in Vagrant for good, if I go that way. And FreeBSD 13.3 is well supported.
I'd love to have the bootstrap script for the betas/rcs, so we would have some amount of overlap here.
How do you guys
install early access versions of OPNsense in VMs? We use `vagrant up` for everything. Boot a simulation of our entire data centre with 5 virtual machines? There's a Vagrantfile for that.
Well you can use the 24.1 tag to bootstrap FreeBSD 13.3 although it takes that for tolerance of EoL policy and really wanted 13.2 instead. The fun thing is when 24.1 was tagged the pkg-unlock bug didn't exist.
Cheers,
Franco
Quote from: franco on July 24, 2024, 10:29:38 PM
Well you can use the 24.1 tag to bootstrap FreeBSD 13.3 although it takes that for tolerance of EoL policy and really wanted 13.2 instead.
Ah ... so master still does support 24.1/13.x? Should have checked the source before posting. Thanks. I just assumed it would now error out with any FreeBSD version but 14.x.
The tag can, but HEAD will not. I don't know if GitHub allows easy per file pull of a tag so after thinking it over I'm just going to disable FreeBSD major version check over bootstrap when having fetched the thing without a proper make-based install phase which would adjust the correct value into the script to check:
https://github.com/opnsense/update/commit/faa8226602
Cheers,
Franco
Perfect! Thanks a lot. My project is updated and pushed, just toying with 24.7 now ...