pkg search/install etc

[cdburgess75]

When I search pkg's, I notice that the repository is linked to opnsense,  Is there a way to change it to freebsd repo?  I would like to try additional software.  I can report my progress here, it may help someone one day :)

[franco]

Yes, the historical reason being that multi-repo layouts did not work as expected in pkgng versions as far as early 1.4.x. There have been some changes that might make these setups viable, but I haven't checked them closely yet. You *can* enable the FreeBSD package mirror by editing /etc/pkg/FreeBSD.conf and setting "enabled: no" to "enabled: yes".

Note, however, that doing this may cause severe pains and package weirdness. A simple "pkg upgrade" might delete your current PHP version 5.6 and replace it with 5.4, rendering the GUI useless...

We discussed more exploration here and will share findings as we make progress.

[cdburgess75]

Yes perfect,  thank you.  I will be careful to test before production use ...as well as keep the forum informed with adventures :)

[pdf]

Seems like enabling the:

Code Select Expand

CONSERVATIVE_UPGRADE=yes

option in /usr/local/etc/pkg.conf does the trick.

But stock packages probably need to have their repository set:

Code Select Expand

pkg info | awk '{print $1}' | sed -e 's#-[^-]*$##g' | xargs -I {} pkg annotate -y -A "{}" repository OPNsense

After those two steps, pkg should only try to upgrade from the repository that a package is originally installed from.

[franco]

Yes, the repo annotation being the missing glue to hold it together in current installations. I think this will make its way into 15.1.6. Additionally, there is also: PRIORITY=1 which can be set in /usr/local/etc/pkg/repo/OPNsense.conf to make pkg aware of the fact that packages in there should always come from our mirror before FreeBSD is considered.

Note that FreeBSD 10 and OPNsense 15.1 are not completely ABI compatible, or that we build packages with the OpenSSL port as opposed to FreeBSD using the base system version. That version is still installed, because it cannot be easily switched off so you may be left in a state of lower security. More issues may loom in the dark.

Let's make a deal. If you need to do this let us know which packages you are looking for and we can add those into our build (as long as they don't pull in a ton of dependencies) and into the OPNsense mirror as optional packages?

[franco]

Code Select Expand

pkg info | awk '{print $1}' | sed -e 's#-[^-]*$##g'

can be replaced by

Code Select Expand

pkg query %n

:)

Annotations don't seem to be working for me on 1.4.12, at least `pkg annotate -S packagename repository' says there is no annotation even though `pkg annotate -A packagename repository OPNsense' ran fine... I'll take a closer look later.

[pdf]

Thanks for the tip on the pkg query - I haven't run FreeBSD for probably over 10 years, so I'm just sussing all of this out.

Annotations appear to work fine for me on 1.4.12:

Code Select Expand

# pkg --version
1.4.12
# pkg query %n | xargs -I {} pkg annotate -qy -A "{}" repository OPNsense
# pkg annotate -S zip repository
zip-3.0_1: Tag: repository Value: OPNsense


Things I immediately need to install are haproxy and rsync, but having access to the full FreeBSD package list is certainly desirable.

[franco]

I can offer to add haproxy and rsync to our optional ports build so you can install those via the command line without modification. That's not a long term solution for all the ports that people need, but it might cut the time until better solutions (full ports build maybe?) have been put in place.

[pdf]

Thanks franco, for now I'm just running them straight from the FreeBSD repos because I had to get this sorted over the weekend.

I guarantee I'll find other stuff that I need/want.  I've also installed socat, vim(-lite) and bash.

I'm running the latest snapshot, that reports 10.1-RELEASE-p5, so at least for right now, I believe openssl is patched up past the last advisory.  Going forward, is there any analogue to `freebsd-update` that would let us update the base OS for stuff like this?

[franco]

We have `opnsense-update`, which is new since 15.1.6.1. It is supposed to be a little bit easier to handle than freebsd-update, but it is currently experimental due to its young age and need for further development until we finally wrap it up into the GUI as a single shiny button.

We have vim-lite in our mirror. I also think that bash gets built, but not pushed into the mirror because it is a build-only dependency. Upon further reflection, there is not much we can do other than "opening up" the prebuilt ports more and more.

[pdf]

Right you are, installed vim-light from the OPNsense repo.

If PRIORITY=1 worked, this would be nice, but are you sure that's valid?  I can't find it documented anywhere, and it's not the correct format for the repo config, which is a JSON-like format, as opposed to pkg.conf.  There's already other stuff I'm pulling in from the FreeBSD repos, like duply/duplicity to get this box in my regular backups.

As for opnsense-update, it seems to be lacking a manpage, and:

Code Select Expand

opnsense-update --help
Fetching kernel---help-amd64.txz... fetch: http://pkg.opnsense.org/sets/kernel---help-amd64.txz: Not Found
failed

That's a bit scary...

[franco]

As for opnsense-update, it seems to be lacking a manpage, and:

Code Select Expand

opnsense-update --help
Fetching kernel---help-amd64.txz... fetch: http://pkg.opnsense.org/sets/kernel---help-amd64.txz: Not Found
failed

That's a bit scary...

I am aware of the missing man page. The tool is experimental and like its cousin, freebsd-update, a shell script so you can take a peek.

While it seems scary it is designed to be resilient to upgrade issues. The usage is:

opnsense-update [version] [mirror]

By default, it'll upgrade to its embedded version and verify SHA256 checksums of all fetched files.

If you don't use the defaults, it'll try to fetch a different version and upgrade that instead. This is for people wanting to upgrade to custom versions of OPNsense that they do themselves, or for quick testing before a release. You can also change the mirror. If the files can't be found, nothing bad happens.

If you have concerns with the code itself please let me know. Fixing is no issue. Man page coming soon, too. Before it is released as "stable" in any case.

Yeah, good catch with the syntax of PRIORITY, here is the respective commit: https://github.com/freebsd/pkg/commit/3f8794c06de01ba3517da4177564f3de42d33684

[pdf]

Okay, modifiying /usr/local/etc/pkg/repos/OPNsense.conf to contain:

Code Select Expand

OPNsense: {
  url: "pkg+http://pkg.opnsense.org/${ABI}/latest",
  mirror_type: "srv",
  priority: 10,
  enabled: yes
}

correctly prioritizes the OPNsense repo over the FreeBSD repo.  So, with CONSERVATIVE_UPGRADE=1 (this may not be necessary with priority, but it seems like the safe option), bumped OPNsense repo priority, and a repository annotation on the base packages, it looks like multiple repos is working pretty optimally.  Did a pkg upgrade to 15.1.6.1 no problem.  I chose priority 10 here so that I can add any additional repositories at a lower priority (I'd like to get this system added to my FreeIPA domain via sssd for example - not yet in the main FreeBSD repos).

I'll take a closer look at opnsense-update when I have a little time, but I'd suggest at least supporting the --help/-h argument to display some usage information.

[franco]

https://github.com/opnsense/core/commit/f1bbc9199daabe1ceff600786f63cfca9bf6871e

Does this look alright? I bumped priority to 11. ;)

conservative_upgrade is still needed when FreeBSD mirror has newer packages. pkgng has an itch in that regard and always wants the latest and greatest, hence the use of the term "conservative". I can see issues when we bring in new packages that somebody installed via FreeBSD. That's something I do need to test before we can safely reenable the default mirror. But in any case thanks for helping this progress so quickly.

I agree with "--help/-h": https://github.com/opnsense/ports/issues/4

[franco]

Fixed via https://github.com/opnsense/ports/commit/fc3b2211db3e58ee253b184ad1502b3ea68d1899

Still ironing out smaller hiccups on our way to 15.1.8.