[SOLVED] Use local ports tree instead or packages from mirror when building core

[interfaSys]

The current process to build and install core, as I understand it is as follows:

• Update all ports
• cd /usr/tools
• Build ports missing from the packages "image" using "make ports"
• make core

make core does this:

• Unpack all packages, mix with compiled missing ports
• Build core
• Update contrib with files from packages?
• Create new packages image

For people using `opnsense/ports` to compile all their ports, there are a few problems:

• All missing ports have to be compiled again, although all those ports have been installed already
• The rest of the packages used are vanilla packages which have not been compiled for the current architecture. It's only a problem is those files are actually used
• If contrib is updated using vanilla packages, then it might introduce some changes compared to what is installed

If the only reason packages are downloaded and unpacked to build core is to update contrib, then only those packages should be used.
Those packages don't even need to be compiled, they're already installed and the needed files can be fetched from the filesystem. pkg info can tell the script if the version is what is expected.

Am I correct and the whole process can be simplified for people using the ports tree?

[franco]

Am I correct and the whole process can be simplified for people using the ports tree?

I am not sure I understand this question. Can you elaborate?

tools.git is for building from scratch, not for providing incremental updates of core for a running system. It does things the way it does in order to verify the full integrity of the build, so core e.g. depends on installing all the currently needed ports from the set and doing it in a totally clean jail to avoid taint.

I think what you are looking for is an in-place build of core.git which yields a package when run on OPNsense itself. Is this correct?

If so, there are quite a few possibilities for simplification... On your box, you can run this:

# pkg install git gettext-tools
# cd /usr/core
# make DESTDIR=/tmp/core-pkg FLAVOUR=OpenSSL install
# make DESTDIR=/tmp/core-pkg scripts
# make DESTDIR=/tmp/core-pkg manifest > /tmp/core-pkg/+MANIFEST
# make DESTDIR=/tmp/core-pkg plist > /tmp/core-pkg/plist
# pkg create -m /tmp/core-pkg -r /tmp/core-pkg -p /tmp/core-pkg/plist
# rm -r /tmp/core-pkg
# ls *.txz
# pkg add -f ...

[franco]

I've added it as a simple target... FLAVOUR needs to be set if not OpenSSL and packages "git" and "gettext-tools" must be installed in order to create the package.

# make package

Also see: https://github.com/opnsense/core/commit/445504da33c16ca4253023e6a1fdebd517a3bb61

[interfaSys]

That worked perfectly, thanks.

So for each release, someone who wants to compile ports using the the opnsense ports tree would simply need to

• update kernel+base with opnsense-update
• cd /usr/ports; switch to tagged release; update all ports
• cd /usr/core; switch to tagged release; run the script you've provided

Easier than using make-core in /usr/tools

[interfaSys]

And thanks for the new target, even easier for users than having to add a script.

[franco]

Ok, now I see. Yes, your environment is now the native build environment for the core package and you don't have to jump through any other hoops. That makes perfect sense. Thanks for the explanations and prodding.

[franco]

There will also be an upgrade target, that should be as far as we could possibly go

# make upgrade

[interfaSys]

Awesome, thanks