Building OPNsense from Source

[ckurlinski]

First off, I want to say thank you for OPNsense.
I have been using it since the fork, I love the focus and code clean up effort.
Currently I'm trying to build the latest from git source 17.1.
Build system -

Code: [Select]

OPNsense 17.1.b - FreeBSD 11.0-RELEASE-p5 on Dual Xeon E5-2620 with 32GB
I have tried FreeBSD 11.0 and FreeBSD 10.3 per the steps outlined here https://github.com/opnsense/tools
Currently I did a OPNsense 17.1.b install and installed opnsense-code tools ports, and pulled the git for src and core.


On every build attempt I use a variation on the same make command

Code: [Select]

make SETTINGS=17.1 base
make SETTINGS=17.1 kernel
make SETTINGS=17.1 distfiles
make SETTINGS=17.1 core


base and kernel build without issue
core stops at net/dhcp6

Code: [Select]

make[1]: stopped in /usr/obj/usr/ports/net/dhcp6/work/wide-dhcpv6-20080615.1
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.*** Error code 1

Environment Setup for build is:

Code: [Select]

>>> Running build step: base

>>> Reusing base set: /tmp/sets/base-17.1.b_19-amd64.txz
.MAKE.LEVEL.ENV=MAKELEVEL
ABI_FILE=/usr/lib/crt1.o
BLOCKSIZE=K
CONFIGDIR=/usr/tools/config/17.1
CONFIG_XML=/usr/local/etc/config.xml
COREBRANCH=stable/17.1
COREDIR=/usr/core
CPUS=24
DEVICEDIR=/usr/tools/device
EDITOR=vi
ENV_FILTER=env -i USER=root LOGNAME=root HOME=/root SHELL=/usr/local/etc/rc.initial BLOCKSIZE=K MAIL=/var/mail/root PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin TERM=screen HOSTTYPE=FreeBSD VENDOR=amd OSTYPE=FreeBSD MACHTYPE=x86_64 PWD=/usr/tools/build GROUP=wheel HOST=fubar.home.kurlinski.com EDITOR=vi PAGER=less ABI_FILE=/usr/lib/crt1.o
GROUP=wheel
HOME=/root
HOST=fubar.home.kurlinski.com
HOSTTYPE=FreeBSD
IMAGESDIR=/tmp/images
LABEL=OPNsense
LOGNAME=root
MACHTYPE=x86_64
MAIL=/var/mail/root
MAKEFLAGS= .MAKE.LEVEL.ENV=MAKELEVEL SETTINGS=17.1
MAKELEVEL=1
META_MODE=normal
OLDPWD=/usr/obj/usr/tools
OSTYPE=FreeBSD
PACKAGESDIR=/.pkg
PAGER=less
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
PLUGINSBRANCH=master
PLUGINSDIR=/usr/plugins
PORTSBRANCH=master
PORTSDIR=/usr/ports
PORTSREFBRANCH=master
PORTSREFDIR=/usr/freebsd-ports
PRINT_ENV_SKIP=1
PRODUCT_ADDITIONS=
PRODUCT_ARCH=amd64
PRODUCT_DEVICE=a10
PRODUCT_FLAVOUR=OpenSSL
PRODUCT_HOST=amd64
PRODUCT_KERNEL=SMP
PRODUCT_MIRROR=http://mirror.sfo12.us.leaseweb.net/opnsense
PRODUCT_NAME=OPNsense
PRODUCT_PKGNAME=opnsense
PRODUCT_PKGNAMES=opnsense opnsense-stable opnsense-devel
PRODUCT_PRIVKEY=/usr/tools/config/17.1/repo.key
PRODUCT_PUBKEY=/usr/tools/config/17.1/repo.pub
PRODUCT_RELEASE=OPNsense-201701180833-OpenSSL
PRODUCT_SETTINGS=17.1
PRODUCT_SIGNCHK=/usr/tools/scripts/pkg_fingerprint.sh /usr/tools/config/17.1/repo.pub
PRODUCT_SIGNCMD=/usr/tools/scripts/pkg_sign.sh /usr/tools/config/17.1/repo.pub /usr/tools/config/17.1/repo.key
PRODUCT_SPEED=115200
PRODUCT_SUFFIX=
PRODUCT_TARGET=amd64
PRODUCT_TYPE=opnsense
PRODUCT_UEFI=yes
PRODUCT_VERSION=201701180833
PWD=/usr/tools/build
REMOTEHOST=10.0.1.142
SETSDIR=/tmp/sets
SETTINGS=17.1
SHELL=/usr/local/etc/rc.initial
SHLVL=2
SRCBRANCH=master
SRCDIR=/usr/src
SSH_CLIENT=10.0.1.142 49430 22
SSH_CONNECTION=10.0.1.142 49430 10.0.1.254 22
SSH_TTY=/dev/pts/0
STAGEDIR=/usr/obj/usr/tools/config/17.1/OpenSSL:amd64
STAGEDIRPREFIX=/usr/obj
STY=32120.up
TERM=screen
TERMCAP=SC|screen|VT 100/ANSI X3.64 virtual terminal:\
TOOLSBRANCH=master
TOOLSDIR=/usr/tools
USER=root
VENDOR=amd
WINDOW=0


I'm at a loss,
Any help would be appreciated.


Cheers

[franco]

Hi there,

We may be looking at this commit...

https://github.com/opnsense/ports/commit/57638c

It moved the dhcp6 home to https://github.com/hrs-allbsd/wide-dhcpv6

The history of the error snippet is not enough to see what's going on. Can you post the last few lines of the compiler output? I haven't seen this in our builds so far.


Cheers,
Franco

[ckurlinski]

Apologies for the late response, time, time, no time

Here is some of the build output for wide-dhcpv6

Code: [Select]

--- dhcp6_ctlclient.o ---
cc -O2 -pipe  -fPIC -fPIE -fstack-protector -fno-strict-aliasing -I. -DPACKAGE_NAME=\"wide-dhcpv6\" -DPACKAGE_TARNAME=\"wide-dhcpv6\" -DPACKAGE_VERSION=\"20080615\" -DPACKAGE_STRING=\"wide-dhcpv6\ 20080615\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DYYTEXT_POINTER=1 -DINET6=1 -DHAVE_GETADDRINFO=1 -DHAVE_GETNAMEINFO=1 -DHAVE_GETIFADDRS=1 -DHAVE_IF_NAMETOINDEX=1 -DHAVE_STRLCPY=1 -DHAVE_STRLCAT=1 -DHAVE_DAEMON=1 -DHAVE_WARNX=1 -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_FCNTL_H=1 -DHAVE_SYS_IOCTL_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_SYSLOG_H=1 -DHAVE_UNISTD_H=1 -DHAVE_IFADDRS_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_STRUCT_TM_TM_ZONE=1 -DHAVE_TM_ZONE=1 -DHAVE_SIG_ATOMIC_T=1 -DGETPGRP_VOID=1 -DRETSIGTYPE=void -DHAVE_MKTIME=1 -DHAVE_SELECT=1 -DHAVE_SOCKET=1 -DHAVE_CLOCK_GETTIME=1 -DHAVE_ARC4RANDOM=1 -DHAVE_TAILQ_FOREACH_REVERSE_OLD=1 -DHAVE_SA_LEN=1 -DHAVE_STDARG_H=1 -DSYSCONFDIR=\"/usr/local/etc\"  -DLOCALDBDIR=\"/var/db\" -c dhcp6_ctlclient.c -o dhcp6_ctlclient.o
--- auth.o ---
auth.c:145:21: warning: passing 'char *' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign [-Wpointer-sign]
        hmacmd5_init(&ctx, key->secret, key->secretlen);
                           ^~~~~~~~~~~
auth.c:91:65: note: passing argument to parameter here
static void hmacmd5_init __P((hmacmd5_t *, const unsigned char *,
                                                                ^
auth.c:146:23: warning: passing 'char *' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign [-Wpointer-sign]
        hmacmd5_update(&ctx, buf, len);
                             ^~~
auth.c:94:67: note: passing argument to parameter here
static void hmacmd5_update __P((hmacmd5_t *, const unsigned char *,
                                                                  ^
auth.c:181:21: warning: passing 'char *' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign [-Wpointer-sign]
        hmacmd5_init(&ctx, key->secret, key->secretlen);
                           ^~~~~~~~~~~
auth.c:91:65: note: passing argument to parameter here
static void hmacmd5_init __P((hmacmd5_t *, const unsigned char *,
                                                                ^
auth.c:182:23: warning: passing 'char *' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign [-Wpointer-sign]
        hmacmd5_update(&ctx, buf, len);
                             ^~~
auth.c:94:67: note: passing argument to parameter here
static void hmacmd5_update __P((hmacmd5_t *, const unsigned char *,
                                                                  ^
--- dhcp6c.o ---
dhcp6c.c:762:12: warning: promoted type 'int' of K&R function parameter is not compatible with the parameter type 'u_int16_t' (aka 'unsigned short') declared in a previous prototype [-Wknr-promoted-parameter]
        u_int16_t command;
                  ^
dhcp6c.c:123:54: note: previous declaration is here
static int client6_ifctl __P((char *ifname, u_int16_t));
                                                     ^
--- dhcp6_ctlclient.o ---
dhcp6_ctlclient.c:255:12: warning: promoted type 'int' of K&R function parameter is not compatible with the parameter type 'u_int16_t' (aka 'unsigned short') declared in a previous prototype [-Wknr-promoted-parameter]
        u_int16_t val;
                  ^
dhcp6_ctlclient.c:66:55: note: previous declaration is here
static inline int put16 __P((char **, int *, u_int16_t));
                                                      ^
--- common.o ---
common.c:2216:12: warning: promoted type 'int' of K&R function parameter is not compatible with the parameter type 'u_int16_t' (aka 'unsigned short') declared in a previous prototype [-Wknr-promoted-parameter]
        u_int16_t type, len;
                  ^
common.c:111:38: note: previous declaration is here
static int copy_option __P((u_int16_t, u_int16_t, void *, struct dhcp6opt **,
                                     ^
common.c:2216:18: warning: promoted type 'int' of K&R function parameter is not compatible with the parameter type 'u_int16_t' (aka 'unsigned short') declared in a previous prototype [-Wknr-promoted-parameter]
        u_int16_t type, len;
                        ^
common.c:111:49: note: previous declaration is here
static int copy_option __P((u_int16_t, u_int16_t, void *, struct dhcp6opt **,
                                                ^
common.c:3105:12: warning: promoted type 'int' of K&R function parameter is not compatible with the parameter type 'u_int16_t' (aka 'unsigned short') declared in a previous prototype [-Wknr-promoted-parameter]
        u_int16_t code;
                  ^
./common.h:173:44: note: previous declaration is here
extern char *dhcp6_stcodestr __P((u_int16_t));
                                           ^
--- y.tab.h ---
mv y.tab.c cfparse.c
--- cftoken.c ---
flex cftoken.l
--- dhcp6s.o ---
dhcp6s.c:2820:12: warning: promoted type 'int' of K&R function parameter is not compatible with the parameter type 'u_int16_t' (aka 'unsigned short') declared in a previous prototype [-Wknr-promoted-parameter]
        u_int16_t stcode;
                  ^
dhcp6s.c:192:57: note: previous declaration is here
static int make_ia_stcode __P((int, u_int32_t, u_int16_t,
                                                        ^
--- cftoken.c ---
mv lex.yy.c cftoken.c
--- cftoken.o ---
cc -O2 -pipe  -fPIC -fPIE -fstack-protector -fno-strict-aliasing -I. -DPACKAGE_NAME=\"wide-dhcpv6\" -DPACKAGE_TARNAME=\"wide-dhcpv6\" -DPACKAGE_VERSION=\"20080615\" -DPACKAGE_STRING=\"wide-dhcpv6\ 20080615\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DYYTEXT_POINTER=1 -DINET6=1 -DHAVE_GETADDRINFO=1 -DHAVE_GETNAMEINFO=1 -DHAVE_GETIFADDRS=1 -DHAVE_IF_NAMETOINDEX=1 -DHAVE_STRLCPY=1 -DHAVE_STRLCAT=1 -DHAVE_DAEMON=1 -DHAVE_WARNX=1 -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_FCNTL_H=1 -DHAVE_SYS_IOCTL_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_SYSLOG_H=1 -DHAVE_UNISTD_H=1 -DHAVE_IFADDRS_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_STRUCT_TM_TM_ZONE=1 -DHAVE_TM_ZONE=1 -DHAVE_SIG_ATOMIC_T=1 -DGETPGRP_VOID=1 -DRETSIGTYPE=void -DHAVE_MKTIME=1 -DHAVE_SELECT=1 -DHAVE_SOCKET=1 -DHAVE_CLOCK_GETTIME=1 -DHAVE_ARC4RANDOM=1 -DHAVE_TAILQ_FOREACH_REVERSE_OLD=1 -DHAVE_SA_LEN=1 -DHAVE_STDARG_H=1 -DSYSCONFDIR=\"/usr/local/etc\"  -DLOCALDBDIR=\"/var/db\" -c cftoken.c -o cftoken.o
--- dhcp6_ctlclient.o ---
1 warning generated.
--- cftoken.o ---
cftoken.l:75:9: warning: 'YYDEBUG' macro redefined [-Wmacro-redefined]
#define YYDEBUG 1
        ^
./y.tab.h:37:10: note: previous definition is here
# define YYDEBUG 0
         ^
--- cfparse.c ---
mv y.tab.c cfparse.c
mv: y.tab.c: No such file or directory
*** [cfparse.c] Error code 1

make[1]: stopped in /usr/obj/usr/ports/net/dhcp6/work/wide-dhcpv6-20080615.1
--- auth.o ---
4 warnings generated.
--- dhcp6c.o ---
1 warning generated.
--- cftoken.o ---
1 warning generated.
--- common.o ---
3 warnings generated.
--- dhcp6s.o ---
1 warning generated.
1 error

make[1]: stopped in /usr/obj/usr/ports/net/dhcp6/work/wide-dhcpv6-20080615.1
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped in /usr/ports/net/dhcp6

Thanks for the help

[franco]

I ran into this today during cross-building for armv6:

https://github.com/opnsense/ports/commit/bdc4433b8b


Cheers,
Franco

[ckurlinski]

Thanks
Home sick right now, I will try it.

[franco]

Ok, no rush. Get well soon!

[vivekmauli14]

Hi,

I'm trying to automate the build process for my newer commits in core.git. Does OPNsense currently have any CI/CD mechanisms implemented? If not, are there any recommended practices or tools to streamline this process?

Any leads would be appreciated.

Thanks,
VivekSP

[franco]

Take a look at the "nightly" build target.

https://github.com/opnsense/tools?tab=readme-ov-file#composite-build-steps


Cheers,
Franco