Trying to build opnsense from scratch (make serial), failed on cciss_vol_status

[john314]

Hello,

I'm a long time pfsense user, thought I would give opnsense a try, and I would like to get familiar with the source code and how to build it, so I tried building opnsense from scratch.  Followed directions from here:
https://github.com/opnsense/tools/blob/master/README.md

I installed FreeBSD 13.2 Release for amd64 in a proxmox VM (i440fx), 32GB ram, 100GB disk space. 
I cloned the freebsd source code into my VM:
root@freebsd1:~ # pkg install git
root@freebsd1:~ # git clone --branch releng/13.2 https://git.FreeBSD.org/src.git /usr/src

Then:
root@freebsd1:~ # cd /tmp
root@freebsd1:/tmp # mkdir opnsense
root@freebsd1:/tmp # cd opnsense/
root@freebsd1:/tmp/opnsense # git clone https://github.com/opnsense/tools
root@freebsd1:/tmp/opnsense # cd tools
root@freebsd1:/tmp/opnsense/tools # ROOTDIR=/tmp/opnsense make update
<this succeeded>
root@freebsd1:/tmp/opnsense/tools # env ROOTDIR=/tmp/opnsense make serial

This ended up failing on the pkg version.  I googled that, found and did the fix:

root@freebsd1:/tmp/opnsense/tools # cd /tmp/opnsense/ports/ports-mgmt/pkg
root@freebsd1:/tmp/opnsense/ports/ports-mgmt/pkg # make && make reinstall

This succeeded, I went back into /tmp/opnsense/tools, and ran make again:

root@freebsd1:/tmp/opnsense/ports/ports-mgmt/pkg # cd /tmp/opnsense/tools
root@freebsd1:/tmp/opnsense/tools # env ROOTDIR=/tmp/opnsense make serial

This eventually fails with a bunch of errors compiling cciss_vol_status (see detailed output below).  Looking closely at the error messages and the configure output above the compile errors, it says it can't find "/usr/src/sys/dev/ciss/cissio.h", but this file IS on my system.  So I'm not sure why it can't find it.

Also in the output it says:
make[2]: stopped in /usr/obj/tmp/opnsense/ports/sysutils/cciss_vol_status/work/cciss_vol_status-1.12a
1 error

But that build directory doesn't even exist:
root@freebsd1:/usr/obj/tmp/opnsense # ls
src     tools
root@freebsd1:/usr/obj/tmp/opnsense #


I tried re-running it with MAKE_JOBS_UNSAFE=yes:
root@freebsd1:/tmp/opnsense/tools # env ROOTDIR=/tmp/opnsense  MAKE_JOBS_UNSAFE=yes make serial
Got the same build errors for cciss_vol_status. 

Any help would be greatly appreciated.  I tried following the directions closely, but maybe I missed something.


Here are the specific cciss_vol_status build errors:

[663/664] Deinstalling rhash-1.4.4_1...
[663/664] Deleting files for rhash-1.4.4_1: 100%
[664/664] Deinstalling mbedtls-2.28.7...
[664/664] Deleting files for mbedtls-2.28.7: 100%
[20240223154217] ===> License GPLv2 accepted by the user
===>   cciss_vol_status-1.12a depends on file: /usr/local/sbin/pkg - not found
===>   Installing existing package /.pkg-cache/All/pkg-1.19.2_1.pkg
Installing pkg-1.19.2_1...
Extracting pkg-1.19.2_1: 100%
===>   cciss_vol_status-1.12a depends on file: /usr/local/sbin/pkg - found
===>   Returning to build of cciss_vol_status-1.12a
[20240223154218] => cciss_vol_status-1.12a.tar.gz doesn't seem to exist in /tmp/opnsense/ports/distfiles/.
[20240223154218] => Attempting to fetch https://downloads.sourceforge.net/project/cciss/cciss_vol_status/cciss_vol_status-1.12a.tar.gz
cciss_vol_status-1.12a.tar.gz                          117 kB  649 kBps    00s
[20240223154219] ===> Fetching all distfiles required by cciss_vol_status-1.12a for building
[20240223154219] ===> Extracting for cciss_vol_status-1.12a
[20240223154219] => SHA256 Checksum OK for cciss_vol_status-1.12a.tar.gz.
[20240223154219] ===> Patching for cciss_vol_status-1.12a
[20240223154219] ===> Configuring for cciss_vol_status-1.12a
configure: loading site script /tmp/opnsense/ports/Templates/config.site
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
/usr/obj/tmp/opnsense/ports/sysutils/cciss_vol_status/work/cciss_vol_status-1.12a/missing: Unknown `--is-lightweight' option
Try `/usr/obj/tmp/opnsense/ports/sysutils/cciss_vol_status/work/cciss_vol_status-1.12a/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing
checking for a thread-safe mkdir -p... (cached) /bin/mkdir -p
checking for gawk... (cached) /usr/bin/awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking whether cc understands -c and -o together... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of cc... gcc3
checking how to run the C preprocessor... cpp
checking for grep that handles long lines and -e... (cached) /usr/bin/grep
checking for egrep... (cached) /usr/bin/egrep
checking for ANSI C header files... (cached) yes
checking for sys/types.h... (cached) yes
checking for sys/stat.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for memory.h... (cached) yes
checking for strings.h... (cached) yes
checking for inttypes.h... (cached) yes
checking for stdint.h... (cached) yes
checking for unistd.h... (cached) yes
checking for stdbool.h that conforms to C99... (cached) yes
checking for _Bool... (cached) yes
checking for errno.h... (cached) yes
checking for fcntl.h... (cached) yes
checking for inttypes.h... (cached) yes
checking for netinet/in.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for sys/ioctl.h... (cached) yes
checking for unistd.h... (cached) yes
checking scsi/sg.h usability... no
checking scsi/sg.h presence... no
checking for scsi/sg.h... no
configure: WARNING: scsi/sg.h not found so MSA1000 support will not be compiled.
checking scsi/scsi.h usability... no
checking scsi/scsi.h presence... no
checking for scsi/scsi.h... no
checking linux/cciss_ioctl.h usability... no
checking linux/cciss_ioctl.h presence... no
checking for linux/cciss_ioctl.h... no
checking /usr/src/sys/dev/ciss/cissio.h usability... no
checking /usr/src/sys/dev/ciss/cissio.h presence... no
checking for /usr/src/sys/dev/ciss/cissio.h... no
checking for an ANSI C-conforming const... yes
checking whether cc needs -traditional... no
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... (cached) yes
checking for getopt... (cached) yes
checking for getopt_long... yes
checking for memset... (cached) yes
checking for strerror... (cached) yes
checking for strstr... (cached) yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: executing depfiles commands
[20240223154221] ===> Building for cciss_vol_status-1.12a
--- all ---
/usr/bin/make  all-am
--- cciss_vol_status.o ---
cc -DHAVE_CONFIG_H -I.      -O2 -pipe  -fstack-protector-strong -fno-strict-aliasing -MT cciss_vol_status.o -MD -MP -MF .deps/cciss_vol_status.Tpo -c -o cciss_vol_status.o cciss_vol_status.c
cciss_vol_status.c:94:2: error: Neither cciss_ioctl.h (linux) nor cissio.h (FreeBSD) were found.
#error Neither cciss_ioctl.h (linux) nor cissio.h (FreeBSD) were found.
 ^
cciss_vol_status.c:198:2: warning: Since <scsi/sg.h> is not around, MSA1000 support will not be compiled. [-W#warnings]
#warning Since <scsi/sg.h> is not around, MSA1000 support will not be compiled.
 ^
cciss_vol_status.c:722:11: error: unknown type name 'IOCTL_Command_struct'
                int rc, IOCTL_Command_struct *cmd, int ldrive_num)
                        ^
cciss_vol_status.c:762:29: error: unknown type name 'IOCTL_Command_struct'
static void setup_for_ioctl(IOCTL_Command_struct *cmd,
                            ^
cciss_vol_status.c:773:27: error: use of undeclared identifier 'TYPE_CMD'
        cmd->Request.Type.Type = TYPE_CMD;
                                 ^
cciss_vol_status.c:774:32: error: use of undeclared identifier 'ATTR_SIMPLE'
        cmd->Request.Type.Attribute = ATTR_SIMPLE;
                                      ^
cciss_vol_status.c:775:52: error: use of undeclared identifier 'XFER_WRITE'
        cmd->Request.Type.Direction = (write_direction) ? XFER_WRITE : XFER_READ;
                                                          ^
cciss_vol_status.c:775:65: error: use of undeclared identifier 'XFER_READ'
        cmd->Request.Type.Direction = (write_direction) ? XFER_WRITE : XFER_READ;
                                                                       ^
cciss_vol_status.c:809:2: error: use of undeclared identifier 'IOCTL_Command_struct'
        IOCTL_Command_struct cmd;
        ^
cciss_vol_status.c:824:19: error: use of undeclared identifier 'cmd'; did you mean 'rcmd'?
        setup_for_ioctl(&cmd, zero_lun, cdb, 12, 0, lun, *count * ;
                         ^~~
                         rcmd
/usr/include/unistd.h:546:6: note: 'rcmd' declared here
int      rcmd(char **, int, const char *, const char *, const char *, int *);
         ^
cciss_vol_status.c:825:17: error: use of undeclared identifier 'CCISS_PASSTHRU'
        rc = ioctl(fd, CCISS_PASSTHRU, &cmd);
                       ^
cciss_vol_status.c:825:34: error: use of undeclared identifier 'cmd'
        rc = ioctl(fd, CCISS_PASSTHRU, &cmd);
                                        ^
cciss_vol_status.c:831:38: error: use of undeclared identifier 'cmd'; did you mean 'rcmd'?
        CHECK_IOCTL(filename, cmdname, rc, &cmd, -1); /* macro which may return... */
                                            ^~~
                                            rcmd
cciss_vol_status.c:759:49: note: expanded from macro 'CHECK_IOCTL'
        if (check_ioctl_results(filename, command, rc, cmd, ldrive_num) !=0) \
                                                       ^
/usr/include/unistd.h:546:6: note: 'rcmd' declared here
int      rcmd(char **, int, const char *, const char *, const char *, int *);
         ^
cciss_vol_status.c:849:2: error: use of undeclared identifier 'IOCTL_Command_struct'
        IOCTL_Command_struct cmd;
        ^
cciss_vol_status.c:860:19: error: use of undeclared identifier 'cmd'; did you mean 'rcmd'?
        setup_for_ioctl(&cmd, controller_lun, cdb, 16, 0, buffer, 512);
                         ^~~
                         rcmd
/usr/include/unistd.h:546:6: note: 'rcmd' declared here
int      rcmd(char **, int, const char *, const char *, const char *, int *);
         ^
cciss_vol_status.c:861:17: error: use of undeclared identifier 'CCISS_PASSTHRU'
        rc = ioctl(fd, CCISS_PASSTHRU, &cmd);
                       ^
cciss_vol_status.c:861:34: error: use of undeclared identifier 'cmd'
        rc = ioctl(fd, CCISS_PASSTHRU, &cmd);
                                        ^
cciss_vol_status.c:862:55: error: use of undeclared identifier 'cmd'; did you mean 'rcmd'?
        CHECK_IOCTL(filename, "IDENTIFY_LOGICAL_DRIVE", rc, &cmd, -1); /* macro which may return... */
                                                             ^~~
                                                             rcmd
cciss_vol_status.c:759:49: note: expanded from macro 'CHECK_IOCTL'
        if (check_ioctl_results(filename, command, rc, cmd, ldrive_num) !=0) \
                                                       ^
/usr/include/unistd.h:546:6: note: 'rcmd' declared here
int      rcmd(char **, int, const char *, const char *, const char *, int *);
         ^
cciss_vol_status.c:870:2: error: use of undeclared identifier 'IOCTL_Command_struct'
        IOCTL_Command_struct cmd;
        ^
cciss_vol_status.c:879:19: error: use of undeclared identifier 'cmd'; did you mean 'rcmd'?
        setup_for_ioctl(&cmd, controller_lun, cdb, SENSE_CACHE_CONFIG_CDB_LEN, 0,
                         ^~~
                         rcmd
/usr/include/unistd.h:546:6: note: 'rcmd' declared here
int      rcmd(char **, int, const char *, const char *, const char *, int *);
         ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
*** [cciss_vol_status.o] Error code 1

make[2]: stopped in /usr/obj/tmp/opnsense/ports/sysutils/cciss_vol_status/work/cciss_vol_status-1.12a
1 error

make[2]: stopped in /usr/obj/tmp/opnsense/ports/sysutils/cciss_vol_status/work/cciss_vol_status-1.12a

make[1]: stopped in /usr/obj/tmp/opnsense/ports/sysutils/cciss_vol_status/work/cciss_vol_status-1.12a
[20240223154221] ===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped in /tmp/opnsense/ports/sysutils/cciss_vol_status
[20240223154222] ===> Cleaning for cciss_vol_status-1.12a
Updating database digests format: 100%
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages:

Installed packages to be REMOVED:
        pkg: 1.19.2_1

Number of packages to be removed: 1

The operation will free 15 MiB.
[1/1] Deinstalling pkg-1.19.2_1...
[1/1] Deleting files for pkg-1.19.2_1: 100%
Creating repository in /usr/obj/tmp/opnsense/tools/config/24.1/amd64/.pkg-new/: 100%
Packing files for repository: 100%
>>> Removing packages set
>>> Creating package mirror set for 24.1.2_110-amd64... done
-rw-r--r--  1 root  wheel   846M Feb 23 09:42 packages-24.1.2_110-amd64.tar
>>> ERROR: The build encountered fatal issues!
>>> Aborted version 1.12a for sysutils/cciss_vol_status (cciss_vol_status)
*** Error code 1

Stop.
make: stopped in /tmp/opnsense/tools
root@freebsd1:/tmp/opnsense/tools #
root@freebsd1:/tmp/opnsense/tools #
root@freebsd1:/tmp/opnsense/tools # ls /usr/src/sys/dev/ciss
ciss.c          cissio.h        cissreg.h       cissvar.h
root@freebsd1:/tmp/opnsense/tools #
root@freebsd1:/tmp/opnsense/tools #



Thanks,
John

[john314]

I thought perhaps the error was from trying to build in /tmp/opnsense and restricting the ROOTDIR to /tmp/opnsense, so I reverted my VM to a snapshot just after cloning opnsense tools, and went to rebuild in /usr/tools:

/usr/tools # make update

This gave me an error:

Code: [Select]

>>> Updating branch stable/24.1 of /usr/src:
error: pathspec 'stable/24.1' did not match any file(s) known to git
Already up to date.
>>> Fetching /usr/tools:
>>> Updating branch master of /usr/tools:
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
root@freebsd1:/usr/tools #

I tried doing "make serial" anyways but it failed with an error very quickly (I don't have the details, but it didn't like the branch in /usr/src).  So that made me think, perhaps the problem is the contents of my /usr/src dir, which I got by cloning this branch (which I assumed was needed before doing the opnsense tools build):
# git clone --branch releng/13.2 https://git.FreeBSD.org/src.git /usr/src

So I reverted my VM again, and deleted everything under /usr/src.  Then I cloned opnsense/tools, and did "make update", and this time I didn't get any errors for that, and it populated /usr/src with "stable/24.1".  I rebuilt "pkg" to avoid that error and then I built opnsense:

/usr/tools # make serial

This went for hours and failed with this error:

Code: [Select]

[9/10] Deleting files for pkgconf-2.0.3_2,1: 100%
[10/10] Deinstalling expat-2.6.0...
[10/10] Deleting files for expat-2.6.0: 100%
Creating repository in /usr/obj/usr/tools/config/24.1/amd64/.pkg-new/: 100%
Packing files for repository: 100%
>>> Removing packages set
>>> Creating package mirror set for 24.1.2_226-amd64... done
-rw-r--r--  1 root  wheel   820M Mar  1 01:02 packages-24.1.2_226-amd64.tar
>>> ERROR: The build encountered fatal issues!
>>> Aborted version 0.9.2 for databases/py-duckdb@py39 (py39-duckdb)
*** Error code 1

Stop.
make: stopped in /usr/tools
root@freebsd1:/usr/tools #

The scroll history in the console wasn't big enough for me to see the compile error details, so I did "make serial" again, figuring it would fail quickly and give me those details.  But, it ran for a few hours and eventually succeeded! 

Code: [Select]

Populating `/usr/obj/usr/tools/config/24.1/amd64/root.part'
Image `/usr/obj/usr/tools/config/24.1/amd64/root.part' complete
Creating `/usr/obj/usr/tools/config/24.1/amd64/efiboot.img'
/usr/obj/usr/tools/config/24.1/amd64/efiboot.img: 65528 sectors in 65528 FAT32 clusters (512 bytes/cluster)
BytesPerSec=512 SecPerClust=1 ResSectors=32 FATs=2 Media=0xf0 SecPerTrack=63 Heads=255 HiddenSecs=0 HugeSectors=66584 FATsecs=512 RootCluster=2 FSInfo=1 Backup=2
Populating `/usr/obj/usr/tools/config/24.1/amd64/efiboot.img'
Image `/usr/obj/usr/tools/config/24.1/amd64/efiboot.img' complete
>>> Building serial image... done
root@freebsd1:/usr/tools #


I'm wondering if that build failure on "py-duckdb" is because I have 4 cores allocated to this VM, so maybe it was due to a parallel build issue?   Does anyone know if that is the cause of that error?

I'm going to revert and retry the build from scratch, and configure my VM to only have 1 core.  I want to go from start to finish without any errors.

[Maurice]

Multiple cores aren't the issue, I frequently build OPNsense on 16+ cores.
Don't clone the FreeBSD src repo. OPNsense uses its own fork (https://github.com/opnsense/src) which is automatically cloned by make update.

Cheers
Maurice

[john314]

Understood on the "make update" populates /usr/src, thanks for the clarification.

I ran this a couple more times with 4 cores and 32GB RAM, it's reproducible, keeps failing same place.  The build runs for just over 12 hours before it fails/stops, but the fatal error appears to happen rather early, after 1-2 hours.  I tee'd the build output to a file (ended up 175 megs!), and the fatal error related to py39-duckdb is from trying to build one of py39-duckdb's dependencies.  Here are the details for that:

Code: [Select]

20240302203802] => python-dateutil-2.8.2.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
[20240302203802] => Attempting to fetch https://files.pythonhosted.org/packages/source/p/python-dateutil/python-dateutil-2.8.2.tar.gz
python-dateutil-2.8.2.tar.gz                           348 kB 3463 kBps    00s
[20240302203802] ===> Fetching all distfiles required by py39-dateutil-2.8.2 for building
[20240302203802] ===> Extracting for py39-dateutil-2.8.2
[20240302203802] => SHA256 Checksum OK for python-dateutil-2.8.2.tar.gz.
[20240302203802] ===> Patching for py39-dateutil-2.8.2
===>   py39-dateutil-2.8.2 depends on package: py39-setuptools-scm>=0 - not found
[20240302203802] ===> License MIT accepted by the user
===>   py39-setuptools-scm-8.0.4 depends on file: /usr/local/sbin/pkg - found
[20240302203802] => setuptools-scm-8.0.4.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
[20240302203802] => Attempting to fetch https://files.pythonhosted.org/packages/source/s/setuptools-scm/setuptools-scm-8.0.4.tar.gz
setuptools-scm-8.0.4.tar.gz                             72 kB 2259 kBps    00s
[20240302203802] ===> Fetching all distfiles required by py39-setuptools-scm-8.0.4 for building
[20240302203802] ===> Extracting for py39-setuptools-scm-8.0.4
[20240302203802] => SHA256 Checksum OK for setuptools-scm-8.0.4.tar.gz.
[20240302203803] ===> Patching for py39-setuptools-scm-8.0.4
===>   py39-setuptools-scm-8.0.4 depends on package: py39-setuptools>0 - found
===>   py39-setuptools-scm-8.0.4 depends on package: py39-wheel>0 - found
===>   py39-setuptools-scm-8.0.4 depends on file: /usr/local/bin/python3.9 - found
===>   py39-setuptools-scm-8.0.4 depends on package: py39-tomli>0 - found
===>   py39-setuptools-scm-8.0.4 depends on package: py39-build>=0 - found
===>   py39-setuptools-scm-8.0.4 depends on package: py39-installer>=0 - found
[20240302203803] ===> Configuring for py39-setuptools-scm-8.0.4
[20240302203803] ===> Building for py39-setuptools-scm-8.0.4
* Getting build dependencies for wheel...
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
    main()
  File "/usr/local/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/local/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
    return hook(config_settings)
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 177, in get_requires_for_build_wheel
    return self._get_build_requires(
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
    self.run_setup()
  File "/usr/local/lib/python3.9/site-packages/setuptools/build_meta.py", line 174, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 139, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 476, in __init__
    _Distribution.__init__(
  File "/usr/local/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 275, in __init__
    self.finalize_options()
  File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 899, in finalize_options
    for ep in sorted(loaded, key=by_order):
  File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 898, in <lambda>
    loaded = map(lambda e: e.load(), filtered)
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 196, in load
    return functools.reduce(getattr, attrs, module)
AttributeError: module 'setuptools_scm.integration' has no attribute 'infer_version'

ERROR Backend subprocess exited when trying to invoke get_requires_for_build_wheel
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/devel/py-setuptools-scm
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/devel/py-dateutil
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/math/py-pandas
*** Error code 1

Stop.
make: stopped in /usr/ports/databases/py-duckdb
[20240302203805] ===> Cleaning for py39-setuptools_scm-6.4.2
[20240302203805] ===> Cleaning for py39-pybind11-2.11.1
[20240302203806] ===> Cleaning for py39-numpy-1.25.0_6,1
[20240302203807] ===> Cleaning for py39-cython-0.29.37
[20240302203807] ===> Cleaning for py39-pandas-2.0.3_1,1
[20240302203808] ===> Cleaning for py39-versioneer-0.29
[20240302203808] ===> Cleaning for py39-dateutil-2.8.2
[20240302203808] ===> Cleaning for py39-setuptools-scm-8.0.4
[20240302203808] ===> Cleaning for py39-duckdb-0.9.2
Updating database digests format: .......... done
Checking integrity... done (0 conflicting)

Like I said, this error in the build output happens rather early, and it keeps going for ~10 more hours before finally stopping and saying that py39-duckdb failed.  I wish that it would stop right when something fails, rather than keep going.  Am I the only one seeing this?  I'm trying to do the most straightforward build by the instructions.  Any help would be greatly appreciated.

Update:  looking further in the build output after the above error messages, it looks like py39-setuptools-scm-8.0.4 DOES end up building successfully.  And after the build fails, I do "make serial" again, it all completes successfully after another 45 minutes.  Seems like the build errors I'm seeing are enough to eventually trigger the build to fail, but doesn't really stop it from building that component (py39-setuptools-scm-8.0.4).  Weird.

Thanks,
John

[john314]

Looks like I'm not the only one seeing this "py39-setuptools-scm-8.0.4" build issue:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277472

For me it's not intermittent, it happens every time.  But the bug describes the exact same error, so that's good.  I kept wondering if it was my env somehow. 

[franco]

Well, coincidentally, that's my report to FreeBSD ports.

I wrote "intermittent" because although it's reproducible from scratch it stops failing afterwards and might not light up in the FreeBSD package build toolchain for mildly related reasons.


Cheers,
Franco

[franco]

I think it's fixed now in our ports tree. The py-setuptools_scm port was renamed to py-setuptools-scm which didn't reflect in py-duckdb because we are still at version 0.9 while FreeBSD is at 0.10.


Cheers,
Franco

[john314]

Thanks Franco for the additional info.

I retried a full clean build yesterday.  I did a "git pull" in /usr/tools, redid "make update", and then did "make serial" and it succeeded.  I used 8 cores and 64GB of RAM this time, finished in ~10 hours.  So it appears that this is indeed fixed.  And I didn't see any build errors at all in the build output.

-John