Messages

Hi,

I am trying to build ARM64 images for opnsense and now I have to add manually two files after I dd the image to the microSD.

I found a small guide for git on this post, but the images won't have the same files I commit using this commands.

I use the following line to build:

make -C /usr/tools/ VERSION=$TAG_SRC DEVICE=$DEVICE base

Where TAG_SRC on the last build has the value of 25.1.4.

I did:

Code Select Expand

git tag -f -m "message" 25.1.4

and then on the /usr/core (where my commited files are) I run:

Code Select Expand

git checkout 25.1.4

and there my files are what I expect them to be.

My knowledge of git is not much, if anyone has any hints here :)

I want to include my fingerprint for the packages and a new /usr/local/etc/rc script for ARM64 gpart resizing.

Thanks in advance :)

none

Hi Franco,

I got to walk a few steps here, here is the result.

I found that the line:

Code Select Expand

        local IDX=${1##"${DEV}p"}
in /usr/local/etc/rc was returning a value that would make the next "if" always be true:

Code Select Expand

        if [ "${IDX}" = "${1}" ]; then
                # assume schema ada0a used by nano images
                IDX=1
        fi
This way when disks configured as MBR, as is the case here, would always default to IDX=1 and make it to the next "if":

Code Select Expand

       if [ -n "${DEV}" -a "${DEV}" != "${1}" -a \
            -n "${IDX}" -a "${IDX}" != "${1}" ]; then
                gpart recover ${DEV}
                gpart resize -i ${IDX} ${DEV}
       fi
issuing gpart resize on a partition that has no place to grow (and not the root one, also). The partitions here are organized like this:

Code Select Expand

gpart show da11
=>      63  61069249  da11  MBR  (29G)
        63    131009        - free -  (64M)
    131072     65536     1  fat32lba  [active]  (32M)
    196608   6094848     2  freebsd  (2.9G)
   6291456  54777856        - free -  (26G)
Just as the default opnsense scripts build. Running on index 1 would not meet the premise for the growfs step that follows, and then get that message saying it already has the desired size.

For the Nano image that would do fine, as it has only 1 partition, the BSD scheme is written directly on the device, no MBR or GPT is created.

To address this I created a test to know if it is a MBR scheme, and then act accordingly. That way it finds the index for the root file system and the gpart resize step works.

Now I got a message from gpart tellimg me I need to do a commit to the UFS label (my case was ufs/OPNsense. Not doing that would ruin the boot:

Code Select Expand

mmcsd0 recovering is not needed
GEOM_PART: ufs/OPNsense was automatically resized.
  Use `gpart commit ufs/OPNsense` to save changes or `gpart undo ufs/OPNsense` to revert them.
mmcsd0s2 resized
That step is outside the grow_partition(), and later I found was not welcome on AMD64 Nano image (may be arch related?). I then created another test to run only on MBR schemes.

My skills here are far from yours and the opnsense team. Here is the file I ended with, I hope my humble time here can help anyhow.

My tests here are also limited. Nano images works as is now and keeps working after the modifications. ARM64 images now work in the first boot. I just need to learn how to commit changes to the local repo, so images will have it out-of-the-box either.

Thanks for the help, as usual :)

The dmesg from that part is:

Code Select Expand

Booting [/boot/kernel/kernel]...               
Using DTB provided by EFI at 0xef03c000.
����#U}�qqC��� ����!����*���mu��������k�饭�9� ���5������j�j�����������)��y��i*���Ovq4�y���,�J�uQ�����4����+��������몋���
                                                                                                                        ���8�q����i*���
                                                                                                                                       +�v����J��:u�Ѩ��V[]����U8��x��y>k�8�ռy�Ѫ��i�d���i�����.�����8;�u�+����+�ꪨ�,���+���ʊ�����ʊ���+��v�����+��)�ź����*�����l�"�)��᪩��0���uW�S�I+��4��{qɩ��J����+���0��9},����e����Q���}����..��Y��0�ټ�.�g�����+�}���*�����(��k��W;�+*���<�ܑ����������y�J�h������"�(����(��J��$��b*��,��٩���9��ը�����3 �����q�WS[�mq=1>Q2ը��.��z����>A�Y�q5��q��W�騤�ɪ�`�ѤY�Q1QA��8��m+�+��,���m�����)������"�������ъ(�#��������r�*�u�u���������f+����U1��*�b���0�������������+���q�t�9���%��Z�ᩴu�]T���ԼQWI�QS����]�8<������2��*�S�*��j�r��������n�*�j�*[TWS�>��<�,����K
                                                                                                                                                                                          �++��Y)����,(�誶��$�+���uNJ�ي+k����;z���J�������;��գ��ű��
    ����Y�?�����՜u������q���������Y8����񱁙�����(�����a����ɛ�Q:��(���������**�ͣ�����f���Q�Z��銊�����9x��+��e.������       ~qq���z��!��**+���$����(��8x�ዧ������}��(���Ț�Z�)���������动���::I�����1::I�Ŋ�:����!::�����*��8
                                                                                                                                                                                                                     ������uhub0: 1 port with 1 removable, self powered
uhub3: 1 port with 1 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
Mounting filesystems...
tunefs: soft updates remains unchanged as enabled
uhub1: 1 port with 1 removable, self powered
uhub4: 1 port with 1 removable, self powered
camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed
cam_lookup_pass: No such file or directory
cam_lookup_pass: either the pass driver isn't in your kernel
cam_lookup_pass: or mmcsd0 doesn't exist
mmcsd0 recovering is not needed
GEOM_PART: ufs/OPNsense was automatically resized.
  Use `gpart commit ufs/OPNsense` to save changes or `gpart undo ufs/OPNsense` to revert them.
mmcsd0s2 resized
super-block backups (for fsck_ffs -b #) at:
 6402432, 7682880, 8963328, 10243776, 11524224, 12804672, 14085120, 15365568,
 16646016, 17926464, 19206912, 20487360, 21767808, 23048256, 24328704,
 25609152, 26889600, 28170048, 29450496, 30730944, 32011392, 33291840,
 34572288, 35852736, 37133184, 38413632, 39694080, 40974528, 42254976,
 43535424, 44815872, 46096320, 47376768, 48657216, 49937664, 51218112,
 52498560, 53779008, 55059456, 56339904, 57620352, 58900800, 60181248
** /dev/ufs/OPNsense
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 6831398 free (94 frags, 853913 blocks, 0.0% fragmentation)
Setting hostuuid: 00000000-0000-0000-eb8b-ff6f21811d36.
Setting hostid: 0x5f653895.
>>> Invoking import script 'importer'
Press any key to start the configuration importer: .......
Bootstrapping config.xml...done.


Code Select Expand

FreeBSD/arm64 (OPNsense.localdomain) (ttyu1)

login: root
Password:
------------------------------------------------
|       Hello, this is OPNsense 25.1           |           :::::::.
|                                              |           :::::::::.
|  Website:     https://opnsense.org/          |        :::        :::
|  Handbook:    https://docs.opnsense.org/     |        :::        :::
|  Forums:      https://forum.opnsense.org/    |        :::        :::
|  Code:        https://github.com/opnsense    |         `:::::::::
|  Reddit:      https://reddit.com/r/opnsense  |           `:::::::
------------------------------------------------

*** OPNsense.localdomain: OPNsense 25.1.3_33 (aarch64) ***

 LAN (re0)       -> v4: 192.168.1.1/24
 WAN (re1)       ->

 HTTPS: sha256 2B 47 72 39 88 B9 32 31 2F 65 59 19 61 F6 85 76
               08 19 F7 E9 39 34 18 F8 97 70 A1 58 50 46 2C 41

  0) Logout                              7) Ping host
  1) Assign interfaces                   8) Shell
  2) Set interface IP address            9) pfTop
  3) Reset the root password            10) Firewall log
  4) Reset to factory defaults          11) Reload all services
  5) Power off system                   12) Update from console
  6) Reboot system                      13) Restore a backup

Enter an option: 8

root@OPNsense:~ # df -h
Filesystem                   Size    Used   Avail Capacity  Mounted on
/dev/ufs/OPNsense             28G    2.0G     24G     8%    /
devfs                        1.0K      0B    1.0K     0%    /dev
/dev/msdosfs/MSDOSBOOT        32M    832K     31M     3%    /boot/msdos
devfs                        1.0K      0B    1.0K     0%    /var/dhcpd/dev
devfs                        1.0K      0B    1.0K     0%    /var/unbound/dev
/usr/local/lib/python3.11     28G    2.0G     24G     8%    /var/unbound/usr/local/lib/python3.11
/lib                          28G    2.0G     24G     8%    /var/unbound/lib


> so even in the nano image the the resizing is not working?

Says you? Last time I checked it worked, which was 25.1...

No, please don't get me wrong. I misunderstood your post, so I asked to be sure. I thought you were saying that was not working anymore, and now I see I got it wrong.

You said earlier it worked for you on amd64 and as I read it I got confused, as if you had tested again and said was not working anymore. Sorry for the noise :(

I just was trying to get something to compare and have a starting point. And now I have it. Thanks again.

none

Hi Franco,

so even in the nano image the the resizing is not working?

I will fix a pc for testing this and see what I can help.

Thanks again.

none

Hi Franco, can you point me out a case that should work so I can try to look it and reproduce here? What image to test?

I want to know what disk partitioning there should be so the growfs will work.

thanks,

none

Very same thing happens when using USB to boot from, and this time its u-boot firmware:

da0 recovering is not needed
da0s1 resized
growfs: requested size 3.0GB is equal to the current filesystem size 3.0GB

I used the same build files for both images.

none

I started the R5S using microSD and got this:

mmcsd0: 31GB <SDHC SD32G 6.1 SN 77400AE2 MFG 10/2020 by 159 TI> at mmc0 50.0MHz/4bit/1016-block

At least now it can figure all microSD size. Tried create again the file, no good.

Will try USB stick and that image.

none

Hi,

thanks again for the help.

So you think:

Code Select Expand

camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed
cam_lookup_pass: No such file or directory
cam_lookup_pass: either the pass driver isn't in your kernel
cam_lookup_pass: or mmcsd0 doesn't exist
mmcsd0 recovering is not needed

this error could be the culprit?

If I got it right, the scripting system detects the size of the media, and then compares to the size of the partition.

I will try again using usb drive and will boot again the microSD to figure out the line the kernel notices the microSD hardware and size. May that be on the 1500000 part of the log (I will try to change the image to that speed).

Together will try it on R4S, a more stable board that opnsense has a config for building. My goal is to submit a conf for R5S, but I want to have it tested as much as I can. And I don't know if opnsense can have EDK2 bytes on the code. This working image need it. The U-Boot is not working for both 2.5Gbps nics.

Again thanks, I will try to dig further here.

none

the part of dmesg that is not readable is where the kernel changes the serial to 1500000 baud rate. Other times, its 115200 and is readable.

Thanks again,

none

Code Select Expand


?cLoading kernel...
/boot/kernel/kernel text=0x2a8 text=0xa3c030 text=0x28630c data=0x15e678 data=0x
0+0x2c3000 0x8+0x1634b8+0x8+0x18e7bb
Loading configured modules...
/etc/hostid...can't find '/etc/hostid'
failed!
carp.../boot/kernel/carp.ko text=0x3888 text=0x78b8 data=0xe28+0x48 0x8+0x1b48+0
x8+0x13cc
if_gre.../boot/kernel/if_gre.ko text=0x254e text=0x3f10 data=0x950+0x40 0x8+0x16
f8+0x8+0xfdd
ugen...can't find 'ugen'
failed!
/boot/modules/if_re.ko.../boot/modules/if_re.ko text=0x7988 text=0x7ccf8 data=0x
a38+0x14 0x8+0x1bd8+0x8+0x15f7
if_tun...usb...uhid.../boot/kernel/uhid.ko text=0x1fc0 text=0x1660 data=0x6f8+0x
4 0x8+0xe88+0x8+0xa61
/boot/entropy.../boot/entropy size=0x1000
ukbd...pfsync.../boot/kernel/pfsync.ko text=0x38b9 text=0x8fd4 data=0xd00+0x8 0x
8+0x1ba8+0x8+0x1485
loading required module 'pf'
/boot/kernel/pf.ko text=0x12340 text=0x4a610 data=0x6bc8+0x484 0x8+0x7020+0x8+0x
68a0
pflog.../boot/kernel/pflog.ko text=0xff8 text=0x938 data=0x488 0x8+0xa20+0x8+0x7
30
pf...if_vlan...if_tap...if_lagg.../boot/kernel/if_lagg.ko text=0x3b5e text=0x9f5
0 data=0xea0+0x8 0x8+0x1d88+0x8+0x16a5
loading required module 'if_infiniband'
/boot/kernel/if_infiniband.ko text=0x1102 text=0x12ac data=0x308+0x8 0x8+0x8a0+0
x8+0x590
if_gif...if_enc.../boot/kernel/if_enc.ko text=0x168a text=0x974 data=0x840 0x8+0
xd08+0x8+0xb96
if_bridge.../boot/kernel/if_bridge.ko text=0x38cf text=0x6bb0 data=0xee8+0x8 0x8
+0x1c50+0x8+0x1695
loading required module 'bridgestp'
/boot/kernel/bridgestp.ko text=0x12cb text=0x4ca4 data=0x2d0+0x28 0x8+0xb28+0x8+
0x6af
umass...acpi...
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
Using DTB provided by EFI at 0xef03c000.
����"�+#*�������� ш�1������)�$�� ��Y����r��x�)Չ����u�x�l��y9������-j���˫m���Kt���ݪ�����k����Ot���t��ű8�S���
                                                                                                           �h
                                                                                                             ������������qp���i*����+���e�Je�J����ճ��Nqu�vy}V[]N�����:Y���e������y�ѫ�����Ż����k�ѵ��I[ͭQ������Ũ�����J�����ʊ��딣��uQ<�J����8�i����魾���l������K��$"��
                    $����j�Qq���᪩��}���+��!���+����}<���I�����-���*����p�����u�����)��Q��/ռ<�>�&Չ�W�Q�2�J�����S�]]����<�����+�+*���*����<ɮz��ɩ���"���-�J�����������.�����������,��٩�
                                                                                                                                                                                    ��<������<�*��"d������$�����b&�x��zu8��*��"�e�(ɹ�q��ы�   �V��و(��4��ͣ����W2av:��Jm����,)�
                              (�8���W������y���W������*��*��rq:c��r��������թf�
                                                                              ���5�)�bj�i�?1Ẋՠ��ᭉj驩��Q�u�������(���fr���*ѤY�Q1QA�$��,(��������W�(����"�������z������2��?�ܯ�k
                                                                                                                                                                             �i�,����ű�����,(��+���A[YǴ�,(��+�����"����/��+�?�(�������˲Q)?�Qb�鑬��}��|�S����*��<��`�����񱁙�Yu<�`h(���x��k������!���i(���Nu<���ݪ�+��e�髴qq���
                                                                             Q|�Yq}�W��|��(�h���0|�|7�"���$���
                                                                                                              ����2��!��������.�頭�;����$����i���e#�������x�9y�Y>����   j���!��j�����!'���žከ���::I����ʱ��[��骥����!::�����*�Uu2|�w��z�uhub3: 1 port with 1 removable, self powered
uhub0: 1 port with 1 removable, self powered
uhub5: 2 ports with 2 removable, self powered
Mounting filesysuhub2: 2 ports with 2 removable, self powered
tems...
tunefs: soft updates remains unchanged as enabled
uhub4: 1 port with 1 removable, self powered
uhub1: 1 port with 1 removable, self powered
camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed
cam_lookup_pass: No such file or directory
cam_lookup_pass: either the pass driver isn't in your kernel
cam_lookup_pass: or mmcsd0 doesn't exist
mmcsd0 recovering is not needed
mmcsd0s1 resized
growfs: requested size 2.9GB is equal to the current filesystem size 2.9GB
** /dev/ufs/OPNsense
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 200966 free (126 frags, 25105 blocks, 0.0% fragmentation)
Setting hostuuid: 00000000-0000-0000-eb8b-ff6f21811d36.
Setting hostid: 0x5f653895.
>>> Invoking import script 'importer'
Press any key to start the configuration importer: .......
Bootstrapping config.xml...done.
Setting up /var/log memory disk...done.
Configuring crash dump device: /dev/null
.ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/ipsec /usr/local/lib/perl5/5.36/mach/CORE
32-bit compatibility ldconfig path:
done.
nda0 at nvme0 bus 0 scbus0 target 0 lun 1
nda0: <ADATA LEGEND 710 x x>
nda0: Serial Number x
nda0: nvme version 1.4
nda0: 244198MB (500118192 512 byte sectors)
>>> Invoking early script 'upgrade'
>>> Invoking early script 'configd'
Starting configd.
>>> Invoking early script 'templates'
Generating configuration: ERR
>>> Invoking early script 'backup'
>>> Invoking backup script 'captiveportal'
>>> Invoking backup script 'dhcpleases'
>>> Invoking backup script 'duid'
>>> Invoking backup script 'netflow'
>>> Invoking backup script 'rrd'
>>> Invoking early script 'carp'
CARP event system: OK
Launching the init system...done.
Initializing..........done.
Starting device manager...hym8563_rtc0: <HYM8694 RTC> at addr 0xa2 irq 90 on iicbus1
hym8563_rtc0: registered as a time-of-day clock, resolution 1.000000s
done.
Configuring login behaviour...done.

Default interfaces not found -- Running interface assignment option.

Press any key to start the manual interface assignment: 1
Do you want to configure LAGGs now? [y/N]: n
Do you want to configure VLANs now? [y/N]: n

Valid interfaces are:

re0              8e:04:f8:20:xx:xx Realtek PCIe 2.5GbE Family Controller
re1              56:f0:8d:5d:xx:xx Realtek PCIe 2.5GbE Family Controller
eqos0            22:3c:1b:bc:xx:xx DesignWare EQOS Gigabit ethernet

If you do not know the names of your interfaces, you may choose to use
auto-detection. In that case, disconnect all interfaces now before
hitting 'a' to initiate auto detection.

Enter the WAN interface name or 'a' for auto-detection: re1

Enter the LAN interface name or 'a' for auto-detection
NOTE: this enables full Firewalling/NAT mode.
(or nothing if finished): re0

Enter the Optional interface 1 name or 'a' for auto-detection
(or nothing if finished):

The interfaces will be assigned as follows:

WAN  -> re1
LAN  -> re0

Do you want to proceed? [y/N]: y

Writing configuration...done.
Configuring loopback interface...lo0: link state changed to UP
done.
Configuring kernel modules...done.
Setting up extended sysctls...done.
Setting timezone: Etc/UTC
Writing firmware settings: FreeBSD OPNsense
Writing trust files...done.
Scanning /usr/share/certs/untrusted for certificates...
Scanning /usr/share/certs/trusted for certificates...
Scanning /usr/local/share/certs for certificates...
certctl: Modified 206 trust store links.
Writing trust bundles...done.
Setting hostname: OPNsense.localdomain
Generating /etc/resolv.conf...done.
Generating /etc/hosts...done.
Configuring system logging...done.
Configuring firewall.......done.
Configuring hardware interfaces...done.
Configuring loopback interface...done.
Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring LAN interface...done.
Configuring WAN interface...done.
Generating /etc/resolv.conf...done.
Generating /etc/hosts...done.
Configuring firewall.......done.
Starting web GUI...done.
Setting up routes...done.
Starting DHCPv4 service...done.
Starting DHCPv6 service...done.
Starting router advertisement service...done.
Starting Unbound DNS...done.
Configuring firewall.......done.
Setting up gateway monitor...done.
Syncing OpenVPN settings...done.
Starting NTP service...done.
Starting Unbound DNS...done.
>>> Invoking start script 'newwanip'
>>> Invoking start script 'freebsd'
>>> Invoking start script 'syslog'
>>> Invoking start script 'carp'
>>> Invoking start script 'cron'
Starting Cron: OK
>>> Invoking start script 'openvpn'
>>> Invoking start script 'sysctl'
Service `sysctl' has been restarted.
>>> Invoking start script 'beep'
>>> Error in start script '95-beep'
Root file system: /dev/ufs/OPNsense
Wed Apr  2 02:49:09 UTC 2025

*** OPNsense.localdomain: OPNsense 25.1.3_33 (aarch64) ***

 LAN (re0)       -> v4: 192.168.1.1/24
 WAN (re1)       ->

 HTTPS: sha256 70 84 EC 39 83 90 BA 72 B7 6C 74 99 84 8E 8A 87
               2D 10 81 1D D7 27 33 3D 4C 98 C5 CF CD 17 AA FA

FreeBSD/arm64 (OPNsense.localdomain) (ttyu1)

login:



Hi Franco, here are my logs:

The message is:

Code Select Expand

mmcsd0 recovering is not needed
mmcsd0s1 resized
growfs: requested size 2.9GB is equal to the current filesystem size 2.9GB
** /dev/ufs/OPNsense

The microSD is just what opnsense creates and the file is there:

Code Select Expand

root@xxx:/mnt # glabel status
                                      Name  Status  Components
                  diskid/DISK-121220160204     N/A  da11
                         msdosfs/MSDOSBOOT     N/A  da11s1
                    ufsid/67ec8a03acb330e8     N/A  da11s2
                              ufs/OPNsense     N/A  da11s2

root@xxx:/mnt # mount /dev/ufs/OPNsensea temp/
root@xxx:/mnt # ls temp/
.cshrc COPYRIGHT conf entropy libexec proc sys
.probe.for.growfs bin dev etc media rescue tmp
.profile boot efiboot.img home mnt root usr
.snap boot.config efiboot.img.d lib net sbin var
root@xxx:/mnt # gpart show da11
=>      63  61069249  da11  MBR  (29G)
        63    131009        - free -  (64M)
    131072     65536     1  fat32lba  [active]  (32M)
    196608   6094848     2  freebsd  (2.9G)
   6291456  54777856        - free -  (26G)

root@trocatador:/mnt # umount temp/


full dmesg if needed in the next message.

none

Hi Franco,

I tried before using the default image. But that lead me to really small systems :(

You are right, 1GB images should not be enough, but I tried using 3GB and after fisrt boot it doesn't expand. Do I need anything else for it?

As far as I remember, the images have that file you pointed, but nevertheless there is a message on boot telling the fs doesn't need growing. I will try again at home and see if I can get that message and post here.

Thanks again for the help.

none

Hi,

I build images using the line:

Code Select Expand

make -C /usr/tools/ VERSION=25.1.4 DEVICE=R4S arm-8G

and then I get my 8GB image ready for dd'ing. It works but the disk is not resized.

I have 32GB and 64GB microSD in use and it keeps the 8GB partition size.

My research took me to the need for the .probe.for.growfs file on /, and the opnsense scripts do create it as is written in extras.conf. The extras.conf I use (https://github.com/matheusber/opnsense/blob/main/arm64-opnsense-build/opnsense-confs/extras.conf) just add the lines to load if_re from Realtek kmod version 1.98 from ports, that I build here.

The disk layout is what opnsense scripts creates. Have anyone delt with this?

Would really help create 1GB image files that would expand to the microSD size after. Will help too much here in storage space and other people to download.

Thanks,

none

Note this is not RC2, this is the development version of RC2 ;)

# opnsense-patch https://github.com/opnsense/core/commit/c9a3a841275

Thanks for the report!


Cheers,
Franco

Hi Franco, I built for arm64, does this command works for me?

I also noticed mine shows 25.1.r1_2X (not at home, can't remember if _20 or _22) and I checked out git for 25.1.r2 :)

thanks,

none

Can I update a box from 24.7.11 to 25.1.b (devel) and then in the future update to 25.1-RC1 and follow not-devel releases?

I have a Nanopi R5S running 25.1.b as testing, all fine. But my router for my home runs 24.7.11.

Thanks,

none