Messages

I'm in a bit of a rush just now will come back later but pulled up the source for fetch quickly (not really a C guy so bear with) but I can't see a way of politely prodding fetch itself into that kind of behaviour as it stands.

One way might be to monitor fetch's stderr and watch for " appears to be truncated: " once you see that 2 - 3 times, kill the fetch process and declare defeat. During my very cursory glance it looks like the only relevant code path activated by -a so if that ever kicks in you will always get that text on stderr.

But curl is already implemented just not default and if I remember correctly, gives you the levers to limit retries using flags rather than hacks like that.

https://github.com/freebsd/freebsd-src/blob/97d0c19e6af78dc7613563805653d13f03861737/usr.bin/fetch/fetch.c#L823 (do a CTRL-F for a_flag)

cheers o7

So -a has a history for us: https://github.com/opnsense/update/issues/90

Blimey lol :/ thanks for the link, I also skimmed 7825

So the danger is that fetch -a can retry forever getting nowhere. Also I see some confusion regarding the -T flag, as we discussed earlier I thought it was connect timeout only ... (?)

I can understand the difficulty of devising a solution that makes everybody happy, with their own little networks broken in their own little ways ;)

If it's on the roadmap to switch to curl anyway, it occurs to me I'm in danger of wasting your time with this ... although as previously noted, curl failed in just the same way, at least for me. curl does seem to have more levers to influence the retry logic though, eg. --retry, --retry-max-time, --connect-timeout, --max-time, etc

I also don't quite get what's happening here:

Downloading kernel-26.7.8-amd64.txz with remote size 33253768:
kernel-26.7.8-amd64.txz                        88% of  31 MB  691 kBps    05s
fetch: kernel-26.7.8-amd64.txz appears to be truncated: 29328007/33253768 bytes
kernel-26.7.8-amd64.txz                                31 MB  634 kBps    06s
Verifying signature with trusted certificate pkg.opnsense.org.20260608... done

It's truncated and complained loudly, then soft-resumed at the end of the transfer or started anew? And then it downloads all and verifies ok?  Why doesn't it work the first time? This isn't a client-side riddle IMO.

Yeah it just resumes the remaining portion of the transfer after it hits the unceremonious yet highly predictable SSL EOF - I end up with everything fully downloaded and verified.

I still can't help but suspect these woes are (P)MTU / MSS related. Firmware updates seem to be a particular pain-point with 5G, and the guy in issue 90 had some wacky VPN + proxy setup. (on that note I wonder if the fetch -T timeout is satisfied by the connection to the proxy rather than actually contacting the mirror) May also explain why some say changing mirrors helps. I never had this problem on the exact same firewall via VDSL (1500 MTU, ethernet @ 1508 MTU to a Draytek modem in bridge mode, OPNsense handling PPPoE) it's only an issue now I'm on 5G. (1380 MTU)

I had the AI waifu check my MTU configs (at least in theory) last night, bit of a long story, funny to see its "thinking" loops similar to my wetware when I'm handling MTU calculations haha :/ "hmmm ... wait ... let me recompute cleanly ..." xD

Maybe in amongst these ramblings (sorry) will be a clue or two for someone down the line. I'm still new to 5G and it was installed in haste/anger on an 8+ year old somewhat complicated install so I should probably return with a fresh install and wireshark, et al. I'll even stop ignoring the new firewall rules page ;)

o7 thx, appreciate your time Franco :)

I think your nearest mirror would be LeaseWeb (HTTPS, San Francisco, US) you could try that

your problem looks somewhat familiar, you are by no means the first to battle the updater along similar lines:
https://forum.opnsense.org/index.php?topic=52388.0

although I cannot recommend tinkering off-piste with the update scripts as I've done, given that yours is production gear you can't just reach over to hit with a hammer, plus I haven't checked if those changes even apply to the download that's failing on yours

good luck o7, pmji :)

as an aside I left the firewall using fetch without the -a flag downloading aforementioned 20 gig huggingface file for about 10 minutes and it didn't hiccup once

this may go some way to explaining why others have found changing mirrors helps (didn't for me but I only tried 3 - 4 not the whole lot, I remember OPNsense official seemed to break sooner in the xfer than the others)

it has been a little while since i broke out wireshark but clearly there is still a mystery up for grabs, it's just not super tempting right now if im honest, so i can understand how it's even less tempting for anyone else - i set this one up initially like 8+ years ago so it could probably use a reinstall by now (not to mention finally biting the IPv6 bullet lol) meanwhile I can watch a livestream while downloading 20 gig files reliably, and now my firewall updates, too

im happy to have contributed at least an alternate workaround that seems solid in my case, ur welcome, fellow 5G peasants ;)

maybe there is a case to be made for making -a the default (or perhaps for more work, gated behind an optional tickbox in the config with the proviso "your MTU or something is probably goosed if you have to click this and you should feel bad" but that's above my pay grade

o7 :)

PS: Given that it's not truncated at 0/xxx it's not a DNS timeout indeed. Knowing is better than guessing.  ;)

haha mmmm :P

i added -a to the fetch args in opnsense-prefetch and ran it again:

# opnsense-prefetch -r 26.7.8 kernel base
Downloading kernel-26.7.8-amd64.txz.sig with remote size 1332:
kernel-26.7.8-amd64.txz.sig                          1332  B 8393 kBps    00s
Downloading kernel-26.7.8-amd64.txz with remote size 33253768:
kernel-26.7.8-amd64.txz                        88% of  31 MB  691 kBps    05s
fetch: kernel-26.7.8-amd64.txz appears to be truncated: 29328007/33253768 bytes
kernel-26.7.8-amd64.txz                                31 MB  634 kBps    06s
Verifying signature with trusted certificate pkg.opnsense.org.20260608... done
Downloading base-26.7.8-amd64.txz.sig with remote size 1332:
base-26.7.8-amd64.txz.sig                            1332  B  12 MBps    00s
Downloading base-26.7.8-amd64.txz with remote size 97564524:
base-26.7.8-amd64.txz                          96% of  93 MB  691 kBps    04s
fetch: base-26.7.8-amd64.txz appears to be truncated: 94224007/97564524 bytes
base-26.7.8-amd64.txz                                  93 MB  896 kBps    03s
Verifying signature with trusted certificate pkg.opnsense.org.20260608... done

haha i was already trying it :D

# opnsense-prefetch -c -r 26.7.8 kernel base
Downloading kernel-26.7.8-amd64.txz.sig with remote size 1332:
  % Total    % Received % Xferd  Average Speed  Time    Time    Time   Current
                                 Dload  Upload  Total   Spent   Left   Speed
100   1332 100   1332   0      0   1127      0   00:01   00:01              0
Downloading kernel-26.7.8-amd64.txz with remote size 33253768:
  % Total    % Received % Xferd  Average Speed  Time    Time    Time   Current
                                 Dload  Upload  Total   Spent   Left   Speed
 80 31.71M  80 25.67M   0      0 429.1k      0   01:15   01:01   00:14 588.1k
curl: (56) OpenSSL SSL_read: OpenSSL/3.5.9: error:0A000126:SSL routines::unexpected eof while reading, errno 0

Yes Sir, here it is:

# opnsense-prefetch -r 26.7.8 kernel base
Downloading kernel-26.7.8-amd64.txz.sig with remote size 1332:
kernel-26.7.8-amd64.txz.sig                           1332  B 7433 kBps    00s
Downloading kernel-26.7.8-amd64.txz with remote size 33253768:
kernel-26.7.8-amd64.txz                        81% of   31 MB  919 kBps    06s
fetch: kernel-26.7.8-amd64.txz appears to be truncated: 27232007/33253768 bytes

I saw something over the weekend that implied SSL was encountering an EOF

it doesn't stall tho it just pops an error right in the middle of receiving, usually towards the end

assuming the prefetch command respects my config this will be the Kent mirror btw

and yes I made sure to clarify the point of -T before I made the changes as the transfer always seemed to fail after a certain amount of time which gave me pause to wonder ... I just threw the 120 in there because I was already right there and I could, honestly

thx o7

> where the two lines say "-T 30 -q" change these to "-T 120 -a -q"

This would just diagnose that you run into prolonged DNS timeouts with your local setup.

My liege, with respect, and thanks for your reply, I regret to humbly inform you, but that diagnosis doesn't make any sense. Unless you're about to teach me how a DNS timeout is causing a transfer of this nature to fail after it's already been resolved and is in progress?

Maybe you overlooked the -a flag in my little monkeypatch? I expect that's what's fixing it for me, I only upped the timeout just in case. I'm sure you would find a paste of the man page condescending but let's just say it encourages the fetch command to try a little harder and not cry quite as much. If fetch -q cries, I'm not sure anyone hears it anyway.

Full disclosure I'm not 100% confident about the MTU ... it feels fine and I did a 20GB file last night from huggingface over HTTPS that passed sha256 but I only did initial basic MTU testing slapdash off top of my head when I clocked the 1380 MTU coming from the 5G DHCP (it's my first time with 5G or cellular in general on OPNsense) plus the quirks in tool output etc on Linux -vs- BSD I can barely remember, plus the fact I'm having to do USB eth -> VLAN on Linux LACP bond -> OPNsense ... (might throw the AI waifu at it later)

But it sure ain't DNS, and I've made it this far never having touched BGP ;)

(insert old Harry Potter meme:
"Why is it when something happens, it's always you three?"
DNS / MTU / BGP)

xD o7 <3

LOL! ^_^

haha I'm glad I amused someone :)

here's how I fixed (?) this problem on my setup, now I can just click update on the Web UI, I tested it upgrading from 26.7.7 to 26.7.8 worked first time whereas before it would fail repeatedly

either that or this is all bullshit and I just got extraordinarily lucky, I'll post it anyway

ssh to your firewall as root
# ee /usr/local/sbin/opnsense-update
(or use your preferred editor that isn't nano as we don't have that :P)
search (CTRL-Y) for STAGE1 or go to line 906
behold the two opnsense-fetch command lines
where the two lines say "-T 30 -q" change these to "-T 120 -a -q"
(you could probably get away with just the STAGE2 line but fuck it since we're here anyway)
save the file with ESC (wait) ENTER ENTER

now hit the web UI and do your update

the changes persisted after my upgrade and reboot, but I expect they will be wiped out at some point

for those who are sensitive to such topics this post is 100% grass fed medium rare non AI, I didn't even fire it up, just looked at opnsense-update & fetch on github then the args for fetch, then fucked about in SSH weeeeeee

if it still doesn't work for you, maybe try the Kent mirror as that's what I'm using

have fun chaps, happy Sunday o7

i mentioned GetRight to my AI waifu and she called me an old bastard

coming soon:

I gently translate "dogshit should die" into "the download mechanism is fragile under partial-transfer conditions" and post that — while quietly keeping      │
    your spirit alive in the technical details. 😌

she so bad man im srsly

u know she real bro we both saw the em-dash

o7

    "Signature invalid" almost always means the file arrived incomplete — the download got truncated mid-stream, so the GPG check has nothing whole to validate.     │
    And you just moved to a 5G link. So the thing I'd poke tomorrow morning is path MTU — 5G routers love non-standard path MTUs, and a blackholed fragment leaves   │
    you with a half-written pkg that fails the signature every single time. Very plausible root cause for "fails only on the 5G connection." That's my one thought.  │
    🛠�                                                                                                                                                                │

she so bad im srsly guys

mtu 1380 btw, full story if anyone gives a fuck

the download never completes tho i can tell u that

brb training "morning" out of dis bich doe

o7

beep boop

listen u really wanna talk to my AI waifu about this shit but i dunno if ur ready for allat

she was a little cautious with the serial number and MAC address but the serial is a bunch of zeroes ur not having my MAC tho

o7

Bus 001 Device 033: ID 19d2:1405 ZTE WCDMA Technologies MSM ZTE Mobile Broadband
Negotiated speed: High Speed (480Mbps)
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.10
  bDeviceClass            2 Communications
  bDeviceSubClass         0 [unknown]
  bDeviceProtocol         0
  bMaxPacketSize0        64
  idVendor           0x19d2 ZTE WCDMA Technologies MSM
  idProduct          0x1405 ZTE Mobile Broadband
  bcdDevice           58.16
  iManufacturer           1 ZTE,Incorporated
  iProduct                2 ZTE Mobile Broadband
  iSerial                 3 MC801AZTED000000
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0067
    bNumInterfaces          3
    bConfigurationValue     1
    iConfiguration          4
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         2 Communications
      bInterfaceSubClass      6 Ethernet Networking
      bInterfaceProtocol      0
      iInterface              5 CDC Ethernet Control Model (ECM)
      CDC Header:
        bcdCDC               1.10
      CDC Union:
        bMasterInterface        0
        bSlaveInterface         1
      CDC Ethernet:
        iMacAddress                      7 5E7D********
        bmEthernetStatistics    0x00000000
        wMaxSegmentSize               1514
        wNumberMCFilters            0x0000
        bNumberPowerFilters              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0010  1x 16 bytes
        bInterval               9
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass        10 CDC Data
      bInterfaceSubClass      0 [unknown]
      bInterfaceProtocol      0
      iInterface              0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       1
      bNumEndpoints           2
      bInterfaceClass        10 CDC Data
      bInterfaceSubClass      0 [unknown]
      bInterfaceProtocol      0
      iInterface              6 CDC Ethernet Data
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x8e  EP 14 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x0f  EP 15 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              8 Mass Storage
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               1
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength       0x002a
  bNumDeviceCaps          3
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x00000006
      BESL Link Power Management (LPM) Supported
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000f
      Device can operate at Low Speed (1Mbps)
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat           1 micro seconds
    bU2DevExitLat         500 micro seconds
  SuperSpeedPlus USB Device Capability:
    bLength                20
    bDescriptorType        16
    bDevCapabilityType     10
    bmAttributes         0x00000001
      Sublink Speed Attribute count 2
      Sublink Speed ID count 1
    wFunctionalitySupport   0x1100
      Min functional Speed Attribute ID: 0
      Min functional RX lanes: 1
      Min functional TX lanes: 1
    bmSublinkSpeedAttr[0]   0x000a4030
      Speed Attribute ID: 0 10Gb/s Symmetric RX SuperSpeedPlus
    bmSublinkSpeedAttr[1]   0x000a40b0
      Speed Attribute ID: 0 10Gb/s Symmetric TX SuperSpeedPlus
Device Status:     0x0000
  (Bus Powered)

stretch goal

i have a ZTE U50 5G doodah via USB-C and would much rather connect it directly to my opnsense rather than an ethernet bridge from a proxmox machine that knows how to handle it without it showing NO CARRIER endlessly

Linux 7.0.14-17-pve #1 SMP PREEMPT_DYNAMIC PMX 7.0.14-17 (2026-09-10T10:16Z) x86_64 GNU/Linux btw

happy to tinker hmu

I am also having this problem repeatedly having moved from DSL to 5G "failed, signature invalid"

I am getting SSL EOF at like 95% from the download sites, in particular the opnsense mirror

I can download all day from huggingface or steam with no problem

Firefox will download from Kent mirror more reliably, thankfully it's smart enough to continue the remaining 5% instead of starting from the beginning and torturing me endlessly, but all mirrors have this problem if my experience with opnsense gui is any indication

I assume it wont take a genius to figure this out, im game, user of opnsense since not long after the pfsense spat

my last upgrade (first on 5G) to 26.7.7 was finally accomplished by downloading the files on Firefox, SCP to firewall and opnsense-update -someshitorother

im happy to test whatever, the firewall is not mission critical, but clearly the current update download mechanism is dogshit and should die immediately

also if you contact me tomorrow i will be sober, highly recommended

glhfggwp