Need help with update (log posted)

[cake]

I keep getting this in the log when updating. I updating over wiregaurd remotely and don't want to brick it.
Anyone know what I should do? I am trying to fix another problem I am having but first thought I would update first.

Code Select Expand

***GOT REQUEST TO UPDATE***
Currently running OPNsense 23.1.5_4 at Sun Jun 25 22:36:17 +07 2023
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
All repositories are up to date.
Checking for upgrades (56 candidates): .......... done
Processing candidates (56 candidates): .......... done
Checking integrity... done (0 conflicting)
The following 59 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
easy-rsa: 3.1.5
pkcs11-helper: 1.29.0
py39-tzdata: 2023.3_1

Installed packages to be UPGRADED:
ca_root_nss: 3.89 -> 3.89.1
curl: 7.88.1 -> 8.1.2
dhcp6c: 20200512_1 -> 20230530
glib: 2.76.1,2 -> 2.76.3,2
ifinfo: 13.0 -> 13.0_1
krb5: 1.20.1 -> 1.21
libnghttp2: 1.52.0 -> 1.53.0
libpsl: 0.21.2_2 -> 0.21.2_3
libxml2: 2.10.3_1 -> 2.10.4
lighttpd: 1.4.69 -> 1.4.71
mpd5: 5.9_13 -> 5.9_16
nettle: 3.8.1 -> 3.9.1
nss: 3.89 -> 3.90
ntp: 4.2.8p15_5 -> 4.2.8p17
openssh-portable: 9.2.p1,1 -> 9.3.p1,1
openvpn: 2.5.8 -> 2.6.5
opnsense: 23.1.5_4 -> 23.1.10_1
opnsense-update: 23.1.5 -> 23.1.8_2
os-dnscrypt-proxy: 1.12_1 -> 1.13_1
pftop: 0.8_2 -> 0.8_4
php81: 8.1.17 -> 8.1.20
php81-ctype: 8.1.17 -> 8.1.20
php81-curl: 8.1.17 -> 8.1.20
php81-dom: 8.1.17 -> 8.1.20
php81-filter: 8.1.17 -> 8.1.20
php81-gettext: 8.1.17 -> 8.1.20
php81-ldap: 8.1.17 -> 8.1.20
php81-mbstring: 8.1.17 -> 8.1.20
php81-pdo: 8.1.17 -> 8.1.20
php81-phalcon: 5.2.1 -> 5.2.2
php81-session: 8.1.17 -> 8.1.20
php81-simplexml: 8.1.17 -> 8.1.20
php81-sockets: 8.1.17 -> 8.1.20
php81-sqlite3: 8.1.17 -> 8.1.20
php81-xml: 8.1.17 -> 8.1.20
php81-zlib: 8.1.17 -> 8.1.20
py39-bottleneck: 1.3.6 -> 1.3.7_1
py39-certifi: 2022.12.7 -> 2023.5.7
py39-charset-normalizer: 3.0.1 -> 3.1.0
py39-cython: 0.29.33 -> 0.29.35
py39-dnspython: 2.2.1_1,1 -> 2.3.0,1
py39-idna: 3.4 -> 3.4_1
py39-markupsafe: 2.1.2 -> 2.1.3
py39-numexpr: 2.8.4 -> 2.8.4_1
py39-numpy: 1.24.1,1 -> 1.24.1_4,1
py39-pandas: 1.5.3,1 -> 2.0.2,1
py39-requests: 2.28.2 -> 2.31.0
py39-sqlite3: 3.9.16_7 -> 3.9.17_7
py39-ujson: 5.7.0 -> 5.8.0
py39-urllib3: 1.26.14,1 -> 1.26.16,1
python39: 3.9.16_2 -> 3.9.17
sqlite3: 3.41.0_1,1 -> 3.42.0,1
squid: 5.8 -> 5.9
strongswan: 5.9.10_1 -> 5.9.10_2
suricata: 6.0.9_1 -> 6.0.13
syslog-ng: 3.38.1 -> 4.2.0

Number of packages to be installed: 3
Number of packages to be upgraded: 56

The process will require 6 MiB more space.
[1/59] Upgrading python39 from 3.9.16_2 to 3.9.17...
[1/59] Extracting python39-3.9.17: .......... done
python39-3.9.16_2: missing file /usr/local/lib/python3.9/ensurepip/_bundled/pip-22.0.4-py3-none-any.whl
python39-3.9.16_2: missing file /usr/local/lib/python3.9/lib2to3/Grammar3.9.16.final.0.pickle
python39-3.9.16_2: missing file /usr/local/lib/python3.9/lib2to3/PatternGrammar3.9.16.final.0.pickle
python39-3.9.16_2: missing file /usr/local/share/licenses/python39-3.9.16_2/LICENSE
python39-3.9.16_2: missing file /usr/local/share/licenses/python39-3.9.16_2/PSFL
python39-3.9.16_2: missing file /usr/local/share/licenses/python39-3.9.16_2/catalog.mk
pkg-static: Fail to rename /usr/local/lib/python3.9/test/test_tools/__pycache__/.pkgtemp.test_lll.cpython-39.opt-1.pyc.dRm2Lr8DrAGh -> /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc:Invalid argument
Starting web GUI...done.
Generating RRD graphs...done.
***DONE***

[newsense]

Try this command and then re-run the upgrade:

Code Select Expand

mv /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc-old

It is a bug in PKG that's reported and now Franco has two different reports to add to the bug, and if I understood correctly a fix it will land in OPNsense soon regardless of how fast it is addressed upstream. Unsure if this will be a hotfix or 23.1.11/23.7.

We'll find out either here from Franco or in the release notes :)

[cake]

Try this command and then re-run the upgrade:

Code Select Expand

mv /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc-old

It is a bug in PKG that's reported and now Franco has two different reports to add to the bug, and if I understood correctly a fix it will land in OPNsense soon regardless of how fast it is addressed upstream. Unsure if this will be a hotfix or 23.1.11/23.7.

We'll find out either here from Franco or in the release notes :)

I get

Code Select Expand

mv: rename test_lll.cpython-39.opt-1.pyc to test_lll.cpython-39.opt-1.pyc-old: Invalid argument when in that directory. Thanks for the tip, I'm not sure why I get invalid argument. I can see the filelisted when I type ls

I guess I will wait a while, opnsense still works for me.

[cookiemonster]

why is there a "rename" there? No such thing in *nix. The command is simply "mv". Maybe you inserted "rename" by mistake?

[cake]

I don't know enough kung foo to know why either, but here is a screen shot. 

[cookiemonster]

Ok fair enough. I never noticed the "rename" message in mv.
You had me intrigued so I tested it and works fine. I renamed and renamed back.

Code Select Expand


[penguin@OPNsense /usr/local/lib/python3.9/test/test_tools/__pycache__]$ ls -lha test_lll.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   1.4K Jan 17 00:30 test_lll.cpython-39.opt-1.pyc
[penguin@OPNsense /usr/local/lib/python3.9/test/test_tools/__pycache__]$ cd ~
[penguin@OPNsense ~]$ sudo mv /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc-old
Password:

[penguin@OPNsense ~]$ ls -lha /usr/local/lib/python3.9/test/test_tools/__pycache__
total 215
drwxr-xr-x  2 root  wheel    38B Jun 26 21:05 .
drwxr-xr-x  3 root  wheel    15B Mar 31 15:45 ..
-rw-r--r--  1 root  wheel   1.5K Jan 17 00:30 __init__.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   1.4K Jan 17 00:30 __init__.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   1.5K Jan 17 00:30 __init__.cpython-39.pyc
-rw-r--r--  1 root  wheel   229B Jan 17 00:30 __main__.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   229B Jan 17 00:30 __main__.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   229B Jan 17 00:30 __main__.cpython-39.pyc
-rw-r--r--  1 root  wheel   2.8K Jan 17 00:30 test_fixcid.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   2.8K Jan 17 00:30 test_fixcid.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   2.8K Jan 17 00:30 test_fixcid.cpython-39.pyc
-rw-r--r--  1 root  wheel   1.4K Jan 17 00:30 test_gprof2html.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   1.3K Jan 17 00:30 test_gprof2html.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   1.4K Jan 17 00:30 test_gprof2html.cpython-39.pyc
-rw-r--r--  1 root  wheel   9.6K Jan 17 00:30 test_i18n.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   8.6K Jan 17 00:30 test_i18n.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   9.6K Jan 17 00:30 test_i18n.cpython-39.pyc
-rw-r--r--  1 root  wheel   1.4K Jan 17 00:30 test_lll.cpython-39.opt-1.pyc-old
-rw-r--r--  1 root  wheel   1.4K Jan 17 00:30 test_lll.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   1.4K Jan 17 00:30 test_lll.cpython-39.pyc
-rw-r--r--  1 root  wheel   3.4K Jan 17 00:30 test_md5sum.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   3.3K Jan 17 00:30 test_md5sum.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   3.4K Jan 17 00:30 test_md5sum.cpython-39.pyc
-rw-r--r--  1 root  wheel   3.7K Jan 17 00:30 test_pathfix.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   3.7K Jan 17 00:30 test_pathfix.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   3.7K Jan 17 00:30 test_pathfix.cpython-39.pyc
-rw-r--r--  1 root  wheel   1.3K Jan 17 00:30 test_pdeps.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   1.2K Jan 17 00:30 test_pdeps.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   1.3K Jan 17 00:30 test_pdeps.cpython-39.pyc
-rw-r--r--  1 root  wheel   8.4K Jan 17 00:30 test_pindent.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   8.3K Jan 17 00:30 test_pindent.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   8.4K Jan 17 00:30 test_pindent.cpython-39.pyc
-rw-r--r--  1 root  wheel   1.5K Jan 17 00:30 test_reindent.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   1.3K Jan 17 00:30 test_reindent.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   1.5K Jan 17 00:30 test_reindent.cpython-39.pyc
-rw-r--r--  1 root  wheel   1.8K Jan 17 00:30 test_sundry.cpython-39.opt-1.pyc
-rw-r--r--  1 root  wheel   1.6K Jan 17 00:30 test_sundry.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   1.8K Jan 17 00:30 test_sundry.cpython-39.pyc

[penguin@OPNsense ~]$ sudo mv /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc-old /usr/local/lib/python3.9/test/test_tools/__pycache__/test_lll.cpython-39.opt-1.pyc
I really can't spot the problem and I suspect some syntax error like a space somewhere but can't spot it.

Please use autocomplete (tab completion) instead of typing the path or copying from forum, might introduce extraneous characters. Only thing I can think of.

[cake]

I tried, no luck, file shows when I "ls" but not "ls -lah". I tested on another file in directory and it listed the file. Some of the files have something like a hash at the end of the file name.

Code Select Expand

root@J1900:/usr/local/lib/python3.9/test/test_tools/__pycache__ # ls -lah test_lll.cpython-39.opt-1.pyc
ls: test_lll.cpython-39.opt-1.pyc: Invalid argument
root@J1900:/usr/local/lib/python3.9/test/test_tools/__pycache__ # ls -lah test_sundry.cpython-39.opt-2.pyc
-rw-r--r--  1 root  wheel   1.6K Mar 28 05:14 test_sundry.cpython-39.opt-2.pyc
root@J1900:/usr/local/lib/python3.9/test/test_tools/__pycache__ # ls -lah
ls: test_lll.cpython-39.opt-1.pyc: Invalid argument
ls: test_lll.cpython-39.opt-2.pyc: Invalid argument
ls: test_lll.cpython-39.pyc: Invalid argument
ls: test_md5sum.cpython-39.opt-1.pyc: Invalid argument
ls: test_md5sum.cpython-39.opt-2.pyc: Invalid argument
ls: test_md5sum.cpython-39.pyc: Invalid argument
ls: test_pathfix.cpython-39.opt-1.pyc: Invalid argument
ls: test_pathfix.cpython-39.opt-2.pyc: Invalid argument
ls: test_pathfix.cpython-39.pyc: Invalid argument
ls: test_pdeps.cpython-39.opt-1.pyc: Invalid argument
ls: test_pdeps.cpython-39.opt-2.pyc: Invalid argument
ls: test_pdeps.cpython-39.pyc: Invalid argument
ls: test_pindent.cpython-39.opt-1.pyc: Invalid argument
ls: test_pindent.cpython-39.opt-2.pyc: Invalid argument
ls: test_pindent.cpython-39.pyc: Invalid argument
ls: test_reindent.cpython-39.opt-1.pyc: Invalid argument
total 660
drwxr-xr-x  2 root  wheel   9.0K Jun 25 22:51 .
drwxr-xr-x  3 root  wheel   3.0K Jun 25 22:51 ..
-rw-r--r--  1 root  wheel   1.4K Jun 21 05:17 .pkgtemp.test_lll.cpython-39.opt-1.pyc.F21qEh0gnaWr
-rw-r--r--  1 root  wheel   1.4K Jun 21 05:17 .pkgtemp.test_lll.cpython-39.opt-1.pyc.SLAtPp6mSC72
-rw-r--r--  1 root  wheel   1.4K Jun 21 05:17 .pkgtemp.test_lll.cpython-39.opt-1.pyc.dRm2Lr8DrAGh
.................................
................................



edit: I have a ton of these msgs in dmesg.yesterday "/mnt: inode 758: check-hash failed
/mnt: inode 757: check-hash failed
/mnt: inode 754: check-hash failed
/mnt: inode 756: check-hash failed
/mnt: inode 752: check-hash failed
/mnt: inode 753: check-hash failed
/mnt: inode 755: check-hash failed
/mnt: inode 764: check-hash failed
/mnt: inode 766: check-hash failed
/mnt: inode 767: check-hash failed
/mnt: inode 760: check-hash failed
/mnt: inode 765: check-hash failed
..........." after a reboot. I wonder if I have a corrupt filesystem. I have a lot of lightning at my house and freq power outages that exceed the UPS. Just a thought.

edit#2
I ran fsck and it says a bunch of stuff like "INODE CHECK-HASH FAILED I=753  OWNER=4294967295 MODE=177777
SIZE=18446744073709551615 MTIME=Jan  1 06:59 1970
FIX? no
.........."

That means trouble right? lol

[cookiemonster]

Without knowing any more, I'd say yes. You have an unhealthy filesystem. Needs fixing asap before anything else.

[franco]

I'd try a fsck in single user mode (looks like this is an UFS install after all) and then for good measure reinstall the python package cleanly before proceeding with the upgrade:

# opnsense-revert python39


Cheers,
Franco

[cake]

Thanks for the help, Im going to let it ride since wont be home for several more months.


fingers crossed :-D