OPNsense Forum
Archive => 15.7 Legacy Series => Topic started by: mircsicz on August 10, 2015, 01:32:07 pm
-
Hi all,
this is more or less my first post on the forum. I've just installed a first ALIX with the latest "i386-nano.img".
Have done some changes to the setup, all working as expected. Then I tried upgrading: The first update "pkgng" ran smooth but the second upgrade following with all the other packages only restart's the webconfigurator:
***GOT REQUEST TO UPGRADE: all***
***STARTING UPGRADE***
Updating OPNsense repository catalogue...
OPNsense repository is up-to-date.
All repositories are up-to-date.
Updating OPNsense repository catalogue...
OPNsense repository is up-to-date.
All repositories are up-to-date.
Checking for upgrades (99 candidates): .......... done
Processing candidates (99 candidates): ...... done
The following 55 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
libedit: 3.1.20150325_1
Installed packages to be UPGRADED:
sudo: 1.8.13 -> 1.8.14p3
squid: 3.5.3_1 -> 3.5.6
py27-pytz: 2014.10,1 -> 2015.4,1
py27-Babel: 1.3_2 -> 2.0
png: 1.6.17 -> 1.6.17_1
php56-zlib: 5.6.10 -> 5.6.11
php56-xml: 5.6.10 -> 5.6.11
php56-tokenizer: 5.6.10 -> 5.6.11
php56-sqlite3: 5.6.10 -> 5.6.11
php56-sockets: 5.6.10 -> 5.6.11
php56-simplexml: 5.6.10 -> 5.6.11
php56-session: 5.6.10 -> 5.6.11
php56-pdo_sqlite: 5.6.10 -> 5.6.11
php56-pdo: 5.6.10 -> 5.6.11
php56-openssl: 5.6.10 -> 5.6.11
php56-mysql: 5.6.10 -> 5.6.11
php56-mcrypt: 5.6.10 -> 5.6.11
php56-mbstring: 5.6.10 -> 5.6.11
php56-ldap: 5.6.10 -> 5.6.11
php56-json: 5.6.10 -> 5.6.11
php56-hash: 5.6.10 -> 5.6.11
php56-gettext: 5.6.10 -> 5.6.11
php56-filter: 5.6.10 -> 5.6.11
php56-dom: 5.6.10 -> 5.6.11
php56-curl: 5.6.10 -> 5.6.11
php56-ctype: 5.6.10 -> 5.6.11
php56-bz2: 5.6.10 -> 5.6.11
php56-bcmath: 5.6.10 -> 5.6.11
php56: 5.6.10 -> 5.6.11
phalcon: 2.0.3 -> 2.0.6
pcre: 8.37_1 -> 8.37_2
os-update: 15.7 -> 15.7.6
opnsense: 15.7 -> 15.7.7_3
openssh-portable: 6.8.p1_8,1 -> 6.9.p1_2,1
libressl: 2.2.0 -> 2.2.1
isc-dhcp42-server: 4.2.8 -> 4.2.8_1
freetype2: 2.5.5 -> 2.6_1
filterdns: 0.1 -> 0.2
dnsmasq: 2.73,1 -> 2.74,1
ca_root_nss: 3.19.1_1 -> 3.19.2
bind910: 9.10.2_5 -> 9.10.2P3_1
Installed packages to be REINSTALLED:
voucher-0.1_4 (needed shared library changed)
syslogd-10.1_1 (direct dependency changed: clog)
strongswan-5.3.2 (needed shared library changed)
relayd-5.5.20140810_1 (needed shared library changed)
python27-2.7.10 (needed shared library changed)
openvpn-2.3.7 (needed shared library changed)
openldap-client-2.4.41 (needed shared library changed)
ntp-4.2.8p3 (needed shared library changed)
miniupnpd-1.9_1,1 (needed shared library changed)
lighttpd-1.4.35_5 (needed shared library changed)
libxml2-2.9.2_3 (options changed)
libevent2-2.0.22_1 (needed shared library changed)
curl-7.43.0_2 (needed shared library changed)
The process will require 1 MiB more space.
46 MiB to be downloaded.
Restarting webConfigurator...done.
***DONE***
Or am I with the wrong expectations?
Edit: another [thread]https://forum.opnsense.org/index.php?topic=1228.0[/thread] gave me a hint in the right direction: Tried to upgrade from console and got the same output plus one extra line:
pkg: Not enough space in /var/cache/pkg, needed 46 MiB available 37 MiB
So I'll increase the RAM-Disk for var and see how it goes...
Just increased the size from default "no entry" which should be 60 to 75. Then rebooted but look what I got:
(http://snag.gy/SMP7D.jpg)
Seems it's best to wait till the Image-Size got increased to 4GB...
-
The smaller images are fragile like that: we've ran into all sorts of issues with low RAM + install and upgrades just because the base system grew by a large amount in an effort to get back to FreeBSDish freedom of how to use and deploy ones appliance.
I'll provide bigger 15.7.8 nano images shortly after 15.7.8 is released later this week. We have a kernel patch coming up with 15.7.8 so that's got to be in there.
-
THX, so I'll wait till next week and will test again with bigger images after the release 15.7.8...
Thank's Franco
-
Took a bit longer, but official images are up now!! :)
https://pkg.opnsense.org/releases/15.7.11/
-
I already have an Alix running OPNsense. Do I have to re-flash with this new image to get enhancements provided by it?
-
If you don't have any issues you don't have to reflash. We've only pulled the old images because it had a few annoying problems that people would run into when the firmware wasn't upgraded. :)
-
I have always problems while upgrading the kernel (not enough space). I need to reboot and then try the upgrade again.
Will re-flashing fix that?
-
Can you provide me with your output of
# df -h
please?
If upgrades fail at some point or another because they are all fetched at once, try this serialised sequence instead:
# opnsense-update -k
# opnsense-update -b
# /usr/local/etc/rc.reboot
-
Once rebooted, the whole kernel upgrade always works. So I think that there is not enough space in the ram partition because of packages that were upgraded before. Maybe more stuff needs to be cleaned up?
Here is the requested output. Note that the device is currently not in a state where kernel cannot be upgraded.
root@router:~ # df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/OPNsense0 936M 734M 127M 85% /
devfs 1.0K 1.0K 0B 100% /dev
tmpfs 13M 372K 13M 3% /tmp
tmpfs 39M 26M 13M 68% /var
devfs 1.0K 1.0K 0B 100% /var/dhcpd/dev
-
Is this a 128 MB ALIX? This barely works. We've just updated the hardware specs and think that 512 MB the barrier where things just run all of the time (except for the installer on USB, but that is another story), everything below may fail due to OOM. If push comes to shove, 256 should also work, but 128 is too little to completely avoid it.
Base and kernel update are 20MB and 40MB each. Packages vary from a few kilobytes but sum up up to 150MB, too. All this needs to be in RAM for SD/CF systems, so that's why it's just not "humanly" possible.
I can also see that you're running the older 2GB nano image, we've since upgraded to 4GB (2GB slice internally), which gives enough room to grow and fetch updates onto the card, too (e.g. disabling the /var and /tmp MFS completely).
I'm also going to split /var and /tmp toggle into individual options, because /tmp definitely makes sense, but /var is a challenge in itself. I recommend /tmp MFS, but not /var MFS. But anyway, sorry for the tangent there.
I'd suggest re-flashing if you have a 4GB SD/CF card, disabling /tmp and /var MFS can help you with upgrades when it keeps failing then at least.
-
It's a 256MB Alix. So from what I understand, I should consider an hardware upgrade ;).
In the meantime, I will re-flash with the new image.
-
Hmmm, any news on this? :)
-
Hello,
I have a recently reflashed 4 GB CF (reflashed with 15.7.11 nano 386 image). When trying to upgrade to 15.7.12 it fails with :
"pkg: Not enough space in /var/cache/pkg, needed 25 MiB available 5300 KiB"
df -h brings :
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/OPNsense0 1.8G 645M 1.0G 38% /
devfs 1.0K 1.0K 0B 100% /dev
tmpfs 11M 3.5M 7.7M 31% /tmp
tmpfs 25M 17M 7.7M 69% /var
devfs 1.0K 1.0K 0B 100% /var/dhcpd/dev
What should i do ??
Thanks for your help.
Regards
-
How much RAM do you have in total? Are you using IDS? Your disk seems rather full as well.
I'd recommend switching off /var /tmp MFS option in System: Settings: Misc and rebooting, try to upgrade again. After successful update you can switch the option back on and reboot.
-
RAM : based on dmesg.today : "real memory = 268435456 (256 MB)" / "avail memory = 226619392 (216 MB)"
IDS : I tried but it always fail with "kernel: pid 62902 (suricata), uid 0, was killed: out of swap space"
As far as disk space, root filesystem has 38% free / 645 Mb available which is quite large for what I'm exepcting to do (no squid cache or things like that, just firewall / dchp / dns / ntp and possibly IDS.
=> To upgrade sucessfully I had to
1) disable the "/var /mem in memory" parameter
2) reboot
3) upgrade
4) re-enable the "/var /mem in memory" parameter
5) reboot
Nota : at step 2 I had to play with my .ssh/known_host since the RSA fingerprint of the OPNsens had changed...
Suggestion : why not putting the /var/cache/pkg on disk rather than memory to ensure upgrade will work ? (this should not be written frequently ?).
-
256 MB and IDS are really not meant to go together. The fact that it works is miracle. It also explains your low tmpfs capacity when suricata is running in the background. My advice: don't use Intrusion Detection or Proxy Server on the hardware, or replace it, or switch off those services when you upgrade (might be a tad quicker). We cannot afford to strip down OPNsense like other distributions, because it is one of ours strengths. If anything, others should step in and do a lightweight version of OPNsense if there is much demand.
Moving the pkg fetch location only shifts the problem, it doesn't solve it. Especially for nano images, space is always scarce and only RAM in newer devices offers enough space for future upgrades. The main issue is that FreeBSD and in turn OPNsense have outgrown certain older hardware.
The keys changed since OpenSSH 7.1 doesn't have support for SSHv1 anymore, so the keys were rotated by our scripts. This won't happen again any time soon though. :)
-
Thanks a lot for your feedback.
Based on your advice I just switched off suricata (who any way never achieved to start completely and was failing while loading). After rebooting, df -h brings :
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/OPNsense0 1.8G 783M 919M 46% /
devfs 1.0K 1.0K 0B 100% /dev
tmpfs 97M 48K 97M 0% /tmp
tmpfs 110M 13M 97M 12% /var
devfs 1.0K 1.0K 0B 100% /var/dhcpd/dev
From you point of view what kind of minimum hardware would be required for a "nano" type of system able to run IDS ?
Thanks for your advice.
Best regards.
-
That's a little better, I see, but might still give you trouble on larger updates (normally the updates are applied except the base/kernel).
Our recommendations for the nano image are here: https://opnsense.org/users/get-started/#hardware-requirements
-
Hello,
I'm back with my poor 256 Mb Alix...
To have automatic updates working, I'v plugged a 2 Gb USB key on the alix and mounted that USB key on /var/cache/pkg.
Now the point is when a kernel update is required opnsense-update uses /tmp to fetch its .txz and .obsolete files.
+ Why aren't they going in /var/cache/pkg like other .txz files ?
+ Is there a way to pass a parameter to opnsense-update to use another directory as working directory ?
Thanks for your kind advice.
Best regards.
-
You can manually edit the working path of /usr/local/sbin/opnsense-update, it's a shell script. Note that the update of the utility currently overwrites these changes.
The reason was that opnsense-update is written by us and we did not want to write to a supposed disk for CF and SD cards while FreeBSD has more of a server focus and persistency of downloaded files on proper hard disks. There's some more things on my TODO list like a config file for opnsense-update, I will try to weave that in for 16.1. :)
-
OK Thanks a lot for the reply.
Basically changing the lines
rm -rf /tmp/opnsense-update.* (somewhere at the beginning of the file)
and
WORKDIR=/tmp/opnsense-update.${$} (somewhere in the middle of the file after the KERNEL UPDATE warning)
by
rm -rf /var/cache/pkg/opnsense-update.*
WORKDIR=/var/cache/pkg/opnsense-update.${$}
would be great help for me while staying consistent with what the updater is doing for other packages (fetching them in /var/cache/pkg).
Regards.
-
/var/cache/pkg is not a directory we should reuse, the utilities serve different purposes and I don't know the side effects, you should use:
rm -rf /var/cache/opnsense-update.*
WORKDIR=/var/cache/opnsense-update.${$}
This behaviour has unintended side effects on non-Nano images as the updates will become non-volatile and will also make issues with the /var MFS and reversal of the setting, as we had very very early in 15.7 with pkg.
opnsense-update is pretty cool, but was designed because freebsd-update was too complex a tool for the system upgrades. But truth is opnsense-upgrade will become useless with FreeBSD 11 when we an have pkg take care of kernel and base system too.
So all in all I'm not sure how much work should be put in here in the long run, potentially adding regressions and seeing old problems reappear. Not very good style. :(
I'll have to rethink this some more.
-
OK I understand.
Then another idea could be to have a different directory name while staying in /tmp :
If you would use /tmp/opnsense-update/${$} instead of /tmp/opnsense-update.${$}, then one could create a mount to a permanent slice of a USB stick for /tmp/opnsense-update
-
Hmm, we can do that actually and just push the files in there flat. Just have to make sure two processes don't race on the same files there (more hypothetical madness, but better safe than sorry when it comes to upgrades).
See: https://github.com/opnsense/update/issues/2
Thanks! :)
-
I've added your suggestion to opnsense-update. It will be updated as soon as the next base/kernel update is needed. I don't think there's any need before this, right?
-
Now it works, thanks a lot (kernel updates are going to /tmp/opnsense-update/...)
So I re-formated by USB key with 2 partitions (one for /var/cache/pkg et the other one for /tmp/opnsense-update), and my alix is now updating the kernel very slooooooowly.
Finally it took 21 minutes for the kernel upgrade stage followed by a 6 minutes reboot !
-
@gpac: which filesystem do you use on your USB pen drive? FAT32 or UFS?
Do you mount the drives manually before each upgrade or did you configure them in the fstab? Is the /etc/fstab persistent between updates?
I've just tried an update from 15.7.18 to 15.7.20 (the fifth time in a row) and failed again. I always end up with a corrupted file system. That time I had two external UFS partitions mounted to /var/cache and /tmp/opnsense-update and previously changed the opnsense-update script to adopt a predictable WORKDIR.
Is there any other way in upgrading to 15.7.20? I guess I'll build my own image and flash it directly to the CF...
-
Mf, how much RAM does the board have? This sounds like an Out-Of-Memory kill mid-upgrade.
/etc/fstab entries are never touched. I guess FAT32 works, if you need the stick for other things. We'll improve the procedure further, but the CF card upgrades are tricky because the base system has grown considerably for OPNsense to allow easy access to build tools and a multitude of programming languages (PHP, Python, Perl) along with full manual pages.
-
Sorry for late feedback.
With recent upgrades I had to create 2 partitions on my USB key
/dev/da0p1 on /var/cache/pkg (ufs, local)
/dev/da0p2 on /tmp/opnsense-update (ufs, local)
Nota an unclean shutdown of my Alix recently corrupted the file system on /var/da0p1 => adding an fsck in the small batch who mounts those seems usefull.
I mounting them manuall through a simple batch file I'm launching via ssh. For sure it would be better to be able to add something in a rc file to have it launched at startup. But I don't know exactly where / how to do that (in regular linux adding a Sxx and a Kxx file in the init.d would to the trick but on freebsd I don't how to do that), nor how to do that in a location / file not touched by future upgrades.
Best regards.
-
Gpac, the best way to launch a script is to use our rc.syshook facility.
Rename the (executable) shell script to /usr/local/etc/rc.syshook.d/mymount.early (mymount is a name you can freely choose)
The boot will then pick it up automatically. :)
-
Hello,
Please note that it doesn't seem t o work (ie rc.syshook not launched at startup).
Per you instructions I've created a symlink called mymount.early in /usr/local/etc/rc.syshook.d
Launching /usr/local/etc/rc.syshook.d/mymount.early from command line works, but when booting the alix the mounts are not done and I have no error messages in /var/log/system.log.
Credentials are the following :
-rwxr-xr-x 1 root wheel 1912 Jan 18 10:59 rc.syshook
drwxr-xr-x 2 root wheel 512 Jan 13 20:57 rc.syshook.d
and
/usr/local/etc/rc.syshook.d:
total 4
drwxr-xr-x 2 root wheel 512 Jan 13 20:57 .
drwxr-xr-x 27 root wheel 3584 Jan 23 09:38 ..
lrwxr-xr-x 1 root wheel 29 Jan 13 20:57 mymount.early -> /root/var_cache_pkg_by_FXL.sh
and content of the batch file is :
mkdir -p /var/cache/pkg
fsck -y -t ufs /dev/da0p1
mount /dev/da0p1 /var/cache/pkg
mkdir -p /tmp/opnsense-update
fsck -y -t ufs /dev/da0p2
mount /dev/da0p2 /tmp/opnsense-update
If have an idea of the problem that would be very helpfull.
Best regards.
-
lrwxr-xr-x 1 root wheel 29 Jan 13 20:57 mymount.early -> /root/var_cache_pkg_by_FXL.sh
It only accepts a file, move /root/var_cache_pkg_by_FXL.sh to /usr/local/etc/rc.syshook.d/mymount.early
(A link could be a directory as well.)