OPNsense install from a USB key

[jhaines]

Hi,

I can't appear to see how to get OPNsense on a USB key to install, I have tried Rufus but it complains that the ISO can't be used as not bootable.

Is there a USB version?

[STk]

Hi,

had the same problem with windows - I gave up after servera hours of trying.
Use an unix based system to create the stick, e.g. finde someone who has mac or linux.
And use the img-version, not the iso.

Regards,
Sebastian

[phoenix]

Hi,

I can't appear to see how to get OPNsense on a USB key to install, I have tried Rufus but it complains that the ISO can't be used as not bootable.

Is there a USB version?

To answer your second question first, there isn't a specific USB version.

I've never had any problems burning the .iso version to a USB stick using rufus under Windows 7 & 10 in fact I've recently installed a clean OPNsense after going through the 'rufus' procedure. Which image did you download? There is a refresh of the download files with the latest version on it if you haven't used that perhaps you could try it and see how you get on. Do you see any errors when you try and burn the image with rufus?

[csmall]

Use physdiskwrite

[franco]

The VGA and Serial images are USB images, for VGA or Serial hardware, respectively.

https://pkg.opnsense.org/releases/17.1.4/README


Cheers,
Franco

[Deepcuts]

For some time now I have been trying to install OPNSense from USB.
Today I tried again, hoping the issue is fixed. It seems not.

I can write any other distro to any of the 3 USB sticks I tried with, using any method: physdiskwrite, win32diskimage,Rufus, Yumi  from Windows AND dd from my linux machine and they will boot without any problem.
Yet, OPNSense refuses to boot from any USB written using any method on any machine. (My PC, laptop or router box)

Now, either someone is trolling and wants new users off of OPNSense or ...nobody cares, despite numerous reports about USB sticks not booting with OPNSense img.

Just letting you know that you are loosing new users with this.

[franco]

How shall we measure the users lost against the users gained from the way the images are built? It's an impossible choice, especially when we consider the way the images are built help e.g. UEFI users to install without fuzz, now with 17.1.4 more than ever.

Then there is the obscure issue of how to debug a failed boot or write of boot media? Will anyone help get to the bottom of this? How many fixes have we have incorporated since how many releases? We've released 17.1.4 just so UEFI users will have a better time.

Yes, it's gotten more difficult to write install media, yet it eludes me why, or what we can do in the scope of Windows or Linux or BSD or hardware to get "it" "fixed". I love all the help that has been given by users so far.


Cheers,
Franco

[csmall]

I have yet to have physdiskwrite fail.

If you have tried Rufus, unetbootin etc... or even win32diskwrite without success, try physdiskwrite.

[smoore]

I have tried downloading several of the r.17 releases (including today r17.1.2 64-bit amd) both *.ISO and *.IMG files and have not succeeded in creating installation media. I have used different computers and different approaches (i.e. Rufus, physdiskwrite, dd, etc.). I checked the downloaded files against the MD5 checksums posted in the source directories (i.e. using WinMD5Free in Windows).

The CDROM *.ISO when unzipped is 879,030KB and there is not an MD5 checksum for the unzipped ISO image. The ISO image is too large to be writable on a standard CDROM. I have performed the download and unzipping on both Windows 7 and Linux computers. Specifically, I performed the unzipping on two separate computers using 7-Zip (Windows 7) and bzip2 (Linux). Both computers resulted in the same 858MB ISO file, which could not be written to an ordinary CDROM.

The USB *.IMG is 928,658KB when unzipped and there is not an MD5 checksum for the unzipped image. I downloaded and unzipped the files using two different computers using the same process as before (7-Zip in Windows and bzip2 in Linux). Both operating systems resulted in the same >900MB *.img file.

I attempted several different utilities to write both the *.iso and *.img file to several different brands of USB keys (Mushkin, Lexar, SanDisk).  I did not attempt writing a CDROM because the *.iso file was too big to fit on standard CD media. The *.iso file is not formatted to create a bootable USB, so all *.iso attempts resulted in unsuccessful installation media.

When using Windows utilities (Rufus, physdiskwrite, yumi) to create USB installation media with the *.img file, the resulting USB partition table is corrupt and immediately blue-screens any Windows 7 computer it is inserted. I tried plugging a written USB drive into another Windows laptop, and it blue-screened immediately. I could recover these badly-written USB drives by cleaning up the mess with a few minutes of DBAN followed by a fresh partition table with partd.

I attempted dd if=OPNsense.img of=/dev/sdd (Linux). When viewed in GParted after writing, the drive (/dev/sdd) had GPT (partition table) errors. GParted attempted to fix the GPT errors, but the boot record and partition contents remained corrupt (in some cases, unreadable).

My conclusion is the img/ISO files are bad. They unzip way too large (>800MB) and the partition tables are corrupt (seen in parted/GParted). It doesn't matter what write utilities are used if the img/ISO file are bad from the start.

There are several posts on this forum of people not being able to write the *.img to USB. Most of these forum posts are answered with the typical and unhelpful "you're doing it wrong" responses. For those users experiencing media installation creation problems, you should wait until either the developers or knowledgeable users are able to re-create the problem for themselves and present a solution.

[franco]

There are several posts on this forum of people not being able to write the *.img to USB. Most of these forum posts are answered with the typical and unhelpful "you're doing it wrong" responses. For those users experiencing media installation creation problems, you should wait until either the developers or knowledgeable users are able to re-create the problem for themselves and present a solution.

Fair enough. Assuming this is true, how do we fix it? And what shall be our base line for "fixed"?


Cheers,
Franco

[smoore]

I'd like to start down the "root cause" path, starting with the downloading and unzipping the *.img file.

For someone who has successfully created USB installation media from the OPNsense-17.1.4-OpenSSL-vga-amd64.img.bz2 file (MD5 checksum 6e1563a155a8715aa73e62be4cf0d542) please post:

- confirm the MD5 of your downloaded *.bz2 file before continuing
- unzip the *.bz2 file
- report the file size and MD5 of the unzipped *.img file
- the partition table of the USB installation media (i.e. a screenshot of GParted)
- Any GPT error reports from GParted

Same goes for the CDROM image file OPNsense-17.1.4-OpenSSL-cdrom-amd64.iso.bz2. I'm confused to why the unzipped file is >720MB and won't fit onto standard CDROM media.  I tried mounting the unzipped *.iso file and found errors, so something is going wrong.

For reference, here are my results:

OPNsense-17.1.4-OpenSSL-vga-amd64.img     928,658KB   MD5: b4d7579895eab34ff6193fc2422f58be
OPNsense-17.1.4-OpenSSL-cdrom-amd64.iso   879,030KB   MD5: 66a2ecc498689ea16510ee47243448db

[franco]

Using MacOS... fresh downloads, confirmed, extracted, summed up:

$ md5 *17.1.4*
MD5 (OPNsense-17.1.4-OpenSSL-cdrom-amd64.iso) = 66a2ecc498689ea16510ee47243448db
MD5 (OPNsense-17.1.4-OpenSSL-vga-amd64.img) = b4d7579895eab34ff6193fc2422f58be

The ISO works in VirtualBox, passes package integrity test (pkg check -sa). Image also boots in UEFI mode in VMware Fusion. Installs ok, integrity tests pass.

Yes, the CDROM is bigger as it should, it's missing a rename to DVD. It's a convenience target for VM deployments for the most part. Shrinking the image is difficult. FreeBSD grows, packages grow, ... There's much room for a minimal OPNsense "fork" if someone is willing to do the work necessary.

Writing the image to USB:

$ sudo dd if=OPNsense-17.1.4-OpenSSL-vga-amd64.img of=/dev/disk2 bs=1m

Yields the following disk layout:

$ diskutil list
/dev/disk2 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   *2.1 GB     disk2

In the sense that MacOS can't make out anything specific, which it doesn't have to.

This is what I think is where things go south as image files can never be rewritten, but a particular OS sees a mounted freshly flashed disk and an "error", which it tries to correct, destroying the boot information in the process.

I can boot that image fine with my old T400 laptop. Package integrity test is ok as well.

There is no particular way an image could mess with our users or their setups. It's a byte sequence and as long as that byte sequence is retained on the boot media, the chances are pretty high that it will boot. Or is there something wrong with that assumption?

Don't forget that i386 images can provide data points because these images do not have UEFI support and therefore lack the extra partitions that could complicate booting. Nano images are also quite different and could help diagnose the problematic area.


Cheers,
Franco

[html]

My experience with rufus 2.11 under Win7:

Rufus dosen't like the iso files. It says "This image is either non-bootable, or it uses a boot or compression method that is not supported by Rufus..."

The img files can be written with the "DD Image" mode. Ok, i also got a bluescreen after the write process. But I think it's my Win7 system which has problems with USB Sticks in gpt format. The created USB Stick itself boots fine and i was able to install opnsense without problems.

Someone else says GParted shows errors on the USB Stick. In my case it says "The backup GPT table is not at the end of the disk as it should be". Yes, I wrote the 906MB image to a 8GB USB Stick. So this warning make sense to me.

[smoore]

As a datapoint, the nano-amd64 build OPNsense-17.1.4-OpenSSL-nano-amd64.img writes smoothly to a USB key and runs without a problem.

I attempted another run with the vga-amd64 build using linux and dd. GParted reports “Invalid partition table- recursive partition on /dev/sdd” and has trouble viewing the partitions. I did not make any changes using GParted, restarted the computer, and it successfully booted from the USB. It took 53 minutes to load the kernel and I had a console login prompt.

Not sure what happened. The nano-amd64 booted to console prompt in a minute, whereas the vga-amd64 took an hour. Regardless, I used the “install” username at the console login, and it worked.

[bogi]

It's not your Windows 7 computer.  I also got blue screens when writing the img file using rufus in dd mode or physdiskwrite.  It happened on two of my computers. I do not think it happened on my Windows 10 computer. I don't remember at this point.

Your best bet is to download TrueOS (Desktop BSD OS) and run that in live mode on your machine  Download opnsense in TrueOS and then follow instructions on how to write the image in FreeBSD.  Might not be easy if your not familliar with linux/bsd command line.   Took me a while.  I found renaming the opnsense filename to something short and moving it into the root of user's home folder to simplify the processs as you don't have to type in a long path/filename.