Immortal ghosts from the past

[MrCCL]

Today, in the year 2016, I was going to update the BIOS of my SuperMicro motherboard and went to the download section of SM's homepage.
The guide and the attached README file explained how I should do this with a floppy disk...WTF



By some extra searching in their knowledge-base I found some words about doing this using a DOS bootable USB key....and a link to an external site with guides and tools. It was obvious SuperMicro didn't believe the vast majority of their customers would go down that path 



Installing OPNsense.....now I run into words like "CD-ROM" and "serial"?...was is that?...Wikileaks was my savior 




Are valuable developer time wasted which could be used for better things?

On top of that, having this serial option around, might give someone excuses for not enabling LAN access initially (telnet/ssh)?

Looking at the installation guide, one could almost get the impression that the USB-Key install path is the least likely method to be used, although I believe it is the opposite.

I believe the vast majority would prefer to just plug-in two things: LAN cable and USB-Key and then be happy 

No fooling around with a keyboard, monitor, video-cable and/or serial cable...which probably include USB-COM converter, drivers, and now - how was it with those evil serials, should it be Null Modem and Straight Through ??...I throw out my last one in 1997 and have had a more enjoyable life since...and I cannot remember when one of my laptops had a COM interface. And then not to mention tipping your cola while trying to connect your VGA cable to the back of your monitor....the sufferings just go on and on 

Maybe the installation and initial configuration could be optimized for the vast majority in the year of our Lord 2016

Extra note: I started with PFsense....but recently moved to OPNsense, and I just love OPNsense and the community more and more, I'm a huge fan and very grateful for work of the developers!
It just include some pleasure to sit on your ass and grumble about other people's work, when you don't have developer skills yourself, so it's my only way to contribute

[weust]

Not to be too blunt, but you've never worked with network appliances, right?
They all have serial connections because that's what is showing you a console right from the start of a power up.
Routers/firewall with serial connections, same thing.

[Zeitkind]

50% of my firewalls do not have a video connector nor any video hardware on the board. That's why most firewall distro come with a flavor for serial connections.

[franco]

Hi all,

Again, welcome, thanks for joining and enjoy the show! This is an interesting discussion, let me give you a brief overview of how/why we got into this "mess" and that we cannot possibly escape it.

We started with VGA, SERIAL and CDROM, similar to what our parent project provided. We were aiming for amd64-only with our initial 15.1 as we hoped we could get away with it, but very quickly provided i386 images as well (popular demand doesn't do justice, rather an imposed community requirement), doubling the image and package repository efforts we do. That added 3 images to our initial stash: CDROM, VGA and SERIAL.

All those images are full live systems with an installer, you can use them to try temporary changes in images or system setup by importing the configuration from the disk of the system you're booting, then exit the installer to see how your changes perform with another version. That's highly useful, but adds complexity as well. This is not a good trend... but wait there's more!

Having tackled the 15.1 i386 needs, just one month after 15.1 came out we were hit by the m0n0wall EOL endorsement and people quickly asked for a NanoBSD version to run on SD cards, so we added the NANO flavour in a simplified form (only one active slice). The image is just a disk copy to be pushed to USB, SSD or SD or CF storage. That's 8 images we have now.

VGA images are a GPT-boot based variety that just now gained optional UEFI boot support. During 16.7-RC1 we didn't provide these temporarily and people quickly asked why they were missing. In theory they are like SERIAL, but people will ask for VGA when they want it. I guess it's a labelling thing.

SERIAL images are a MBR-boot based variety for embedded hardware (well, anything that doesn't have VGA) and cannot support UEFI boot. Not much to say about this other than that it's an essential image type. There is potential to merge VGA and SERIAL, but up until now it's been too difficult to change that due to challenges in the base OS. Attempts to help pull this off are more than welcome.

In 2016, ISOs are perfect for installing VMs short of having native VM images and just now gained UEFI support. Though we added support for native VM disk images in 16.7, we don't build them by default because they have too many options. This don't even includes default disk sizes, people asking for 10G images instead of 20G... the ISO install from a CDROM image is so much easier for all of this. Cannot get rid of those and provide all of:

qcow: Qemu, KVM (legacy format)
qcow2: Qemu, KVM (not backwards-compatible)
raw: Unformatted (sector by sector)
vhd: VirtualPC, Hyper-V, Xen (dynamic size)
vhdf: Azure, VirtualPC, Hyper-V, Xen (fixed size)
vmdk: VMWare, VirtualBox (dynamic size)

As a sidenote for CDROM, there is a hybrid image that also boots when pushed to USB, essentially emulating a VGA or SERIAL image, but it's not native to FreeBSD as it is rather Linux-y, requires third party tools to be installed in the build system and for UEFI I'm not sure they do work that way anymore easily.

In terms of my daily usage with systems, this is the order of importance:

1. CDROM
2. SERIAL
3. NANO
4. VGA

That's a bit different from the OP perspective but not necessarily true for the average user.


Cheers,
Franco

[MrCCL]

Thanks Franco for an interested inside of OPNsence.....everything is always more complex than it looks like at first glance...and I'm even more impressed of the work put into this. :-)

To all the replies...I haven't heard one argument for not starting a telnet or SSH process when FreeBSD boot up! I can only imagine it's  for the sake of security, but I don't really see that as a real issue, it's more a theoretical one.

I'm also familiar with the fantastic OpenWRT project....their solution is to disable telnet automatically as soon as the root password is set.....works like a charm and that's non-VGA and non-serial devices!

[weust]

Well, there is nothing secure about telnet. Much like FTP isn't save.
All traffic is unencrypted.

SSH is, but unless you really need it you don't enable it on a firewall.
Unless you have the management port in a separate VLAN. And your client connects to a machine in that VLAN which is then the only machine allowed to connect to that firewall. Or other network components.
At least, in a production environment it's how I would go for it.

Everything you need to do, you can by using the webconfig page. So why do you want SSH enabled by default?
Nice for a server, not an appliance like OPNsense.

[Zeitkind]

To all the replies...I haven't heard one argument for not starting a telnet or SSH process when FreeBSD boot up!

It might be a little late then to enter BIOS settings..

[fabian]

To all the replies...I haven't heard one argument for not starting a telnet or SSH process when FreeBSD boot up!

It might be a little late then to enter BIOS settings..

Or it does not even get started because of an issue in the boot process

[MrCCL]

@weust: I think anyone in here knows about telnet and what it implies in regards to security.
And no one says anything about keeping your telnet open!
And if you go for something like OPNsense you have a certain level anyway.....don't be overprotective for no good reason.

One of OPNsense main objective is to have all configuration done through your ethernet cable...but they just don't follow this objective to start with?

Again, I hear a lot of talking about all the nice things you can do with serial, and I'm not saying it's useless, but no argument against enabling telnet/SSH to start with. I claim it would make the configuration easier for the vast majority. How many of OPNsense's users don't use serial and would benefit of telnet/SSH...70%, 80% or 95%?

I only have experience with debian, but I have no reason to believe it's not as easy to enable it in FreeBSD!

It seems it's just at matter of changing mindset, no technical reason or because it would imply a lot of work....correct me if I'm wrong.

[weust]

But you can't use telnet/SSH straight from startup (BIOS/UEFI) so it's either Serial or VGA (IPMI).
The telnet/SSH daemon won't be started till right before the login prompt is available. So how will you diagnose problems?

You can't just replace serial/VGA with telnet or SSH.
Same with Ethernet. What will Ethernet do for you if it isn't available until the interface is set with an IP address, and then have to wait for the telnet/SSH daemon to start.
Unless you have IPMI.

You're really mixing two very different things.

[franco]

One of OPNsense main objective is to have all configuration done through your ethernet cable...but they just don't follow this objective to start with?

I'm not entirely sure how this would fail, except for hardware reasons related to booting FreeBSD.

If FreeBSD runs on the hardware OPNsense will boot up and either:

(a) auto-configure for LAN on the only adapter it finds so that you can get a DHCP lease for your laptop that you plug in from that port and configure from there.

(b) auto-configure for LAN on the first adapter it finds, WAN on the second. LAN will offer DHCP leases as described in (a) here too.

The trouble is probably with the installer which auto-boots early so you require a working screen and keyboard of some sort, either through VGA/keyboard or serial.

I remember using untangle once, which booted a full linux X server environment and openend the installation wizard in a browser and I thought that's nice, but why go through all the trouble? This still requires a VGA/keyboard and even mouse and doesn't work on a serial.

So, how should the installer behaviour on install media be improved? We can boot up on install media, make the default to not start the installer early, but after the system is up and running? Would that change anything?

It sounds like something that can be done and there is no excuse for the current behaviour except it's always been done the that way.

The question is whether this really improves things, or that we still have to make a GUI-based installer using valuable human resources, which will take a lot of time to fix something I don't know being broken or just overly technical, or let's say "domain-specific".


Cheers,
Franco

[franco]

PS: As funny as it is, I just remembered the installer works via SSH, so maybe the location of the (default) installer invoke is really wrong and ought to be changed for 17.1.

[MrCCL]

@weust: yes, I'm aware you'll not be able to edit the bios with telnet/SSH.
But we're talking about what's beneficial for the vast majority. Yes....there will in rare occasion be need for a low-lever access like serial/VGA...I agree.
But that doesn't have to be "supported" by OPNSense, does it?
Maybe we can cut down the amount of images and make life easier for Franco & co. :-)

@franco
Maybe I get it wrong. But when I boot from installation media e.g. USB flash drive, bootstrap is loaded (I don't know what it is, perhaps a boot-manager). But the important thing is "1: Boot Multi User" will automatically start after a few seconds, i.e. no need for keyboard, VGA or serial here....it will just continue.
(Do any ever need that boot menu?)

Now FreeBSD is booting/loading. And why not load the NIC driver (maybe it already is) and the telnet service/process, pick the first NIC it finds and set IP to 192.168.1.1. I would prefer myself a static IP...isn't the at standard way....that's actually also the current setup now after installation ends I believe.

After FreeBSD is finish loading, play a sound as a signal, and you can now logon using telnet.
I guess you have to start the installation program manually typing a command. Maybe a message on screen saying "Type START to install OPNsense 16.7"

I don't now if the current installation "program" is telnet-console-compatible?
If not I can see that's a challenge if the install "program" has to be rewritten to be that :-(
But maybe it just work out-of-the-box? :-)

[MrCCL]

Oh....I see you once again are ahead of things....good to hear the installer works from a remote console :-)

[packet loss]

MrCCL support for serial and vga is essential and they aren't going anywhere any time soon. I use serial in particular so I don't have to connect a monitor, keyboard or mouse to my firewall appliance. Yes telnet and SSH can reach the command prompt and be beneficial, but I don't enable services that have a history of being exploited unless I have to,