Quick tutorial: how to deploy OPNsense easily on Oracle Cloud for free.

[jazzy]

Hi Everyone,

Been using pfsense for many years before switching to opnsense at its inception.
Used the forums for years silently, and am very happy with this fantastic piece of software.

Today I am reporting a successful deployment of Opnsense on an Oracle Cloud free instance.
Thought I'd share this short tutorial to give back to the community, hoping it'll be useful to someone out there:

Some Background (you can skip this section and go straight to the OCI tutorial):

I have been using Opnsense on several VPS providers successfully over the years.
First Digital Ocean, then Vultr, then Upcloud.
The switch was done each time to select a VPS location closer to me for better latency.
The three were good from a technical standpoint, although Vultr is to be avoided if you do not want to be outright scammed. 
Was most recently on Upcloud, they have a great platform, great customer service, but they jacked up their monthly subscription price by nearly 50% overnight due to rising energy costs, so I decided to check out alternatives.
A few months ago I started a free trial on OCI. Unfortunately the location I had selected did not have any available capacity during the initial free trial period.
Recently some "Always Free" AMD instances, 1CPU core / 1GB ram became available in my preferred location, so I decided to deploy Opnsense.
Freebsd & Opnsense are not offered as options on OCI for X64, so a bit of creativity was in order.

OCI free instance deployment steps:

-Prepare an xml config for the Oracle deployment on a local machine.
The main required changes from vanilla were:
      - 1 NIC only for WAN, configured via DHCP
      - Disable HTTP_REFERER enforcement check
     
- Create a VCN and Public subnet on OCI, setup the Oracle firewall to allow SSH/HTTP/HTTPS ingress

- Fire up an Oracle Linux instance on AMD (mine was VM.Standard.E2.1.Micro -1 core 2.55 GHz AMD EPYC\u2122 7J13).
An SSH key was uploaded during instance setup for remote access.

- Convert the Oracle Linux instance into a FreeBSD instance via SSH using:

Code: [Select]

# wget https://download.freebsd.org/ftp/releases/VM-IMAGES/13.1-RELEASE/amd64/Latest/FreeBSD-13.1-RELEASE-amd64.raw.xz
# xz -dc FreeBSD-13.1-RELEASE-amd64.raw.xz | sudo dd of=/dev/sda bs=1M conv=fdatasyncWhere sda is the boot disk.
Reboot to Freebsd

- Upload the prepared config.xml to the Freebsd instance using ssh

- Convert FreeBSD into Opnsense via SSH using opnsense-bootstrap:

Code: [Select]

# pkg install ca_root_nss
# fetch https://raw.githubusercontent.com/opnsense/update/master/src/bootstrap/opnsense-bootstrap.sh.in
# sh ./opnsense-bootstrap.sh.in -r 23.1
- Using web based cloud console interface, load the previously prepared config.xml during opnsense first boot

Now the Opensense instance is accessible via the web interface for further configuration.
Since this setup was done using an Always Free instance, the good news is this cloud based edge router comes at no cost.

Enjoy 

[Maurice]

Great tutorial! Much easier than my old method which involved uploading your own customised OPNsense image.

I would be a bit concerned that overwriting the volume while Linux is running from it could cause data corruption. Anyhow, you can avoid this by temporarily attaching the boot volume to a separate compute instance.

Uploading a config.xml is optional btw. You can also use the cloud console to perform the initial interface assignment and then use the Web GUI for everything else. If a machine only has a WAN interface, OPNsense by default allows Web GUI access from the WAN.

Next challenge: OPNsense on OCI Ampere instances (aarch64). More bang for your no buck. There even is an official FreeBSD 13.1 image for OCI Ampere, so you could skip the dd part. Unfortunately, opnsense-bootstrap doesn't support aarch64 yet.

Cheers
Maurice

[arkanoid]

can confirm that this works, and you don't need to prepare or download anything locally. Every step can be accomplished using the cloud console

thanks a lo! Wiping /dev/sda while running an OS mounted on it is wild!

[Maurice]

In hindsight, I don't think these wild workarounds are actually worth it. I've since switched to the "normal" workflow (upload qcow2 image to storage bucket, then import as custom "Generic Linux" image). Much faster and easier. You need a "Pay As You Go" account, but get to keep all "Always Free" resources. My September bill was 8 Cents - because I needed a little more than the free 3000 Compute A1 OCPU hours. I don't think I've ever exceeded the free quota for custom images.

Oh, and I've also completed the switch from VM.Standard.E2.1.Micro to VM.Standard.A1.Flex. Much more powerful!

Cheers
Maurice

[jazzy]

Hello

Many thanks for the heads up regarding aarch64, and for hosting the files.
I have managed to install opnsense on a free A1 instance (4cores,24GB).

In summary:
Using a linux instance, I copied the contents of your qcow2 file into a boot disk, that I then attached to a full blown A1 instance.

Everything works well so far, there was no need to upgrade to a paid account (like you said, I agree a "pay as you go upgrade" would have made it a tiny bit easier, but it can be done whilst staying on the Free Tier).

[danielhainich]

i have uploaded the xml file to freebsd.
how i can import the file within config import?

Press any key to start the configuration importer: .
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)

<QEMU QEMU TARGET 2.5>             at scbus2 target 0 lun 0 (pass0)
<ORACLE BlockVolume 1.0>           at scbus2 target 0 lun 1 (pass1,da0)

Select device to import from (e.g. ada0) or leave blank to exit: da0

Starting import for partition '/dev/da0p3'.

Running fsck...done.
mount: /dev/da0p3: Invalid fstype: Invalid argument
The device could not be mounted.
The file /conf/config.xml could not be found.


where i have to put file in?

/conf/config.xml ?



thanks
daniel

[jazzy]

Sorry I just read your post, and responded to your private msg.

As explained there, one way to do it is to use a separate block volume.

[Maurice]

There is no need for a config import during first boot. Just perform the interface assignment (WAN = vtnet0), set a secure root password, log in on the Web GUI and go from there.

[danielhainich]

i got it running.
with cloud-console i didnt got a prompt.
i placed the config now in /conf/config.xml
after running the opnsense-script the config was imported automatically during boot. now all is working fine!

thanks a lot!

[Alfa1234]

Hi guys, congratulations for the excellent work!

My dream would be to add a kubernetes cluster on oracle (OKE) to this setup that is protected by the OPNsense firewall, i.e. having the k8s cluster nodes behind the firewall.
This way I could make an on premise firewall scenario that has a site to site vpn with the OPNsense on oci, which would give me access to the micro services on the kubernetes cluster (like pi-hole).

I don't know which of the 2 solutions may be best:
1) use an oke cluster
2) create a VM and install a k8s cluster as if we were in an on premise scenario, but losing flexibility of oke

Any advice/suggestions?

Just to make sense of my post on this thread, you guys are going to use OPNsense for what/how on OCI?

[Maurice]

Hey Alfa,

I'm actually not using OPNsense OCI instances as a firewall for other machines in OCI. Primary use cases currently are:

• DNS-over-TLS server with ad blocking, to be used with Android devices.
• WireGuard tunnel endpoint. When the cable WAN at home fails and my home firewall falls back to LTE, I can no longer access it remotely because the mobile ISP blocks all inbound connections. "OPNsense Home" then connects to "OPNsense OCI" via WireGuard and the home network remains accessible via OPNsense OCI.

I have no experience with Kubernetes, sorry.

Cheers
Maurice

[Bob.Dig]

In hindsight, I don't think these wild workarounds are actually worth it. I've since switched to the "normal" workflow (upload qcow2 image to storage bucket, then import as custom "Generic Linux" image). Much faster and easier. You need a "Pay As You Go" account, but get to keep all "Always Free" resources.

Hi Maurice, can you give a detailed explanation how to solve the "Disable HTTP_REFERER enforcement check" problem? I don't want to upload a config file if I don't have to.
A oneliner would be good because I can't save with nano in the Cloud Shell.

[Maurice]

This problem is caused by Oracle's use of IPv4 SNAT, so make sure the instance has an IPv6 address. You can configure this when creating the instance or add one later in the instance's network settings. Then, access the Web GUI using its IPv6 literal, e. g. https://[2001:db8:1:2::a].

Next, go to System: Settings: General and enter the hostname and domain you want to use to access the Web GUI. You must create matching AAAA / A records in your DNS zone of course. You can then access the Web GUI using this hostname, e. g. https://opnsense.example.com. Adding a valid certificate is strongly recommended, you can use the ACME plugin for this.

[Bob.Dig]

This problem is caused by Oracle's use of IPv4 SNAT

Sry, I can't follow, IPv6 isn't supported on the free tier right now.

The only problem for me so far is that I get the "Disable HTTP_REFERER enforcement check" in the web-UI.
I need to disable it!
In pfSense this can be easily done in the console, not so much in OPNsense.
Also I can't connect via putty at this point, although I allowed this in OCI.

So I was trying changing the config with "sed" but no luck so far. Also I am only a novice user.

[Maurice]

I already used IPv6 when I was still on the free tier and would be very surprised if they removed this. I'm happy to assist with issues specific to OPNsense on OCI, but I can't give a full introduction to OCI in general. There is a slight learning curve, but Oracle has pretty decent documentation.

Oracle assigns a private IPv4 address to the instance and then uses SNAT to translate it to the public IPv4 address. OPNsense is not aware of this and only sees the private IPv4 address. That's why it refuses connections when you enter the public IPv4 address in your browser.

IPv6 doesn't have this issue because the instance gets assigned a public address - there is no NAT.

You can't initially use putty because SSH is disabled by default. You have to enable it in the OPNsense Web GUI first.

There are certainly other ways like manually editing the config.xml, but I haven't tried that. Have you tried using ee on the cloud console? That should work.