Python Jupyter Notebook for Generating OPNsense Configuration XML Files

[utkonos]

I've written a kludgy Jupyter notebook that takes a template config.xml and performs a set of replacements and additions based on the contents of an INI file. I'm using it for deployment of a project, but I figured that I would share it here if anyone needs something like this, at least it may be a place to start. There are a number of optimizations that are just for my project, so you may need to bend it to your will to make it useful.

https://gist.github.com/utkonos/57c79f1a0b68dd6a79cbf2de68db995a

[Wolfspyre]

Thanks for sharing this!

I've been looking for something sorta similar myself:

https://forum.opnsense.org/index.php?topic=25901

but maybe I'm just crazy.

[utkonos]

If you find this useful, that's great.

I have made a major update to the notebook. It was initially using a very kludgey way of modifying the config XML. Just jamming new XML in Python text formatting. This is all now totally replaced by code that operates natively on XML using Python's xml.etree.ElementTree library. There are only two tiny cosmetic differences between the output of this script and the output of OPNsense's own config manupulators: Python adds an extra space in tags like this:

<sometag/>

So, they look like this:

<sometag />

This is not configurable that I can see in Python. Another person in a Github issue has identified which XML libraries add this space and which ones don't:

https://github.com/zeux/pugixml/issues/87#issuecomment-188621862

OPNsense can read config XML with these spaces, so it's irrelevant.

The only other difference is that the XML declaration at the very start has an encoding, but the one generated in OPNsense does not. Again, this has no effect on OPNsense's ability to import the config file.

Here is what it looks like from the script:

<?xml version='1.0' encoding='us-ascii'?>

Here is the native OPNsense:

<?xml version="1.0"?>

I may try to figure out how to get this output correct.

[zerwes]

off topic, as it is not a jupyter Notebook.
But the ansible playbook from https://github.com/Rosa-Luxemburgstiftung-Berlin/ansible-opnsense uses a (fetched) config.xml and ensures some settings based on configurations made in yaml files  before provisioning them back to the device ...
So if you skip the config.xml fetch and push step, you have the same effect ... validating could be surely be implemented if someone is willing to contribute this ...

[utkonos]

Thanks, but that Ansible playbook doesn't scratch my itch. I am deploying a new OPNsense instance rather than configuring an already existing instance.

I have updated the notebook again today and released another overhaul. This new version now has a WireGuard bootstrap. If the INI file contains a section "WGB", then it will add all the config sections needed for a WireGuard server to exist in the OPNsense instance from the very start via OPNsense importer. The INI has a few options. If a server private key is missing, the notebook generates a new keypair and inserts them into config XML. If the client private key is missing, it generates the whole WireGuard configuration along with keypair and writes that for you. The end result is a config.xml that can be used with OPNsense importer along with a WireGuard client configuration file. All with fresh keypairs if none were provided in the INI. There are a couple nuances where I deviated from how OPNsense would have created this WireGuard server. First, the interface is in "OPT0" rather than the last OPT number after others are created. This way it lands in the config XML in the order I want without knowing in advance how many OPTs there will be. Second, the WireGuard server is instance 1 and interface is wg1. The reason for this is that the bootstrap should be replaced by one that is generated in the OPNsense instance and the bootstrap one then deleted. By numbering as I have, the permanent WireGuard server and interface will be wg0 and instance 0.

This notebook is feature complete for what I need, so the next task is to look at unattended install in OPNsense. The goal is to have a fully automated and WireGuard bootstrapped install process. As it stands now, there are still three steps that require human interaction: starting OPNsense importer and selecting drive with config; running the installer; and finally running the first updates from console.

NOTE: I know that all three keys in the example INI are malformed. This is on purpose so someone doesn't go using the example keys in their deployment even inadvertently.

[utkonos]

I did some cleanup and refactoring of code to use tag names rather than positions in the XML. It can also handle hostname and domain now. Here is a screenshot of an initial INI with the WireGuard bootstrap keypairs missing so that it generates them dynamically.



This results in a WireGuard config that can be imported immediately. Here is one:



And all that matches up with the WireGuard bootstrap in the OPNsense config.xml:

[utkonos]

I have converted the notebook to a formal Python project and modules. I have a few unit tests to complete before I release it, but I researched how WireGuard keys are properly generated. I had used subprocess and called the WireGuard command line program in the same way that OPNsense does. However, I have now learned that the keys are from Daniel J. Bernstein's NaCl.

https://nacl.cr.yp.to/

So, I have gone ahead and replaced the subprocess code with nacl's PrivateKey() functions in the notebook.

In case you want to use it elsewhere, here is the boiled down code to just the stuff you may need.

Code: [Select]

$ pip install pynacl


Code: [Select]

import base64

import nacl.public

def wgkeys():
    """Generate a WireGuard keypair."""
    private = nacl.public.PrivateKey.generate()
    privkey = base64.b64encode(bytes(private)).decode()
    pubkey = base64.b64encode(bytes(private.public_key)).decode()

    return privkey, pubkey


[zerwes]

Sounds interesting ...
btw. the ansible playbook I linked surely can be used to create a initial config:
https://github.com/Rosa-Luxemburgstiftung-Berlin/ansible-opnsense-playbook/blob/main/firewalls.yml#L24:L31
and we use it right this way: initial deploy and continuous maintenance based on the same tool and configuration  ... but that is another story, don't mind ...

Is the wireguard cfg you generate working if you apply it on a virgin install?

In the env where we use the generated cfg from scratch we use IPSec VPNs, and these work out of the box: (generate cfg, install opnsense, copy generated config.xml in place, reboot, works)
Just as a test  I tried once a device with additional wireguard config from scratch, but no wg instance was started ... as it is not our prod env and not the main problem at the moment, I did not dive deeper into it. (was at least more then a year ago, maybe the updates since then changed behavior ...)
Just interested if your approach creates a working wg config or the approach of just injecting a valid wg config in xml format is not sufficient ...

So some notes on the deploy experience would be appreciated ...