Messages

Is it possible to make DHCP changes over the API? I have looked the API documentation up and down and I don't see any endpoints that pretain to the DHCPv4 service. My goal is to check existing leases and then convert one of them to static.

What API endpoint is used to configure DHCPv4 service?

And the rest of the answer to the first question: strings like "WireGuard (Group) net" and "OPT1 net" are from the descriptions and are used in the GUI to display these nets. To refer to these same entities in the API, the string to use is found in the "Name" field in the alias listings. They are listed as type "Internal (automatic)". So, the following are the translation for the two I listed with my question:

"OPT1 net" -> "__opt1_network"
"WireGuard (Group) net" -> "__wireguard_network"

and so on. As for the second question: meh. It's probably by design and is just cosmetic so it doesn't matter if it displays the correct information.

I figured out part of the answer to the first question:

"WireGuard (Group)" when referring to it via API is called "wireguard". I'm not sure if any other plugins work in the same way, but the string that the API needs is found in the config.xml section for the interface in the "if" field:

<if>wireguard</if>

I am trying to create firewall rules via API. I am able to create rules using CIDR, but the system aliases like "OPT1 net" or "WireGuard (Group)" or "WireGuard (Group) net" in the source_net or destination_net fields in the POST'ed JSON. Also, I'm unable to refer to the interface "WireGuard (Group)" in the interface field.

Are these strings usable in the API at all?

On a side note, when the rule is created (using CIDR and interface ID), and you look in the web console listing of "Automatically generated rules", the created rule appears to be wide open with "*" in every column on that view. However, when editing that same rule as listed under "Automation -> Filter", everything looks correct.

Is the web UI not able to display the details of the rule correctly? Or is showing all "*" in each column by design and the expected way for the rule to be displayed?

If anyone runs into this installer bug, here is a specific workaround until the problem can be fixed.

1. Login to the OPNsense web console.
2. Navigate to the "Interfaces" menu.
3. Select "Assignments".
4. In the pane that appears on the right, click "Save".

Also, here is the Github issue for the bug:

https://github.com/opnsense/core/issues/5768

I have found a workaround that writes the WireGuard group interface back to the configuration. If I make a change to interfaces such as adding a lock (prevent interface removal) to one of the existing interfaces that does not have a lock, then in addition to the lock, the WireGuard group interface is added back. The message in the config revision section states that "/interfaces.php made changes".

Is there a way to force this PHP to run without making a change? And if so, can this PHP be forced to run via API?

It's on PyPI now.

Code Select Expand


pip install opnsense-confgen


Here is the new package version:

https://github.com/malwarology/opnsense-confgen

I was actually able to ring the 100% unit test coverage bell :)

Code Select Expand

% coverage report -m
Name                   Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------
src/oscg/__init__.py       0      0      0      0   100%
src/oscg/cli.py          105      0     36      0   100%
src/oscg/core.py         171      0     32      0   100%
src/oscg/example.py       11      0      0      0   100%
src/oscg/utils.py         14      0      2      0   100%
------------------------------------------------------------------
TOTAL                    301      0     70      0   100%

I have built a fresh OPNsense install ISO of 22.1.7 via the instructions here: https://github.com/opnsense/tools

The ISO format output for the config.xml works as expected with the same process that I used with the mounted drive and the official OPNsense image. The issue with the WireGuard interface group being deleted during the install is still there, so I noted that.

I will kick it out to PyPI tomorrow so the pip install in the README actually works, but it is late and I'm tired.

Let me know what you think.

I just finished writing unit tests for the package version except for the cli.py file. That's my next task. Once I have 100% unit test coverage for every line of code in the whole package, I will push to Github and you can take a look. I have added the ability to write the config.xml to the correct location in an ISO using "pycdlib". There is a new feature in OPNsense that allows the importer to find CDs, and this fits perfectly.

I think I understand what your difficulty is with regards to no WireGuard instance starting as opposed to IPsec working fine. IPsec is part of OPNsense's base install. There is no plugin required for it to work. Therefore you can just add what you think you need to config.xml and you're good to go. For WireGuard, because it is a plugin, there needs to be a triggering event for the packages for the plugin to be installed. Just having the plugin listed in config.xml is not enough on its own. After the packages corresponfing to the plugin are installed, the WireGuard configuration in the config.xml is picked up and works as one would expect (except for an installer bug that removes the WireGuard Group interface so you have to do without that feature no matter what).

The triggering event that I use is a version upgrade. I start the install and then go to the OPNsense importer. I select the disk with the generated config.xml on it, and then the rest of the first boot occurs. I then run the installer. After the installer is done and the instance has rebooted, I login as root and run the update from console feature. During that update process, the wireguard plugin is detected in the config.xml and rather than updating the plugin packages, they are installed for the first time. After the update process is complete and it has rebooted again, you have a fully working WireGuard bootstrapped OPNsense instance.

When using the OPNsense importer, if the imported configuration includes a WireGuard VPN, the WireGuard (Group) interface is subsequently removed during the install process that follows the importer. I have tried and failed to discover a method for getting it back.



The step where it is removed is "(root): Created web GUI TLS certificate".



I have tried to trigger the creation of the interface by doing the following, but none worked.

1. Uninstalling and reinstalling the plugin.
2. Disabling and enabling the local server configuration.
3. Disabling and enabling WireGuard.
4. Creating a new local server configuration.
5. Creating a new local server configuration with all others disabled and WireGuard disabled.

Anyone have a suggestion for a workaround for getting the interface added back? Also, if there are any pointers as to where this happens in the source code, I could take a stab at fixing it and making a PR.

I have converted the notebook to a formal Python project and modules. I have a few unit tests to complete before I release it, but I researched how WireGuard keys are properly generated. I had used subprocess and called the WireGuard command line program in the same way that OPNsense does. However, I have now learned that the keys are from Daniel J. Bernstein's NaCl.

https://nacl.cr.yp.to/

So, I have gone ahead and replaced the subprocess code with nacl's PrivateKey() functions in the notebook.

In case you want to use it elsewhere, here is the boiled down code to just the stuff you may need.

Code Select Expand


$ pip install pynacl


Code Select Expand


import base64

import nacl.public

def wgkeys():
    """Generate a WireGuard keypair."""
    private = nacl.public.PrivateKey.generate()
    privkey = base64.b64encode(bytes(private)).decode()
    pubkey = base64.b64encode(bytes(private.public_key)).decode()

    return privkey, pubkey


I did some cleanup and refactoring of code to use tag names rather than positions in the XML. It can also handle hostname and domain now. Here is a screenshot of an initial INI with the WireGuard bootstrap keypairs missing so that it generates them dynamically.



This results in a WireGuard config that can be imported immediately. Here is one:



And all that matches up with the WireGuard bootstrap in the OPNsense config.xml:

Thanks, but that Ansible playbook doesn't scratch my itch. I am deploying a new OPNsense instance rather than configuring an already existing instance.

I have updated the notebook again today and released another overhaul. This new version now has a WireGuard bootstrap. If the INI file contains a section "WGB", then it will add all the config sections needed for a WireGuard server to exist in the OPNsense instance from the very start via OPNsense importer. The INI has a few options. If a server private key is missing, the notebook generates a new keypair and inserts them into config XML. If the client private key is missing, it generates the whole WireGuard configuration along with keypair and writes that for you. The end result is a config.xml that can be used with OPNsense importer along with a WireGuard client configuration file. All with fresh keypairs if none were provided in the INI. There are a couple nuances where I deviated from how OPNsense would have created this WireGuard server. First, the interface is in "OPT0" rather than the last OPT number after others are created. This way it lands in the config XML in the order I want without knowing in advance how many OPTs there will be. Second, the WireGuard server is instance 1 and interface is wg1. The reason for this is that the bootstrap should be replaced by one that is generated in the OPNsense instance and the bootstrap one then deleted. By numbering as I have, the permanent WireGuard server and interface will be wg0 and instance 0.

This notebook is feature complete for what I need, so the next task is to look at unattended install in OPNsense. The goal is to have a fully automated and WireGuard bootstrapped install process. As it stands now, there are still three steps that require human interaction: starting OPNsense importer and selecting drive with config; running the installer; and finally running the first updates from console.

NOTE: I know that all three keys in the example INI are malformed. This is on purpose so someone doesn't go using the example keys in their deployment even inadvertently.

PR:

https://github.com/opnsense/plugins/pull/2982

I've been digging into the WireGuard configuration for a project I'm working on and noticed that each time the public and private keys are generated, there are extra newlines and whitespace added to the configuration XML. This whitespace is then removed whenever the WireGuard config is modified for any reason going forward. I've tracked down the source: "tee" is used to write the keys to a file and print them to stdout simultanously, the output to stdout being captured in the config.xml file. This command line adds newlines which cannot be silenced (I checked the tee man page).

This is where tee is used:

https://github.com/opnsense/plugins/blob/3bcfab38f6ea265bf23b5b01eccc4e82f75fbb4e/net/wireguard/src/opnsense/scripts/OPNsense/Wireguard/genkey.sh#L37-L45

Here is where that shell script is called in PHP:

https://github.com/opnsense/plugins/blob/3bcfab38f6ea265bf23b5b01eccc4e82f75fbb4e/net/wireguard/src/opnsense/mvc/app/controllers/OPNsense/Wireguard/Api/ServerController.php#L48-L93

There are three potential fixes along with "just ignore it / who cares".

1. In genkey.sh, the output from tee can be piped to tr: "tee ${TMPDIR}/wireguard.priv | tr -d '\n'" as one example of two.
2. In ServerController.php, each of the four locations where the strings are stored in the config get wrapped in rtrim(): "$node->privkey = rtrim($keyspriv);" as one example of four.
3. Whatever function pretty prints the config XML before writing to file should be called after these modifications are done. I have not found this pretty print function or step, so I don't know what to do for it, but there is clearly a pretty print function being called after subsequent modifications to the config because this whitespace is removed any other time the config is modified.

I can make a PR, but I didn't want to until I check here in case the OPNsense maintainers decide that "just ignore it / who cares" is what they wish.



And, yes, I know that is a private key in the image. It was generated for this screenshot and is not used anywhere.