OPNsense Forum
English Forums => Development and Code Review => Topic started by: fvanroie on January 09, 2018, 07:46:41 pm
-
One of the major advantages that guided me towards OPNsense is its api framework. In my quest to learn more about the OPNsense REST api (https://docs.opnsense.org/development/how-tos/api.html), I decided to combine it with my PowerShell learning path and turn this project into an OPNsense PowerShell Module. It turns out that these two work quite well together and I currently have a development preview working with about 25% of the api call's available in PowerShell. There is still a lot of room for improvement though, but the foundation is there.
I wanted to check with the users on this forum if there is any interest in an OPNsense PowerShell Module?
I made the source code (https://github.com/fvanroie/PS_OPNsense) available on github, in the spirit of the OPNsense project so users can test, review and contribute. It even has a wiki :)
For Example:
# Connect to an OPNsense
Connect-OPNsense -Url 'https://opnsense.localdomain:8080' -Credential $apiCred -WebCredential $webCred -Verbose -Debug
# Get the config.xml
$xml = Backup-OPNsenseConfig -RRDdata
# Add some packages
Install-OPNsensePackage -Name 'unixODBC','xen-guest-tools' -Verbose
# Install updates & Reboot
Update-OPNsense -Verbose
Now that PowerShell Core 6.0 is available cross platform it is possible to run PowerShell scripts on Windows, Linux and MacOS.
Even if you want to use another scripting language, this module can help you learn the api as all of the cmdlets let you use the -Verbose and -Debug parameters so you can investigate which api calls are made in the background using which parameters.
Of course any feedback is very much appreciated.
Best regards,
/fvanroie
-
Wow, good job! I am not experienced with PS but I can help you with the plugins I wrote. I'll have a deeper look :)
-
Hi there,
I don't use Windows, but this is indeed very cool. :)
Please keep pinging this thread as you make progress. It's hard to miss a one-time post for most lurkers.
Thank you,
Franco
-
PS: Added a tweet! https://twitter.com/opnsense/status/951215423185457153
-
Thanks for your kind words and visibility on the twitter feed. In the mean time I installed a 18.1-RC2 VM to play around with instead of my live environment :)
I'm slowly working my way through the api commands, implementing the low hanging fruit first: like the os-arp-scan plugin and new routes api.
Since Quagga is depricated in 18.1 the api percentage mentioned in my OP is probably a bit higher than estimated. I'm not sure which OPNsense plugins are most popular to implement next. Any thoughts?
There's also a need to implement some automated unit testing, because perfoming all tests manually on a new release is a chore and that time is better spend coding. And I'll be doing some testing in PowerShell Core 6.0 on linux to get a feel for what that's like and identify hurdles there.
The changes will be pushed to the /dev/ (https://github.com/fvanroie/PS_OPNsense/tree/dev) branch for the time being. I'll post back when there's more progress to report.
I am not experienced with PS but I can help you with the plugins I wrote. I'll have a deeper look :)
Thanks :D Would you mind sharing the names of some plugins, I'll have a look to implement them into the PS Module asap!
-
I'm slowly working my way through the api commands, implementing the low hanging fruit first: like the os-arp-scan plugin and new routes api.
Since Quagga is depricated in 18.1 the api percentage mentioned in my OP is probably a bit higher than estimated. I'm not sure which OPNsense plugins are most popular to implement next. Any thoughts?
Quagga and FRR do have the same API. I have not changed it (API endpoints are the same as well as the data stored). So currently an API client for FRR is also an API client for quagga. I did not change it a lot because I wanted to keep compatibility of existing scripts.
Another frequently used plugin is HAProxy (often used in combination with the Let's Encrypt plugin). This one might be complicated but from what I read here it feels like to be the most used plugin.
-
I missed this post but glad i discovered it.
I will definitely take a look.
Thank you for your hard work!
-
Quagga and FRR do have the same API. I have not changed it (API endpoints are the same as well as the data stored). So currently an API client for FRR is also an API client for quagga.
Theat's great news :). There was quite an increase in the Action() functions when I did a git grep on the new code. I didn't realize net/frr and net/quagga actually shared the same api calls... so yeah, it shouldn't make much difference in the end. Thanks for this clarification!
Another frequently used plugin is HAProxy (often used in combination with the Let's Encrypt plugin). This one might be complicated but from what I read here it feels like to be the most used plugin.
Alright I'll start working on cmdlets for the HAProxy functionality and see how it goes.
-
Just a quick update as I have merged a new dev version of PS_OPNsense into master today.
It now has cmdlets for most of the HAProxy objects. However not all the advanced options and properties are available yet, but it is coming along fine... It is now possible to get, add and remove Servers, Backend pools, Frontend services, Error files and Lua scripts.
Furthermore, there is a new Examples (https://github.com/fvanroie/PS_OPNsense/tree/master/Examples) folder in the project which shows how to implement these cmdlets into scripts.
I'm also implementing automated Pester tests into the development process to reduce the time needed to test and validate all functions. This already comes in handy when testing the module on new platforms and OPnsense releases. I have PowerShell with PS_OPNsense running in Ubuntu 18.04 beta. The module is indeed working in PowerShell Core 6.0.1 for Linux ;D
So lots of progress and lots of new ideas for additions and improvements to come... Last but not least, any feedback and testing is encouraged. :)
-
Awesome, thank you for the updates!
-
It's been a while, but today I've updated the master banch with version 0.1.5 of PS_OPNsense (https://github.com/fvanroie/PS_OPNsense). There have been a lot of changes under the hood that should make maintaining the code a lot easier towards the future.
Notable changes:
- Improved Firmware Updates (see screencast below)
- Drastically reduce the number of cmdlets and code base
- Use OpenAPI json specification for describing the API calls
- Strong typing of CRUD Items allowing cmdlets to identify objects coming from the pipeline
- All CRUD Items can now be queried and removed using Get-OPNsenseItem ans Remove-OPNsenseItem
- Base functionality to query all Settings, but needs more work
- Install, remove, lock, unlock and list Packages and Plugins
- Added legacy cmdlets to query Vlans and Certificates
e.g. remove HAProxy server named Web001:
Get-OPNsenseItem -HAProxy Server -Filter 'web001' | Remove-OPNsenseItem
I'll be updating the documentation for PS_OPNsense and the API Reference next.
-
Very cool update feature, thank you for your work! :)
Cheers,
Franco
-
Two quick updates to report as I had a small breakthough into making the PowerShell Module more user friendly going forward.
- I'll publish a Dockerfile soon, so anyone can easily test the PowerShell module in a container.
- I am adding the ability to 'mount' and browse the OPNsense Menu and Items like it were a filesystem.
It's just a proof-of-concept, but it's working for some objects already:
-
Amazing module, thank you fvanroie!
I'm having a hard time figuring out how to execute a major update / upgrade though. anyone had any success here?
-
Hello
If you'd like, I've taken fvanroie's module as a starting point and i have made new module to the user part.
Github : https://github.com/G2ft/PSTools/tree/master/OpnSense
it's not perfect, but I use it on a daily basis and improve it as I go along.