os-caddy plugin

[Monviech (Cedrik)]

@mietzen

Getting help sounds like a good idea. But before the scope is adjusted, the base functionality should be reached, I've created the MVC model and views in a way that adheres to other plugins so it can be extended later with more functionality. But don't forget, I don't have much experience, this is like a challenge to see if I can reach a goal with a small scope.

Also, since the caddy binary package will be a dependancy of the os-caddy-plugin package, other extensions could be probably compiled in?

-----

So far, all the additions I have made to other MVC parts in the OPNsense, have mostly been totally confined to the front end. I have adjusted models, views, controllers, but I have never actually layed my hands on the SettingsController. It's where I really struggle right now.

The Hello World example is easily reproducable with the "ApiMutableModelControllerBase", I can save my general settings and retrieve them properly from the config.xml.

But since I have to use a grid with multiple entries to allow for more than 1 reverse proxy entry, I have to write multiple uuids to the config.xml that are all in a big array.

I think once I have reached that first goal, I would be a little more confident. Then I'll commit my current front end to github and we could talk about it if you want.

[mietzen]

Sounds great, just ping me on GitHub.

[Monviech (Cedrik)]

Oh yeah, finally making some progress with the API, by looking at the IPSEC implementation, especially the api/ConnectionsController.

Testing my api/ReverseProxyController:

Add one entry to the array:

/api/caddy/ReverseProxy/add

Code: [Select]

curl -v -k -X POST https://192.168.3.1/api/caddy/ReverseProxy/add \
     -H 'Content-Type: application/json' \
     -H "Authorization: Basic API-SECRET" \
     -d '{"reverse": {"Enabled": "1", "FromDomain": "example.com", "FromPort": "80", "ToDomain": "localhost", "ToPort": "8080", "Description": "Example reverse proxy entry"}}'

Get all entries as array:

/api/caddy/ReverseProxy/get

Code: [Select]

{"reverse":{"87936553-df1f-447e-8284-929daa40a6c1":{"Enabled":"1","FromDomain":"example.com","FromPort":"80","ToDomain":"localhost","ToPort":"8080","Description":"Example reverse proxy entry"},"14219e01-8545-41cb-8ae4-b0232f47a9e2":{"Enabled":"1","FromDomain":"example.com","FromPort":"80","ToDomain":"localhost","ToPort":"8080","Description":"Example reverse proxy entry2"}}}
Get one specific entry as array:

/api/caddy/ReverseProxy/get/87936553-df1f-447e-8284-929daa40a6c1

Code: [Select]

{"reverse":{"87936553-df1f-447e-8284-929daa40a6c1":{"Enabled":"1","FromDomain":"example.com","FromPort":"80","ToDomain":"localhost","ToPort":"8080","Description":"Example reverse proxy entry"}}}

Testing the api/GeneralController:

/api/caddy/General/get

Code: [Select]

{"general":{"enabled":"1"}}


[mimugmail]

I'm not sure if its a good idea to compare with complex core code.
Maybe first start with simple copy +paste .. most of it doesnt need any special Logik

https://www.routerperformance.net/opnsense/plugin-development/

[Monviech (Cedrik)]

I have managed to build a first alpha version that has basic functionality and can be installed via repository.

It works on the Opnsense community edition. I have also tested it on the Business edition, but there I can't seem to get it to work.

Problem: I have prepared the storage path of Caddy to write all files into /usr/local/etc/caddy/ via a directive in its /usr/local/rc.d/caddy service. In the CE the folders "acme" and "lock" are created and Caddy can start and begin to create certificate requests.

In the BE edition the "acme" and "lock" folders won't get created, which means Caddy can't work properly. Everything else does seem to work though. The template generates the Caddy file and all actions work (start/stop/restart), the scripts all work, the API works, the GUI works, everything. I don't find the difference and I'm stuck.

I have set the logging path into /var/log/caddy/caddy.log

If anybody can help me, the current alpha plugin can be installed with: (DISCLAIMER: NOT PRODUCTION READY. EARLY ALPHA. But it won't crash your firewall.)

Code: [Select]

fetch -o /usr/local/etc/pkg/repos/os-caddy-plugin.conf https://os-caddy-plugin.pischem.com/repo-config/os-caddy-plugin.conf
pkg update

Afterwards the "os-caddy" plugin can be installed from the GUI. Make sure the Firewall doesn't listen on port 80/443, since Caddy uses 80 for ACME, and you need 443 to create a proper Reverse Proxy entry.

Under "Services" you will then find "Caddy Web Server"

All source files are uploaded to my github. (Link at the start of the thread)

If you have feedback, please be kind, I am not a professional developer. It's just a hobby on the side. I am not experienced. I have just managed to get something working after a lot of work (probably +100 hours) and I'm rather proud at least something happens.

Thank you. 

[mimugmail]

You can adjust permissions here:
https://github.com/opnsense/plugins/blob/master/net/tayga/src/opnsense/scripts/OPNsense/Tayga/setup.sh

Execute via rc options:
https://github.com/opnsense/plugins/blob/master/net/tayga/src/opnsense/service/templates/OPNsense/Tayga/tayga

[Monviech (Cedrik)]

@mimugmail

Thank you, thats really helpful. 

- I found a little bug in my template and when to port is left empty.
- The Caddy on my firewall didnt start not due to CE or BE... I just had port 80 already occupied. Caddy hates that. I warned everywhere and then I forgot myself to deactivate lighttpd on port 80 by deactivating gui redirect, - Im such a goof.
-I still wrote a script that installs all folders.

The reverse proxy actually works. I'm so relieved. The GUI works and the reverse proxy works... Oh yes. The baseline is functional!!!

[Monviech (Cedrik)]

I would like to know the "secret sauce" that the service is picket up by this api:

/api/core/service/search

I want to have the caddy service in the "service widget" on the dashboard, and I would like if one of those service buttons (start/stop/restart) appear in my "views".

My ServiceController.php already extends the ApiMutableServiceControllerBase.php and I have made sure my service is named correctly in the internalService* properties.
https://github.com/Monviech/os-caddy-plugin/blob/main/usr/plugins/devel/caddy/src/opnsense/mvc/app/controllers/Pischem/Caddy/Api/ServiceController.php

I have overwritten the ApiMutableServiceControllerBase functions for restartAction, startAction and stopAction because when I used the inherited functions the "configdRun" somehow wasn't triggered and the "configctl template reload" action didn't trigger. And I didn't really find out why so I stopped looking deeper.

Thank you for any help or insight 

[mimugmail]

https://github.com/opnsense/plugins/blob/master/net/tayga/src/etc/inc/plugins.inc.d/tayga.inc

Interface section not needed

[Monviech (Cedrik)]

@mimugmail

Thank you, I got so much further with the service API stuff because of your help.

My service is shown in the widget now and can be controlled by it. Using the pluginctl as status action, I could also get the service status api working, using the base function provided by the ApiMutableServiceControllerBase:

/api/caddy/service/status
{"status":"running","widget":{"caption_stop":"stop service","caption_start":"start service","caption_restart":"restart service"}}
{"status":"stopped","widget":{"caption_stop":"stop service","caption_start":"start service","caption_restart":"restart service"}}
{"status":"disabled","widget":{"caption_stop":"stop service","caption_start":"start service","caption_restart":"restart service"}}

Now one of my last remaining mysteries (for now), I just can't find out how to get this to be populated:

<class="btn-group-container" id="service_status_container">

These are the buttons that appear in views directly to be able to control the service with. I have found the "service_status_container" in the base volt view, and I have checked that it is indeed in the source code of my own views. But it is not populated or showing. Are there special requirements? I thought I would have met them by now by having the service widget and the "/api/caddy/service/status" running.

Thank you so much for any suggestions. 

[mimugmail]

Actually I dont know where you cloned the code, but it was a way old plugin, usually you dont need so much JS for basic stuff

[Monviech (Cedrik)]

TLDR; I really need help to clean this up to get it into a presentable state. I really want somebody with experience to help me. 

----------------------------------------------------------------------------------------------------------------------------------------------------

All code I have outright cloned and accustomed to my needs has the appropriate licenses maintained. I have followed the HelloWorld Example and looked at a lot of the other plugins whenever I got stuck. And because I really can't do Javascript I've used tools like ChatGPT and Github Copilot to gradually get it working. I have to say, the volt files consumed most of my time and they will be simplified over time. But my approach was mostly a learning experience to understand how things interact with each other. And this lead to a pretty good understanding of things I didn't have before, coming from the admin side of things.

Sure, a proper trained developer can do miles better than me. But I didn't give up, followed through, and now I have something actually working.

And I'm thankful for your help so far. Really. Please help me if you can clean the code up. I'd be so grateful, and we can have a nice little easy reverse proxy plugin for the community.

If not, I'll just leave it as it is and maintain it for myself. Wouldn't be a big loss since there are already multiple reverse proxies, and also your own caddy plugin.

I didn't intend this to sound like a rant by the way, I just try to be open about everything I do and use.

[mimugmail]

As I said, hop on to IRC during work hours and ping me

[Monviech (Cedrik)]

I reworked almost everything and the plugin is in a good state.

I need a tester or two. I'd be really happy to get some general feedback if it works for you or not, and if you find any weird bugs. If you need a reverse proxy that's relatively easy to configure, try it today.

[Monviech (Cedrik)]

Released a few new versions and the plugin is in a state where I consider it stable. I use it productively on multiple firewalls.

It's pretty much feature complete, there won't be any major changes now. Just occasional fixes, maintanance and keeping the Caddy binary up to date.

So far around 15-20 people seemed to have installed it. I also created a forum post in the Caddy forum and got a little bit of feedback, generally there don't seem to be problems.

https://caddy.community/t/caddy-v2-as-reverse-proxy-with-gui-in-opnsense/22047/13

Thanks @everyone for your help.