Let's improve the docs (and build a new plugin)

[theq86]

Hey!

To ease the learning curve for new contributors and to extend the wiki with some more details regarding development, especially focussing on how to begin, I like to start developing a new plugin for OPNsense.

My goal is to use the documentation as well as the readme files and the source code on github as the only reference. Everywhere I get stuck, because I find the docs are lacking information I want to report here, find a solution and propose improvements for the docs.

Of course, you can make hours-taking deep dive in the source code, travelling from framework A over tool B to source file C. But I want to make the plugins tutorial more streamlined and add some glue knowledge.

About the plugin:
The AutoBackup plugin that I want to write is going to take the config.xml and uploads it to a remote target. Remote targets can be one of FTP, FTPS, SCP/SFTP etc (extendable via Interfaces). A cronjob is responsible for triggering the backup action on defined times. There is already such an option for GoogleDrive but this is just one target.

What needs to be done for this plugin?
- add a cronjob item that can be selected in the cron settings
- creating a GUI (Controllers, Models, Forms, Views)
- adding PHP libraries for remote file protocols, if not already provided (maybe involving composer during build?)
- actions and API functions etc.

Information about how to achieve these things is hard to find. Sometimes some glue components are not documented. e.g how forms interact with the model and internal Phalcon stuff.

Also I like to find a good way on how to set up a complete development environment. Starting with using the tools repo for building, continuing with how to set up a development instance of OPNsense, along with an IDE. So that it is easy to change code inside the IDE and immediately see the changes in the browser.

There will be some more specific questions in the future. However, I'm not looking for the devs to do my work. It's just sometimes I feel something is missing. And I like to ask my questions and I also like the documentation benefiting from our findings.

[mimugmail]

Hi,

I wrote some guides for beginners, perhaps they'll help you a bit:
http://www.routerperformance.net/opnsense/

There's already a plugin for SCP backup in the devel-tree, perhaps you want to extend it?

A good start e.g. would a plugin for bandwidthd or vnstat, they are easy daemons and should be easy to set up.

Happy to follow your journey here 

[fabian]

so you want to try something like this?
https://github.com/opnsense/plugins/tree/master/sysutils/scp-backup

[theq86]

Hi,

I wrote some guides for beginners, perhaps they'll help you a bit:
http://www.routerperformance.net/opnsense/

There's already a plugin for SCP backup in the devel-tree, perhaps you want to extend it?

A good start e.g. would a plugin for bandwidthd or vnstat, they are easy daemons and should be easy to set up.

Happy to follow your journey here 

Thanks, I will have a look at your guides.

so you want to try something like this?
https://github.com/opnsense/plugins/tree/master/sysutils/scp-backup

Yes. But this plugin will have options on which protocol to use. In a version 2.0 you could also specify multiple backup targets using multiple strategies.

----------

Here some first questions:

How can I install a PLUGIN_DEVEL=yes plugin ?
What is the development lifecycle of a new plugin? What steps it takes from devel=yes to no?

As of starting out:

My computer is a mac, so to get the toolchain up I will probably need a fresh FreeBSD VM. Then cloning opnsense/tools and follow the instructions in the opnsense/tools readme. This leads me to a machine I can use for building.
After make update step is done, I will have cloned all relevant git repositories on my - let me call it BuildServer.

Now, the BuildServer is no OPNsense instance itself. So to have a running system, I could launch a second vm - let's call it DevServer. I have seen a vagrant makefile in the src repository's release directory. Can I somehow use it to start up the exact built OPNsense version I cloned?

[mimugmail]

For only Plugins a small Virtualbox VM with OPN is enough. Just Clone the repo. For Dev I use GitHub for Windows ..

[theq86]

Yes, it is just a little bit uncomfortable.

Either I clone opnsense/plugins and define path mappings inside my IDE, then I can work directly with the repository but I have no real code completion.

Or I work directly on the vm, have full code completion inside the IDE but cannot use the plugins repo. Then I would have to remember which of the files belong to the plugin and copy them over to the repo

[fabian]

If you have the git repository for plugins in the VM, you can use "make install" and "make collect" if you mount the directory on the host using sshfs if that is an option for you.

[theq86]

Okay, so collect is install's anti-target 
That's really helpful. I will test this setup tomorrow. Thank you!

[franco]

I shall update the development workflow guide for 18.1. That guide was written prior to existence of plugins all the way back in 2015 shortly after 15.1 came out.

"collect" is one of those additions, but there have been many more.

FWIW, we should take a moment to consider not creating plugins with feature matches. It will complicate merges, support and create a scenario where one contribution will eventually be favoured over the other, creating a community split where none is necessary.


Cheers,
Franco

[theq86]

I agree. If that autobackup is a feature that sooner or later will be on the roadmap for the core, we should refrain from implementing it as a plugin.