FTP Proxy

[faunsen]

Hi,

I have a simple OPNsense firewall setup with a WAN and a LAN interface and want to make a FTP connection.
As far as I understood I have to use the FTP proxy  for that.

The proxy config is:
General Proxy Settings
  Enable proxy: checked

FTP proxy settings
  FTP proxy interfaces: lan
  FTP proxy port: 2121
  Enable Transparent Mode: checked

Accessing port 21 from LAN is allowed per rule and I can make a control but no data connection.
Squid is listening on 127.0.0.1:2121.

Do I need additional rules, anchors, redirects etc.?


Kind regards
Frank

[franco]

A redirect rule will be necessary, you can use the templates from the help text of HTTP or HTTPS (click to expand all help messages). The only thing you will need to change are the ports in the redirection rule.

[faunsen]

Hi franco,

thank you for the response.

It seems that Squid isn't able to act as a transparent FTP proxy. The data connection always fails.

But I've got it with ftp-proxy.
My setup has now 3 interfaces WAN, LAN and OPT1
The goal is to have transparent FTP from LAN to any via WAN and to OPT1 net via OPT1.

I've added the required "ftp-proxy/*" anchors to /usr/local/etc/inc/filter.inc

Now I've started the ftp-proxy for WAN with listen port 8021 and server address of the WAN interface

Code Select Expand

ftp-proxy -D 6 -d -a <public WAN IP> -r -p 8021

and the ftp-proxy for OPT1 on port 8022

Code Select Expand

ftp-proxy -D 6 -d -a <private OPT1 IP> -r -p 8022

Next the NAT redirection from LAN net to OPT1net port 21 to 127.0.0.1 8022 and
from LAN net to !OPT1net port 21 to 127.0.0.1 8021

and the pass rules from LAN net to 127.0.0.1 8021 and 8022 respectively.

Works with active and passive FTP  :)


And now the question.
What is the correct way to make this in OPNsense?
I guess hacking filter.inc etc. is not a good idea.

I remember old pfSense versions had an option "Start FTP Helper" for each interface.
Is it possible to implement such option?


Many thanks
Frank

[franco]

Hi faunsen,

It's a transparent proxy, but since FTP relies on out-of-band data connections through port 20, it makes it almost impossible to run a full proxy setup.

ftp-proxy works, but our old implementation had sub-optimal solution so it was removed in the hopes of squid being able to replace it. Since it doesn't, I think ftp-proxy is back on the table for 17.1.

This time, we should be able to avoid all potential loopholes, but it requires a more solid integration / configuration than a single checkbox.

I'll write make this a priority after 16.7 is officially out. How does that sound?

Help in getting that done sooner is always appreciated of course. :)


Cheers,
Franco

[faunsen]

I'll write make this a priority after 16.7 is officially out. How does that sound?

GREAT!!

Help in getting that done sooner is always appreciated of course. :)

Of course  8)
Tell me how.
Design, programming, testing, documentation... ?

[franco]

Nice, ok, I've added a new feature ticket here: https://github.com/opnsense/core/issues/1051

Previous problems: listens on all interfaces/opens holes, service start/restart in filter.inc was suboptimal.

The questions posed are important to avoid security risks: would a separate service page be viable with a bit of configuring, mostly which interfaces are allowed to use the proxy? It would give the FTP proxy a more visible feeling, but I'm not entirely sure it's necessary.

What do you think?

[faunsen]

The questions posed are important to avoid security risks: would a separate service page be viable with a bit of configuring, mostly which interfaces are allowed to use the proxy? It would give the FTP proxy a more visible feeling, but I'm not entirely sure it's necessary.

What do you think?

No, I think a transparent FTP proxy should be entirely transparent.
See my comment on GitHub.

For a non-transparent proxy I would stay with squid.

[faunsen]

To answer your questions on github:

• Can / should it be a plugin?
  I think no, because it's not possible to add the needed anchors dynamically. They are hard coded in filter.inc.
• Should we enable this per interface?
  Also no, because ftp-proxy binds to an IP address not to an interface. And this is more flexible. Imagine you want ftp traffic to a server with an origin address other than to a second server but over the same interface.
  To avoid security risks the firewall admin must add appropriate access and redirect rules.
• 
  Should it have its own service page?
  Yes. Define as much ftp proxies here as you need.
• 
  What configuration parameters do we need?
  Basically the parameters from ftp-proxy.
  mandatory parameters:
  
  
  
  • the source address for the control connection (-a 1.2.3.4)
  • the port where the proxy will listen for redirected connections (-p 8021)
  optional parameters:
  
  
  
  • an option to rewrite the source port to 20 in active mode (-r)
  • debug level (-D 5)
  • set the log flag on pf rules (-v)
  • fixed server address for reverse mode (-R 192.168.1.21)
  • idle timeout for the control connection (-t 86400)
  • queued connections (-q queue)

If no one else writes it, I would.
But the 'ftp-proxy/*' anchors are mandatory. Peanuts for the core team  ;)

For the nontransparent proxy ftp/jftpgw could be useful.

[faunsen]

The UI could look like the attached images.

What do you think?

[faunsen]

Hello,

I've written a ftpproxy plugin that manages ftp-proxy processes.
It enables ftp through the firewall and access to ftp servers behind the firewall (reverse mode).
The ftp transfer is completely transparent.

Everything works fine but I miss the hook to synchronize the config in a cluster.
Is it possible to sync a specific config section and do some remote configd actions from a plugin?


Kind regards
Frank

[franco]

Hi Frank,

Sorry, I got too distracted and forgot about this. You did great work!

HA synchronisation isn't pluggable, but we can change that as we have for all types of subsystems that plugins require. We just need the use case to model the plugin-code around.

Do you still require the pf anchor?

Does the plugin have any ports dependencies not currently in OPNsense?

If you want we can pull this into the plugins as a development version?


Cheers,
Franco

[faunsen]

Hi Franco,

great to hear from you. Was quite lost with my monologue  ;)

HA synchronisation isn't pluggable, but we can change that as we have for all types of subsystems that plugins require. We just need the use case to model the plugin-code around.

The use case is to have the proxy configuration at the backup node and start/stop it as we start/stop it on the master node.
The plugin should set, lets say, a "hasync" attribute to the config option that has to be synchronized and the core system handles the synchronization.
Same with remote actions. E.g. a ConfigdActionsField could have the "hasync" attribute and will be executed on the backup node whenever an action on the master was executed.

Do you still require the pf anchor?

Yes, the ftp-proxy depends on it. Please, please add it to the core system.
The plugin contains a patch for /usr/local/etc/inc/filter.inc in the +POST_INSTALL and +POST_DEINSTALL scripts.
This is not optimal.

Does the plugin have any ports dependencies not currently in OPNsense?

No.
But since I know how easy it is to write a plugin I plan to write another plugin for Monit https://mmonit.com/monit/.
There are many services on the firewall simply unmonitored. The admin never gets informed if a service goes crazy.
This plugin would need the sysutils/monit package.

If you want we can pull this into the plugins as a development version?

Yes, that would be nice. The attachment contains the package.

The usage is quite easy.
This is what you would do for a simple setup allowing FTP from LAN to the internet.

• Navigate to "Services -> FTP Proxy Server" and add a ftp-proxy process.
  You can leave the defaults for this setup. The help text is from the ftp-proxy man page.
  The proxy starts automatically if it is enabled and the background color of the row is green.
  If you disable the proxy the process stops and the color changes to yellow.
  To start the proxies on boot the plugin creates a /etc/rc.conf.d/ftpproxy file with the corresponding settings.
• Next add a "Firewall -> NAT -> Port Forward" rule to redirect traffic from LAN to any on port 21 to localhost port 8021.
  Leave the "Filter rule association" untouched to have a automatic "pass" rule.

That's it.


Kind regards
Frank

[faunsen]

Arrrgh! Seems that I've overseen a dependency to jftpgw in the Makefile.
Here is a new one.
Sorry.

[franco]

Yay, thanks! So jftpgw is not needed?
The use case question was rethorical, sorry for the confusion. FTP proxy use case is as good as it gets. :D

For a commit, we do need the plugin directory, not the fixed package. Most of the things can be extracted, but it's probably better to use the ones you wrote.

Need to tweak a few things, make the plugin private (it builds as "ospriv-ftpproxy" so that it's not showing up in the plugins for a release, etc.)

Best would be a pull request for the plugins.git, but I can also bring this in manually.

I like the filter.inc fix, very clever. :)

I will add monit to the binary packages, sure thing.

EDIT: https://github.com/opnsense/tools/commit/d0e52636e5c

[faunsen]

No, jftpgw is not needed. It was an early idea to merge ftp-proxy and jftpgw into one plugin.

The pull request is made. I hope I did it right. I'm very new to GitHub.

Thank you for the filter.inc flowers and for the Monit package  :)


Regards
Frank