Hi,
I have a simple OPNsense firewall setup with a WAN and a LAN interface and want to make a FTP connection.
As far as I understood I have to use the FTP proxy for that.
The proxy config is:
General Proxy Settings
Enable proxy: checked
FTP proxy settings
FTP proxy interfaces: lan
FTP proxy port: 2121
Enable Transparent Mode: checked
Accessing port 21 from LAN is allowed per rule and I can make a control but no data connection.
Squid is listening on 127.0.0.1:2121.
Do I need additional rules, anchors, redirects etc.?
Kind regards
Frank
A redirect rule will be necessary, you can use the templates from the help text of HTTP or HTTPS (click to expand all help messages). The only thing you will need to change are the ports in the redirection rule.
Hi franco,
thank you for the response.
It seems that Squid isn't able to act as a transparent FTP proxy. The data connection always fails.
But I've got it with ftp-proxy.
My setup has now 3 interfaces WAN, LAN and OPT1
The goal is to have transparent FTP from LAN to any via WAN and to OPT1 net via OPT1.
I've added the required "ftp-proxy/*" anchors to /usr/local/etc/inc/filter.inc
Now I've started the ftp-proxy for WAN with listen port 8021 and server address of the WAN interface
ftp-proxy -D 6 -d -a <public WAN IP> -r -p 8021
and the ftp-proxy for OPT1 on port 8022
ftp-proxy -D 6 -d -a <private OPT1 IP> -r -p 8022
Next the NAT redirection from LAN net to OPT1net port 21 to 127.0.0.1 8022 and
from LAN net to !OPT1net port 21 to 127.0.0.1 8021
and the pass rules from LAN net to 127.0.0.1 8021 and 8022 respectively.
Works with active and passive FTP :)
And now the question.
What is the correct way to make this in OPNsense?
I guess hacking filter.inc etc. is not a good idea.
I remember old pfSense versions had an option "Start FTP Helper" for each interface.
Is it possible to implement such option?
Many thanks
Frank
Hi faunsen,
It's a transparent proxy, but since FTP relies on out-of-band data connections through port 20, it makes it almost impossible to run a full proxy setup.
ftp-proxy works, but our old implementation had sub-optimal solution so it was removed in the hopes of squid being able to replace it. Since it doesn't, I think ftp-proxy is back on the table for 17.1.
This time, we should be able to avoid all potential loopholes, but it requires a more solid integration / configuration than a single checkbox.
I'll write make this a priority after 16.7 is officially out. How does that sound?
Help in getting that done sooner is always appreciated of course. :)
Cheers,
Franco
Quote from: franco on July 05, 2016, 12:42:13 PM
I'll write make this a priority after 16.7 is officially out. How does that sound?
GREAT!!
Quote from: franco on July 05, 2016, 12:42:13 PM
Help in getting that done sooner is always appreciated of course. :)
Of course 8)
Tell me how.
Design, programming, testing, documentation... ?
Nice, ok, I've added a new feature ticket here: https://github.com/opnsense/core/issues/1051
Previous problems: listens on all interfaces/opens holes, service start/restart in filter.inc was suboptimal.
The questions posed are important to avoid security risks: would a separate service page be viable with a bit of configuring, mostly which interfaces are allowed to use the proxy? It would give the FTP proxy a more visible feeling, but I'm not entirely sure it's necessary.
What do you think?
Quote from: franco on July 06, 2016, 10:19:45 AM
The questions posed are important to avoid security risks: would a separate service page be viable with a bit of configuring, mostly which interfaces are allowed to use the proxy? It would give the FTP proxy a more visible feeling, but I'm not entirely sure it's necessary.
What do you think?
No, I think a transparent FTP proxy should be entirely transparent.
See my comment on GitHub.
For a non-transparent proxy I would stay with squid.
To answer your questions on github:
- Can / should it be a plugin?
I think no, because it's not possible to add the needed anchors dynamically. They are hard coded in filter.inc. - Should we enable this per interface?
Also no, because ftp-proxy binds to an IP address not to an interface. And this is more flexible. Imagine you want ftp traffic to a server with an origin address other than to a second server but over the same interface.
To avoid security risks the firewall admin must add appropriate access and redirect rules.
Should it have its own service page?
Yes. Define as much ftp proxies here as you need.
What configuration parameters do we need?
Basically the parameters from ftp-proxy.
mandatory parameters:
- the source address for the control connection (-a 1.2.3.4)
- the port where the proxy will listen for redirected connections (-p 8021)
optional parameters:
- an option to rewrite the source port to 20 in active mode (-r)
- debug level (-D 5)
- set the log flag on pf rules (-v)
- fixed server address for reverse mode (-R 192.168.1.21)
- idle timeout for the control connection (-t 86400)
- queued connections (-q queue)
If no one else writes it, I would.
But the 'ftp-proxy/*' anchors are mandatory. Peanuts for the core team ;)
For the nontransparent proxy
ftp/jftpgw could be useful.
The UI could look like the attached images.
What do you think?
Hello,
I've written a ftpproxy plugin that manages ftp-proxy processes.
It enables ftp through the firewall and access to ftp servers behind the firewall (reverse mode).
The ftp transfer is completely transparent.
Everything works fine but I miss the hook to synchronize the config in a cluster.
Is it possible to sync a specific config section and do some remote configd actions from a plugin?
Kind regards
Frank
Hi Frank,
Sorry, I got too distracted and forgot about this. You did great work!
HA synchronisation isn't pluggable, but we can change that as we have for all types of subsystems that plugins require. We just need the use case to model the plugin-code around.
Do you still require the pf anchor?
Does the plugin have any ports dependencies not currently in OPNsense?
If you want we can pull this into the plugins as a development version?
Cheers,
Franco
Hi Franco,
great to hear from you. Was quite lost with my monologue ;)
Quote from: franco on September 22, 2016, 06:16:32 PM
HA synchronisation isn't pluggable, but we can change that as we have for all types of subsystems that plugins require. We just need the use case to model the plugin-code around.
The use case is to have the proxy configuration at the backup node and start/stop it as we start/stop it on the master node.
The plugin should set, lets say, a "hasync" attribute to the config option that has to be synchronized and the core system handles the synchronization.
Same with remote actions. E.g. a ConfigdActionsField could have the "hasync" attribute and will be executed on the backup node whenever an action on the master was executed.
Quote from: franco on September 22, 2016, 06:16:32 PM
Do you still require the pf anchor?
Yes, the ftp-proxy depends on it. Please, please add it to the core system.
The plugin contains a patch for
/usr/local/etc/inc/filter.inc in the
+POST_INSTALL and
+POST_DEINSTALL scripts.
This is not optimal.
Quote from: franco on September 22, 2016, 06:16:32 PM
Does the plugin have any ports dependencies not currently in OPNsense?
No.
But since I know how easy it is to write a plugin I plan to write another plugin for Monit https://mmonit.com/monit/ (https://mmonit.com/monit/).
There are many services on the firewall simply unmonitored. The admin never gets informed if a service goes crazy.
This plugin would need the sysutils/monit package.
Quote from: franco on September 22, 2016, 06:16:32 PM
If you want we can pull this into the plugins as a development version?
Yes, that would be nice. The attachment contains the package.
The usage is quite easy.
This is what you would do for a simple setup allowing FTP from LAN to the internet.
- Navigate to "Services -> FTP Proxy Server" and add a ftp-proxy process.
You can leave the defaults for this setup. The help text is from the ftp-proxy man page.
The proxy starts automatically if it is enabled and the background color of the row is green.
If you disable the proxy the process stops and the color changes to yellow.
To start the proxies on boot the plugin creates a /etc/rc.conf.d/ftpproxy file with the corresponding settings. - Next add a "Firewall -> NAT -> Port Forward" rule to redirect traffic from LAN to any on port 21 to localhost port 8021.
Leave the "Filter rule association" untouched to have a automatic "pass" rule.
That's it.
Kind regards
Frank
Arrrgh! Seems that I've overseen a dependency to jftpgw in the Makefile.
Here is a new one.
Sorry.
Yay, thanks! So jftpgw is not needed?
The use case question was rethorical, sorry for the confusion. FTP proxy use case is as good as it gets. :D
For a commit, we do need the plugin directory, not the fixed package. Most of the things can be extracted, but it's probably better to use the ones you wrote.
Need to tweak a few things, make the plugin private (it builds as "ospriv-ftpproxy" so that it's not showing up in the plugins for a release, etc.)
Best would be a pull request for the plugins.git, but I can also bring this in manually.
I like the filter.inc fix, very clever. :)
I will add monit to the binary packages, sure thing.
EDIT: https://github.com/opnsense/tools/commit/d0e52636e5c
No, jftpgw is not needed. It was an early idea to merge ftp-proxy and jftpgw into one plugin.
The pull request is made. I hope I did it right. I'm very new to GitHub.
Thank you for the filter.inc flowers and for the Monit package :)
Regards
Frank
Thanks again to Frank for his work. We merged the code, improving a few things in core.git in order to make the plugin (and others) more flexible in the future and will likely release it in a 16.7.x, but not yet sure which one it'll be.
Cheers,
Franco
Hi there,
I really appreciate the effort faunsen has done with this Plug-In!
I would like to see that Plug-In within the next 16.7.X release.
Cheers,
Sörnt
It was released some minutes ago along with 16.7.8. :)
And I agree, great work by Faunsen.
I heard he is already working on the next one...
Great!
Just update to "16.7.8-amd64" without any issues - wounderfull!
And installed the "os-ftp-proxy" PlugIn.
This is my network setup, I want to get an FTP-Connection from my Workstation to the FTP-Server:
Workstation OPNsense FritzBox (Modem/Router) FTP-Server
*-------------------------* *----------------------* *---------------*
| WAN Fix: 192.168.180.50 | | WAN: Dyn. IP by ISP | | 85.214.41.254 |
*---------------* | LAN : 192.168.1.1 | | LAN: 192.168.180.1 | *---------------*
| 192.168.10.50 | | VLAN10 : 192.168.10.1 | *----------------------*
*---------------* *-------------------------*
I am not sure what I need to enter in the fields for a new FTP-Proxy-Server:
Listen address: 127.0.0.1 (preconfigured)
Source address: ?
Reverse address: ?
Reverse port : 21 (preconfigured)
Can someone help me here please?
Regards,
Sörnt
Hi Sörnt,
simply use the defaults.
And have look at the FTP Proxy Howto (https://forum.opnsense.org/index.php?topic=3868.0) ;)
Kind regards
Frank
Hi Frank,
no, leaving the defaults and setting up the NAT Portforwarding as in your How-To, is not working for me.
I guess, the FritzBox may be the problem.
I just configured at the FritzBox the so called "Exposed Host" to target the OPNsense Box (192.180.50).
The FritzBox will forward all incomming traffic to the OPNsense box. That didn't help either.
This is what the FTP-Client is telling me, if I try to open, list and transfer a file
"The authentification is successfull...." but than later...
.
.
MLSD
PORT failed, try PASV mode!
PASV
TYPE I
299 Type set to I.
PASV
227 Entering Passive Mode (85,214,41,245,245,238)
PORT 192,168,10,10,218,6
2000 Port command succesfull
STOR IMG_4711.JPG <- Try to transfer an jpg - file.
425 Cannot open data connection.
This is what the FTP-Server is telling me:
[TIME] new connection from XXXXX on 85.214.41.254:21
[TIME] hostname resolved : dyndsl-XXXXXX.ewe-ip-backbone.de
[TIME] sending welcome message.
[TIME] 220 Gene6 FTP Server v3.10.0 (Build 2) ready...
[TIME] USER userXYZ
[TIME] userXYZ, 331 Password required for userXYZ.
[TIME] userXYZ, PASS ****
[TIME] userXYZ, logged in as "userXYZ".
[TIME] userXYZ, 230 User userXYZ logged in.
[TIME] userXYZ, SYST
[TIME] userXYZ, 215 UNIX Type: L8
[TIME] userXYZ, FEAT
[TIME] userXYZ, 211-Extensions supported:
[TIME] userXYZ, AUTH TLS
[TIME] userXYZ, CCC
[TIME] userXYZ, CLNT
[TIME] userXYZ, CPSV
[TIME] userXYZ, EPRT
[TIME] userXYZ, EPSV
[TIME] userXYZ, MDTM
[TIME] userXYZ, MFCT
[TIME] userXYZ, MFMT
[TIME] userXYZ, MLST type*;size*;create;modify*;
[TIME] userXYZ, MODE Z
[TIME] userXYZ, PASV
[TIME] userXYZ, PBSZ
[TIME] userXYZ, PROT
[TIME] userXYZ, REST STREAM
[TIME] userXYZ, SIZE
[TIME] userXYZ, SSCN
[TIME] userXYZ, TVFS
[TIME] userXYZ, UTF8
[TIME] userXYZ, XCRC "filename" SP EP
[TIME] userXYZ, XMD5 "filename" SP EP
[TIME] userXYZ, XSHA1 "filename" SP EP
[TIME] userXYZ, 211 End.
[TIME] userXYZ, CLNT Total Commander (UTF-8)
[TIME] userXYZ, 200 Noted.
[TIME] userXYZ, OPTS UTF8 ON
[TIME] userXYZ, 200 UTF8 OPTS ON
[TIME] userXYZ, PWD
[TIME] userXYZ, 257 "/" is current directory.
[TIME] userXYZ, TYPE A
[TIME] userXYZ, 200 Type set to A.
[TIME] userXYZ, MODE Z
[TIME] userXYZ, 200 Mode Z ok.
[TIME] userXYZ, PORT 91,96,35,57,217,235
[TIME] userXYZ, 200 Port command successful.
[TIME] userXYZ, MLSD
[TIME] userXYZ, 425 Cannot open data connection.
[TIME] userXYZ, PASV
[TIME] userXYZ, 227 Entering Passive Mode (85,214,41,254,227,226)
[TIME] userXYZ, STOR IMG_1693.JPG
[TIME] userXYZ, asked to upload '/IMG_4711.JPG' -> 'D:\IMG_4711.JPG' resuming at 0 --> Access allowed.
[TIME] userXYZ, 425 Cannot open data connection.
[TIME] userXYZ, TYPE A
[TIME] userXYZ, 200 Type set to A.
[TIME] userXYZ, PORT 91,96,35,57,218,8
[TIME] userXYZ, 200 Port command successful.
[TIME] userXYZ, MLSD
[TIME] userXYZ, 425 Cannot open data connection.
[TIME] userXYZ, PASV
[TIME] userXYZ, 227 Entering Passive Mode (85,214,41,254,45,46)
[TIME] userXYZ, 421 Connection closed, timed out.
[TIME] userXYZ, disconnected. (00d00:05:01)
If i connect my Workstation via LAN to the FritzBox, I have no issues.
Can you please guide me here to get FTP working?
Kind regards,
Sörnt
The server output looks ok.
Fritz is at 91.96.35.57 and the server at 85.214.41.254.
Ok, we need some more diagnostic here.
Please add a proxy listening on port 8022 (we need it to keep the anchors in the ruleset).
Then stop the proxy on 8021.
Connect the OPNsense Box via ssh and open a shell.
Test the existence of the anchors
# grep ftp-proxy /tmp/rules.debug
The output should look like this
nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"
anchor "ftp-proxy/*"
Then start the 8021 proxy in debug mode
# /usr/sbin/ftp-proxy -b 127.0.0.1 -p 8021 -D 7 -d
Now start a FTP session and post the output of the proxy together with the server log.
Take care of your passwords in the output.
Careful: if no service is running (stopped) the anchors are removed and can't be found like suggested. ;)
Edit: Frank took care of that, sorry. Just to reiterate: a proxy must be running to get an effective reading.
Hi Frank,
It is working fine here now. That was stupid mistake on my side :-[
My workstation is at the VLAN Interface, and I just blindly followed the HowTo. I need to use the VLAN Interface and not the LAN Interface to configure the Port Forwarding *facepalm*
I woundered why I didn't see any output of the debug proxy at the console, I changed the switch ports to the LAN Interface and et voilà I got some output and the FTP-Client was working....
Thank you for your help and this wounderfull Plug-In!
Kind regards,
Sörnt
Cool, again what learned ;)