HOWTO: Install ICAPrb::Server on OPNsense

[fabian]

Hi all,

Tutorial is on my own website: https://fabian-franz.eu/icaprb/opnsense/2016/11/23/install-icaprb-on-opnsense.html

It can be used with the web proxy server if you want to add some custom features for scanning or content manipulation.

Kind regards

Fabian

[paramedic233]

Fabian,
First post.
I tried installing ruby22-gems following your tutorial, and is what the terminal said:
root@OPNsense:~ # pkg install ruby22-gems
Updating FreeBSD repository catalogue...
Fetching meta.txz: 100%    944 B   0.9kB/s    00:01   
Fetching packagesite.txz: 100%    6 MiB   1.5MB/s    00:04   
Processing entries: 100%
FreeBSD repository update completed. 25860 packages processed.
Updating OPNsense repository catalogue...
OPNsense repository is up-to-date.
New version of pkg detected; it needs to be installed first.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
   pkg: 1.9.3_3 -> 1.9.4_1 [FreeBSD]

Number of packages to be upgraded: 1

3 MiB to be downloaded.

Proceed with this action? [y/N]: N

So I thought I had read in the tutorial it was updated to 23, so I tried that, and I received the same error.
I am on OPNsense 17.1.r1 (amd64/OpenSSL)
Ideas?
Thank you.
Mike

[fabian]

This happens because the version of pkg from FreeBSD is newer. You may have to wait for the next OPNsense release. If you install this version of pkg it should work too, but you risk to break your OPNsense installation (pkg always needs to be upgraded before packages can be installed as you can see on regular updates too).

Kind regards

Fabian

[franco]

I don't recommend enabling the FreeBSD repository for this very reason as it gladly takes any newer package from a foreign repository even though it knows the pkg is installed from the OPNsense repository.

In the meantime....

# pkg lock -y pkg
( install stuff )
# pkg unlock -y pkg


Cheers,
Franco

[paramedic233]

Franco and Fabian,
I thank you for your replies.
I did lock the package, and after a bit of finaggling, install icaprb-server, after I figured out I had to use ruby gem install. I then re-locked the package so I could fudge the the system.
Still can not get it to start and run, but that is a project to work on another day.
Again, thank you.
Mike

[paramedic233]

Fabian,
I'm still running into issues with ICAP.
1: No idea how to start ICAP. I tried using a shell, going to /usr/local/lib/ruby/gems/2.3/gems/icaprb-server-0.0.2/bin. and using start_server.rb. That didn't work. root@OPNsense:/usr/local/lib/ruby/gems/2.3/gems/icaprb-server-0.0.2/bin # ruby start_server.rb
/usr/local/lib/ruby/2.3/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- bundler/setup (LoadError)
   from /usr/local/lib/ruby/2.3/rubygems/core_ext/kernel_require.rb:55:in `require'
   from start_server.rb:2:in `<main>'
2: What will the listen address become? The webpage uses Request Modify URL   icap://192.168.2.1:1344/avscanreq and Response Modify URL icap://191.168.2.1:1344/avscan.
However your website has it listed as icap://localhost:1344/echo
So which one is it? If it looks like yours, will it be the same thing listed twice?
I greatly appreciate the help you and the remainder of the forum members have posted.
Thank you,
Mike

[fabian]

1: No idea how to start ICAP. I tried using a shell, going to /usr/local/lib/ruby/gems/2.3/gems/icaprb-server-0.0.2/bin. and using start_server.rb. That didn't work. root@OPNsense:/usr/local/lib/ruby/gems/2.3/gems/icaprb-server-0.0.2/bin # ruby start_server.rb
/usr/local/lib/ruby/2.3/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- bundler/setup (LoadError)
   from /usr/local/lib/ruby/2.3/rubygems/core_ext/kernel_require.rb:55:in `require'
   from start_server.rb:2:in `<main>'

If you want to run it this way, you need to have the gem "bundler" (a dependency management tool) installed.
The script start_server.rb is written for debugging and NOT for production use.

2: What will the listen address become? The webpage uses Request Modify URL   icap://192.168.2.1:1344/avscanreq and Response Modify URL icap://191.168.2.1:1344/avscan.
However your website has it listed as icap://localhost:1344/echo
So which one is it? If it looks like yours, will it be the same thing listed twice?
I greatly appreciate the help you and the remainder of the forum members have posted.
Thank you,
Mike

ICAPrb does not include an anti malware solution, it is a framework which allows you to write your own. The ICAPrb server uses the services hash for mapping between URLs and ICAP service classes.
For example

Code: [Select]

#https://github.com/fabianfrz/ICAPrb-Server/blob/master/bin/start_server.rb#L34
s.services['echo'] = Services::EchoService.new
will attach an instance of the EchoService class as service "icap://host/echo". You could also name the service avscan etc.

[dredhorse]

just to chime in... I also stumbled upon this.

While the HowTo is nice I guess 99% of people except it to not only cover how to install ICAPrb but also an AntiVirus solution. And I know that those are two different things... now.

I liked that OPNsense could do virus scanning till I found out that the requirement is an external solution. On the other side with more and more sites going https and not running an internal smtp server I wonder if the effort is worth it if I have local anti virus installed.

[fabian]

I liked that OPNsense could do virus scanning till I found out that the requirement is an external solution. On the other side with more and more sites going https and not running an internal smtp server I wonder if the effort is worth it if I have local anti virus installed.

If you want an existing av engine scanning your traffic which is open source, you could use clamav as an ICAP service (see http://squidclamav.darold.net/)

Many people run OPNsense on a hypervisor where it should not be a problem to install a linux machine with c-icap and clamav which can be used by OPNsense. If you install and maintain it, you should be able to run those modules also on OPNsense (but I would not recommend it, because it can produce high cpu load, which is the reason why it should be external).

[Fariz]

If I like to do something related to content filtering or adaptation, i can use your icap server right?
but can u explain about how can i do with ICAP server. I understand the function but maybe you can explain more detail ?

[fabian]

ICAPrb is an ICAP server (probably a full implementation) but the logic is inside a service. ICAPrb gives you anything you need for the communication to squid, but the content filter or whatever needs to be implemented by yourself. The EchoService is an example - You can copy it and try to adopt it (for example add content before </body> by replacing it via a call like this one:

Code: [Select]

variable_which_holds_the_body.gsub!('</body>','ICAP Modified!</body>')
Note that you also have to adjust the content length http header. This is usually easy (string in Ruby has a length method).
For Example:

Code: [Select]

http_resp_header['Content-Length'] = http_resp_body.length.to_s
Blocking stuff is replacing the entire page instead of a single tag.

[deodion]

just to chime in... I also stumbled upon this.

While the HowTo is nice I guess 99% of people except it to not only cover how to install ICAPrb but also an AntiVirus solution. And I know that those are two different things... now.

I liked that OPNsense could do virus scanning till I found out that the requirement is an external solution. On the other side with more and more sites going https and not running an internal smtp server I wonder if the effort is worth it if I have local anti virus installed.

Just realized this also, i thought OPNsense already included with it,

Please anyone, if you can provide best alternative solutions (open source and commercial),

Thank you very much,

[mimugmail]

Clamav and c-icap will be in master in a few days.
To get into stable it will take some time, perhaps 17.7.2 or 17.7.3?

[cdburgess75]

good stuff!