Topics

Hello All,

We've discovered a strange issue related to connecting to one or two (only) upstream web servers. We're running OPNsense 21.7 (details attached) and it's working very well aside from this one thing that's come to light. We have a forward proxy configured to inspect full HTTP plus SNI only for HTTPS, which seems to be working fine, including banking sites etc as far as I've heard.

But a few users reported very slow connections (timeout or 30 seconds+) to one or two specific web sites, and I've been investigating, and I've managed to reproduce what I think is the problem for one particular instance from either my local bash (linux) or from the shell on the OPNsense firewall. Here's a openssl dialog from my local console:

Code Select Expand

$ openssl s_client -connect www.reddit.com:443 -prexit
CONNECTED(00000005)
140277959643584:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:332:


'Connected' is returned fast and then after about 30 secs the error is reported.
Similarly, from the OPNsense shell I get:

Code Select Expand

# openssl s_client -connect www.reddit.com:443 -prexit
1648418660352:error:0200203C:system library:connect:Operation timed out:/usr/src/crypto/openssl/crypto/bio/b_sock2.c:110:
1648418660352:error:2008A067:BIO routines:BIO_connect:connect error:/usr/src/crypto/openssl/crypto/bio/b_sock2.c:111:
1648418660352:error:0200203C:system library:connect:Operation timed out:/usr/src/crypto/openssl/crypto/bio/b_sock2.c:110:
1648418660352:error:2008A067:BIO routines:BIO_connect:connect error:/usr/src/crypto/openssl/crypto/bio/b_sock2.c:111:
connect:errno=60

The error is again reported after a long delay.

This is only perhaps 30% of the requests, and this is the one and only upstream server I have managed to duplicate it for. The rest of the time the openssl handshake returns normally and fast, and lists the certificate, same as it does for all other servers I've tried.
I cannot find anything getting blocked at the firewall that might relate to this.
I've also tried from our infrastructure outside this firewall (AWS) and haven't been able to reproduce the problem.

For now the user workaround is to hit reload a few times, and eventually a request will go through normally.

I can't find much by googling the specific error (and only one other reference in the OPNsense forums), and most of the search hits seem to relate to upstream server misconfiguration, which seems unlikely in this case.
This is not a 'panic stations' issue, the workaround... works, but it does seem very odd, and I'm interested to try and solve it.

Any ideas welcome, thanks!
cwynd

[out]

Hello all,

I'm trying to get a relatively simple OPNsense 21.7.3_3-amd64 set up to the point where I can deploy it.
Brief summary:
* new bare metal hardware, 1 WAN 1 LAN and 1 OPT, one spare RJ45 port.
* One (1) device connected to each of LAN and OPT1 ports by a single piece of wire (no routers and before you ask several different cables tried).

The new setup is intended to clone & upgrade an existing production setup which is OPNsense 18 for a long time, and is aging, and the hardware is slow - hence the attempt to upgrade.
The new LAN is intended for secure business traffic only, and the OPT1 is for more open "retail" type traffic. There are no VLANs involved.

LAN <~> WAN was extensively tested before the attempted deploy and is working fine. However when I tried to deploy during off hours today and replace production OPNsense there is no traffic at all getting passed by OPT1, and I eventually had to roll back to the old system to avoid an extended outage.

On digging subsequently in my test set up I can ping and see packet logs back & forth just fine for LAN, but for OPT1 I can ping out from OPT1 to my test device and get replies, but pinging from the single device to OPT1 never gets a reply, and the firewall logs show

Code Select Expand

action block    dir [in]    src 192.168.129.185    dst 192.168.129.1 <this is OPT1>  - details in the attached screen shot (OPT1 is called DDWRT there for historical reasons).

Everything's been restarted several times.

I've tried putting a OPT1 'allow any from any' rule at the top of the ruleset, but it appears to still get stomped by the Floating 'default deny' and has no effect. Other than that the rules for LAN and OPT1 are substantially identical (the plan had been to spend the day adding all the detailed rules once the basics were working - until we hit this roadblock).

Sorry for long intro, but finally I have three questions:

1) Why is OPT1 behaving differently than LAN interface, with the same rules, i.e. why are the floating rules being applied differently? I'm missing where the asymmetry is coming from.

2) What is the correct way to open anything in the 'Default Deny' rule??

3) Related to #2, I've seen some talk while googling of "weirdness" related to the Floating Default Deny in recent OPNsense releases - should I be thinking about downgrading?

Thanks for any and all advice!

Hello All, we have a 17.1 production OPNsense + ICAP server, all running smoothly - until yesterday.

Yesterday I added the emerging threats rules to suricata (which was already enabled with a limited ruleset), and after a few hours, the system started killing processes, and reporting "out of swap space".

The system is running as a Xen VM with 3G of memory and 3 NICs, and was set up accepting default install options a month or two ago. When I check from the shell

Code Select Expand

swapinfo -h no swap is present. After searching the forums, I am not clear if I should be configuring swap or not.

Would appreciate any advice on what is the right way to configure OS swap.

Thanks!

[but with a high port #, not 4569]

Hi All, we cut over to OPNsense a few days ago, and everything is fine now, with one exception - asterisk IAX2.

Brief background: we have two asterisk servers one inside OPNsense (LAN) and one outside, and we have full control of both ends. There is (should be) an IAX trunk between them, which was working fine for several years with old pfSense box.

For starters I copied over the pfSense rules that were working, which were only:
NAT Port Forward
Interface: WAN
Proto: TCP/UDP
Source: outside-asterisk:4569
Dest: WAN-address:4569
NAT IP: inside-asterisk:4569

-with an autogenerated firewall inbound pass rule to match.

I have tried tweaking / adding to this rule many different ways with no progress. Here's the current 'state of play':

* Outbound calls work fine (inside ~> outside)
* Inbound calls ring until timeout on outside-asterisk with no evidence of any hit on inside-asterisk
* Inside-asterisk shows as registered with outside asterisk but with a high port #, not 4569
* Outside-asterisk shows inside-asterisk registered on same high port#

I am thinking that what is happening is that when a call comes in, outside-asterisk tries to contact inside on that same high port #, and is getting blocked at the firewall, since there is no NAT state (UDP) for it to key into. I dont have the old firewall still in the loop, but if I remember correctly the IAX2 traffic was not getting NATed on the way out, so it was all 4569 end-to-end.

Would really appreciate any hints or advice - or even ideas on what to try next. Thanks!!

[request]

Hi All, I am trying to get squidclamav working with OPNsense 17.1. My problem is I cannot get the redirect to work for eicar test virus files. I have everything working fine for clean urls, and with squidclamav debug enabled I see it receiving and processing the request::

Code Select Expand

DEBUG initializing request data handler.
DEBUG processing preview header.
DEBUG X-Client-IP: 192.168.1.150
DEBUG method GET
DEBUG url http://no.viruses.here
DEBUG URL requested:  http://no.viruses.here

DEBUG Content-Length: -1
DEBUG No body data, allow 204
DEBUG Releasing request data.
DEBUG initializing request data handler.
DEBUG processing preview header.
DEBUG preview data size is 1024
DEBUG X-Client-IP: 192.168.1.150
DEBUG method GET
DEBUG url  http://no.viruses.here
DEBUG URL requested:  http://no.viruses.here

DEBUG Content-Length: 3699
DEBUG Content-Type: text/html
DEBUG End of method squidclamav_check_preview_handler
DEBUG ending request data handler.

Sending zINSTREAM command to clamd.
DEBUG Ok connected to clamd.
DEBUG: Scanning data now
DEBUG Write 3703 bytes on 3699 to socket
DEBUG received from Clamd: stream: OK
DEBUG Closing Clamd connection.
DEBUG Responding with allow 204


But for an eicar test virus url, it seems like the request get's released by squidclamav (if I am reading the below correctly), but a response is never received, and never forwarded to clamd for scanning, and no response or redirect ever goes back to the client browser, which eventually reports a '504 Gateway Error'. This is the complete squidclamav debug log:

Code Select Expand

DEBUG initializing request data handler.
DEBUG processing preview header.
DEBUG X-Client-IP: 192.168.1.150
DEBUG method GET
DEBUG url http://www.eicar.org/download/eicar.com
DEBUG URL requested: http://www.eicar.org/download/eicar.com
DEBUG Content-Length: -1
DEBUG No body data, allow 204
DEBUG Releasing request data.


I have duplicated this problem on two completely separate c-icap/squiidclamav/clamd setups now, most recently with latest c-icap (0.5.2) and squidclamav (6.16)  compiled from sources, so I'm sure it's something I am doing wrong. But after 3 days of poring over logs I cannot see it.

Any help or hints greatly appreciated!!

PS: clwarn.cgi is in the right place, and is visible to the client browser if I load it directly) According to the apache logs it never gets hit unless I load it explicitly.

PPS: This is all HTTP (no HTTPS...  yet)


Edit: Marked solved.

Hi All,

I'm trying to install from the OPNsense-17.1-OpenSSL-vga-amd64.img via USB stick. Everything is fine until the OPNsense first menu, which does not correspond to the instructions: https://opnsense.org/users/get-started/ Step 3

I am seeing this, I see no 'Configure Console', 'Quick/Easy Install' etc:


If I let the default boot continue, I think a live image boots (I hear the little tune) but the console shows 'Booting' (white on blue) followed by 'EFI framebuffer information' and then nothing more. I get the same result on an HDMI output as on the vga panel in the image above.

This same hardware will boot a SystemRescue USB with no issues.

I've used the other *Sense for about 10 years, so I do know a little about what to expect.

What am I missing?