Messages

Well, the wan interfaces of the 2 routers will not talk to each other directly, despite their public IPs being on the same IP segment.

I have a thought about the situation, and after noticing that the correct routing rules and gateways are in place.

I thought that this has to be happening at ARP level, I search around and I found out about ARP-spoofing.

So I traced traffic in the public segment and there it is clear like crystal.

My own fiber provider is spoofing my ARP traffic, I guess in an effort to stop neighbors from attacking each other or something similar.

I wonder if there is an static-ARP feature in opnsense to freeze at least our main public IPS.

Hi!

We have 2 routers in the same /27 public subnet. They deal with different things: public services / workplace internet access.

Both have static IP addresses in the same subnet.

For some reason our opnsense router will not just sent traffic to the other one directly, but instead is routed across 3 hops. And the hops are not even the default gateway of the interface.

I have absolutely no idea what is going on.

Static IP, netmask and default gateway are all OK setup.

The generated routes seem also to be perfect: there is a route for the public network/27  with link#2 as gateway with is the WAN ethernet.

the upstream link is fiber of significant lower rate.

I was wondering if there may be some automatic gateway protocol (BGP?) or something, tweaking my setup without my understanding.

Any idea about what could be going on?

I now found it.

It is related to Gateway monitoring. Our gateway often reports packetloss and even outage. It is strange because traffic not directed at it will perform better.

Gateway monitoring is also used to replaced the default Gateway by another, probably to implement backup links, etc.

There is an advanced Firewall option that is called "Kill States", which kills all states when a gateway is determined to be down.

That is what is killing my TCP sessions.

That option can be disabled in the Firewall, but it is also possible to disable gateway monitoring for the given gateway.

Anyway I am also looking to why is our default gateway lazy replying to ICMP anyway, as I would like to implement a 4G backup line too.

I had have a few SSH sessions opened for hours now, over the OpenVPN, and its just so nice to see it working fine.

HI!

I have a firewall with 4 gigabit interfaces, I was wondering if it would be possible to:

- Setup Routing firewall between LAN and WAN ports (1,2)
- Setup a Transparent Bridge firewall on (3,4)

The traffic on both firewalls is not related:

The routing firewall is going to be used for "management/ops" traffic.
The bridge is for production traffic (cloud platform) traffic, which access the internet on its own lan.

Would this kind of setup work?

arrrhhh, the problem persists.

It is true that it is much worse under packet load but the problem is still there.

I will share my attempts to resolve in case someone else is interested or can help:

My rule to allow the SSH traffic in on the OpenVPN interface. I notice that once the SSH session is closed "default deny" logs on LAN interface are logged.

So, I added the rule to the LAN, and also as a floating rule on OpenVPN + LAN interfaces. And did not help.

Then I set up the following ssh client option:

Host *
  ServerAliveInterval 45

It does not fix the issue but at least the client SSH side is closed with "Timeout Server not Responding", which is better than just freezing the terminal. (in particular if one does not know the escape sequece to kill the ssh session, which is ENTER ~~ .)

Not sure where to continue looking for solutions....

After looking in depth into this issue, looks like my network was undergoing an attack.

After banning the offending IP ranges the problem went away.

I could not notice as the firewall seemed in normal state, for example: no traffic or cpu peaks, yet, connections were hanging.

HI!

I am having problems with my OpenVPN tunnel, TCP connections are getting stuck.

I use SSH over the tunnel and sessions will become unresponsive.

However, ICMP during the session works flawlessly.

I know that OPNSense is to blame because I suffer the problem with different frequency/intensity. When the issue is very frequent I can restart my OPNSense and the problem goes away for some time.

I have no clue what this could be about. It started to happen about a  week ago, and it is very persistent.

I also don't understand how comes that TCP cannot recover itself, at the end of the day this is the protocol that it is supposed to handle network issues to provide a stable session. As said before  by looking at ICMP traffic you could not notice that any problem is going on.

I have now clue how to debug this, or find out which part of my setup is to blame.

Any idea what could be going on? how to debug it?

Rafael

Hi All, @fpieters

I have been runnig this automation for some time. and the result is very satisfying.

Using ansible to compile an XMl cofiguration is a workable strategy while the REST API matures.

OPNSense is mature in the wasy that the configuration file describes pretty well the desired configuration state.

After having run this for some time, I am planing on a next major version that overcomes some issues found, and will improve it a bit further.

https://github.com/naturalis/ansible-opnsense/issues/19

I am also trying to implement Continiuous Integration on Gitlab so that I can test this roles against new updates in OPNSense. H
ere I need to use packer to generate appliances or something.

Thanks, yes, I can see packer script for building a Vagrant image.

That should do.

Yes, naturalis already provided a whole project as documentation.

Also, it allows for creating a virtual networking environment with vagrant for testing.

As for my pull requests, I have added documentation on each of them, with samples.

I have also deployed the role in a lab firewall with most of this functionalities in production.

Hi!

I have started to work on  Ansible support and I would like to implement some kind of continuous integration.

The obvious would be to test test playbooks against code changes and new versions of Ansible and OPNSense, to ensure that nothing breaks, etc.

I could start OPNSense with Vagrant, and try to run unit tests against it.

Is there a Virtual Appliance distribution? I think I saw it somewhere.

Hi!

I have added support to this ansible role for: bridges, static dhcp maps, CAs, certificates, auth servers, tunnables, port forwarding and VPN. Made Aliases compatible with the new implementation.

This on top of the provided support of all the original components provided: interfaces, users, groups, vlans, routes, etc.

The role is far form perfect but a good start. It is easy to extend with support for new objects and there is support for most already.

ok, good to know alias are supported too.

meanwhile naturalis is merging my pull requests, so the legacy ansible way of doing is progressing.

OK, I forked the Naturalis repo here: https://github.com/privazio/ansible-opnsense/tree/dev

I manage add support for bridges interface quite quickly.

I think for the current status of OPNSense, for short term solution, this is definitely a good way to go. I am going to be using it for my current project and will add what I need.

I have also reviewed the Ansible documentation and the Networking Development documentation, thinking on something more long term.

Looks like using the HTTP REST API to implement a Connection plugin as documented:

https://docs.ansible.com/ansible/latest/network/dev_guide/developing_plugins_network.html#developing-plugins-network

... seems to be the way to go. Then the rest of the feature modules use this one for basic connectivity. It seems to be the recommended architecture or at least the only one documented.

With the work already made by @mj84 that should be much easier to achieve.

Since the REST API is fairly generic, it should be possible to create a base class so that the individual functionalities use so that they are easy to implement, i.e. each module could create Dictionary that can be exported to JSON, and the base module knows how to turn it into API calls.... etc.

I will try to put together a prototype. One of these weekends....

it would be good to know if HTTP Rest API is coming to the basic modules (interfaces, aliases, fw rules, etc) anytime soon/or if this is something easy to add.

@mimugmail there is also naturalis opensense ansible role

It is also a XML file composition, config file upload. I don't now how did I miss it when looking at the available work.

I have also been reviewing the Ansible-NETCONF module that there is and the supported transfer is SSH only.

Other network modules use the REST api directly in line with the OPNSense strategy.

I am going to test the naturalis module. which seems pretty complete. Perhaps this is the way to go until the REST api is more complete.