Possible flowd solution

[jnaughto]

Hi all,

I have OPNsense up and running and my lan is sending traffic out no issues.  NAT'ing is working and all.   Yet I've noticed a few things on the opnsense I couldn't get working regarding ntpd peers, or flowd...

Today I decided to ssh into the console of my opnsense sever and poke about.  This is when I discovered I cannot ping the LAN or Loopback interface IP addresses on the OPNsense server.   Let's be clear here... My configuration is quite simple

--- WAN --- 10.0.0.1 ---> OPENSense sever <------ LAN --- 192.168.1.240
                                                                      <------- Loopback --- 127.0.0.1

If I ping 127.0.0.1 on the Opensense server I get:

Code: [Select]

root@opnsen:/var/log/ntpd # ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1): 56 data bytes
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
^C
--- 127.0.0.1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
If I ping the LAN interface it simply hangs until I control-C:

Code: [Select]

root@opnsen:/var/log/ntpd # ping 192.168.1.240
PING 192.168.1.240 (192.168.1.240): 56 data bytes
^C
--- 192.168.1.240 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss

There is no firewall rule issues here was in live view on selecting either the LAN interface or the loopback interface nothing shows anything being blocked.   Just to show that ICMP is not blocked from pinging the lan interface from the LAN itself:

Code: [Select]

workstation ~ $ ping 192.168.1.240
PING 192.168.1.240 (192.168.1.240) 56(84) bytes of data.
64 bytes from 192.168.1.240: icmp_seq=1 ttl=64 time=0.252 ms
64 bytes from 192.168.1.240: icmp_seq=2 ttl=64 time=0.180 ms
64 bytes from 192.168.1.240: icmp_seq=3 ttl=64 time=0.230 ms
^C
--- 192.168.1.240 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2030ms
rtt min/avg/max/mdev = 0.180/0.220/0.252/0.030 ms
No issues...  Just on the firewall itself I cannot ping the LAN or Looback... I can ping the WAN interface no issues. 

The loopback inteface is configured and it has 127.0.0.1:

Code: [Select]

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Is this a routing issue on the firewall itself?   The routing table is:

Code: [Select]

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.0.32           UGS         em1
10.0.0.0/24        link#2             U           em1
10.0.0.1             link#2             UHS         lo0
127.0.0.1          link#4             UH          lo0
192.168.1.0/24     link#1             U           em0
192.168.1.240      link#1             UHS         lo0
I think if I resolve this issue ntpd and netflow may fall into place.

[lilsense]

It seems that you misunderstand loopback. Let's forget that for a moment and try to explain what you are attempting to do with NTPd and Flowd. How are they configured and what does not work with ntp/flow.

[jnaughto]

Hello lilsense,

Thanks for replying.   I'm assuming I'm having a localized routing issue on the OPNsense server itself.   The symptoms that I'm noticing is that flowd doesn't show any sort of traffic and nptd doesn't seem to connect to any peers.   Normally I would assume that neither have any sort of tie to one-another.

My OPNSense server essentially is a NAT providing 192.168.1/24 outbound.  It sits behind a Rogers modem which creates a 10.0.0.0/8 network in-front of it.   Regaring traffic on the 192.168.1/24 LAN everything seems to work fine.  I can surf, ping resolve no issues.

My OPNSense server is also providing nptd services for my LAN.   The service seems to be running but when I click on status it's not connected to any peers.  This is when I thought I might wan to to ssh into my console and look what ntpq provided as feedback.  This is what I found:

Code: [Select]

# ntpq
ntpq> peers
ntpq: write to localhost failed: Can't assign requested address

So this is when I started to question the loopback.  localhost resolves to 127.0.0.1.   I thought the response was odd but when I attempted to ping 127.0.0.1 which I could do easily on any workstation I got the following response:

Code: [Select]

# ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1): 56 data bytes
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
^C
--- 127.0.0.1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
I looked at the assigned IP address on the LAN interface and attempted to ping it.   

Code: [Select]

# ping 192.168.1.240
PING 192.168.1.240 (192.168.1.240): 56 data bytes
^C
--- 192.168.1.240 ping statistics ---
5 packets transmitted, 0 packets received, 100.0% packet loss
First thing I thought was I must be blocking ICMP on the firewall even though I have a pass all on both the LAN and Loopback interface.   I went to Firewall -> Log Files -> Live View.  Selected interface "LAN" and filtered on blocked and nothing.  Yet if I'm on any workstation within the LAN I can ping 192.168.1.240 no issues.  Any suggestions?

[lilsense]

why is 10.0.0.1 pointing on the lo0?

[jnaughto]

The open sense server has a WAN interface with the IP address 10.0.0.1, the LAN Interface has the IP address 192.168.1.240.   You will notice that not only does 10.0.0.1 point to lo0 but so does 192.168.1.240.   I believe this is quite common for the installation.  I re-installed my opnsense server to double check this seeing that I don't assign anything to lo0.   Remember my rogers modem sits in between my ISP and the opnsense serer so the WAN is on the 10.0.0.0/24 network and the WAN interface performs DHCP to get it's interface address.  Once the server was re-isntalled and reconfigured as a NAT this is the setup:

10.0.0.247  --- WAN ->  OPNsense <--   LAN --- 192.168.1.240
                                                        <--    Loop --- 127.0.0.1

Yet after the reinstall I did a dump of the the routing table looks like:

Code: [Select]

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.0.32           UGS         em1
10.0.0.0/24        link#2             U           em1
10.0.0.247         link#2             UHS         lo0
127.0.0.1          link#4             UH          lo0
192.168.1.0/24     link#1             U           em0
192.168.1.240      link#1             UHS         lo0
Normally pinging any workstation interfaces addresses the response comes back.  This  is why I found it odd that pinging the LAN or loopback addresses wouldn't work. Yet pinging the WAN interface address would work.   

I have a few FreeBSD servers where I work and it seems to be quite common to see the IP address of the sever also pointing to the lo0 interface.

[jnaughto]

Well I have no idea what changed... but flowd is now working.   I haven't changed a thing regarding the configuration.   The server did do a reboot due to a building power failure but when I logged back into the server all of a sudden when I click on Insight I can now see the top usage ports and IP addresses.  I can now see traffic patterns....  Not sure what changed.