Messages

Well I have no idea what changed... but flowd is now working.   I haven't changed a thing regarding the configuration.   The server did do a reboot due to a building power failure but when I logged back into the server all of a sudden when I click on Insight I can now see the top usage ports and IP addresses.  I can now see traffic patterns....  Not sure what changed.

The open sense server has a WAN interface with the IP address 10.0.0.1, the LAN Interface has the IP address 192.168.1.240.   You will notice that not only does 10.0.0.1 point to lo0 but so does 192.168.1.240.   I believe this is quite common for the installation.  I re-installed my opnsense server to double check this seeing that I don't assign anything to lo0.   Remember my rogers modem sits in between my ISP and the opnsense serer so the WAN is on the 10.0.0.0/24 network and the WAN interface performs DHCP to get it's interface address.  Once the server was re-isntalled and reconfigured as a NAT this is the setup:

10.0.0.247  --- WAN ->  OPNsense <--   LAN --- 192.168.1.240
                                                        <--    Loop --- 127.0.0.1

Yet after the reinstall I did a dump of the the routing table looks like:

Code Select Expand

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.0.32           UGS         em1
10.0.0.0/24        link#2             U           em1
10.0.0.247         link#2             UHS         lo0
127.0.0.1          link#4             UH          lo0
192.168.1.0/24     link#1             U           em0
192.168.1.240      link#1             UHS         lo0

Normally pinging any workstation interfaces addresses the response comes back.  This  is why I found it odd that pinging the LAN or loopback addresses wouldn't work. Yet pinging the WAN interface address would work.   

I have a few FreeBSD servers where I work and it seems to be quite common to see the IP address of the sever also pointing to the lo0 interface.

Hello lilsense,

Thanks for replying.   I'm assuming I'm having a localized routing issue on the OPNsense server itself.   The symptoms that I'm noticing is that flowd doesn't show any sort of traffic and nptd doesn't seem to connect to any peers.   Normally I would assume that neither have any sort of tie to one-another.

My OPNSense server essentially is a NAT providing 192.168.1/24 outbound.  It sits behind a Rogers modem which creates a 10.0.0.0/8 network in-front of it.   Regaring traffic on the 192.168.1/24 LAN everything seems to work fine.  I can surf, ping resolve no issues.

My OPNSense server is also providing nptd services for my LAN.   The service seems to be running but when I click on status it's not connected to any peers.  This is when I thought I might wan to to ssh into my console and look what ntpq provided as feedback.  This is what I found:

Code Select Expand

# ntpq
ntpq> peers
ntpq: write to localhost failed: Can't assign requested address


So this is when I started to question the loopback.  localhost resolves to 127.0.0.1.   I thought the response was odd but when I attempted to ping 127.0.0.1 which I could do easily on any workstation I got the following response:

Code Select Expand

# ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1): 56 data bytes
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
^C
--- 127.0.0.1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

I looked at the assigned IP address on the LAN interface and attempted to ping it.   

Code Select Expand

# ping 192.168.1.240
PING 192.168.1.240 (192.168.1.240): 56 data bytes
^C
--- 192.168.1.240 ping statistics ---
5 packets transmitted, 0 packets received, 100.0% packet loss

First thing I thought was I must be blocking ICMP on the firewall even though I have a pass all on both the LAN and Loopback interface.   I went to Firewall -> Log Files -> Live View.  Selected interface "LAN" and filtered on blocked and nothing.  Yet if I'm on any workstation within the LAN I can ping 192.168.1.240 no issues.  Any suggestions?

Hi all,

I have OPNsense up and running and my lan is sending traffic out no issues.  NAT'ing is working and all.   Yet I've noticed a few things on the opnsense I couldn't get working regarding ntpd peers, or flowd...

Today I decided to ssh into the console of my opnsense sever and poke about.  This is when I discovered I cannot ping the LAN or Loopback interface IP addresses on the OPNsense server.   Let's be clear here... My configuration is quite simple

--- WAN --- 10.0.0.1 ---> OPENSense sever <------ LAN --- 192.168.1.240
                                                                      <------- Loopback --- 127.0.0.1

If I ping 127.0.0.1 on the Opensense server I get:

Code Select Expand

root@opnsen:/var/log/ntpd # ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1): 56 data bytes
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
ping: sendto: Can't assign requested address
^C
--- 127.0.0.1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

If I ping the LAN interface it simply hangs until I control-C:

Code Select Expand

root@opnsen:/var/log/ntpd # ping 192.168.1.240
PING 192.168.1.240 (192.168.1.240): 56 data bytes
^C
--- 192.168.1.240 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss


There is no firewall rule issues here was in live view on selecting either the LAN interface or the loopback interface nothing shows anything being blocked.   Just to show that ICMP is not blocked from pinging the lan interface from the LAN itself:

Code Select Expand

workstation ~ $ ping 192.168.1.240
PING 192.168.1.240 (192.168.1.240) 56(84) bytes of data.
64 bytes from 192.168.1.240: icmp_seq=1 ttl=64 time=0.252 ms
64 bytes from 192.168.1.240: icmp_seq=2 ttl=64 time=0.180 ms
64 bytes from 192.168.1.240: icmp_seq=3 ttl=64 time=0.230 ms
^C
--- 192.168.1.240 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2030ms
rtt min/avg/max/mdev = 0.180/0.220/0.252/0.030 ms

No issues...  Just on the firewall itself I cannot ping the LAN or Looback... I can ping the WAN interface no issues. 

The loopback inteface is configured and it has 127.0.0.1:

Code Select Expand

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Is this a routing issue on the firewall itself?   The routing table is:

Code Select Expand

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.0.32           UGS         em1
10.0.0.0/24        link#2             U           em1
10.0.0.1             link#2             UHS         lo0
127.0.0.1          link#4             UH          lo0
192.168.1.0/24     link#1             U           em0
192.168.1.240      link#1             UHS         lo0

I think if I resolve this issue ntpd and netflow may fall into place.

Anyone have a suggestion how to start debugging this... Apparently there doesn't seem to be any blatant errors.  Yet I have a number of systems behind this opnsense box and I've pounded the traffic thinking maybe I'm just not generating enough traffic and yet nothing...  Yet flowd_aggregate is running...  Open for any suggestions.

@muchacha_grande

The trick I found is that ones you have outbound NAT configured, you can enable DHCPv6 and assign IP addresses as if you were given a prefix.

Okay so what does your OPNSense LAN configuration look like?   I'm very curious how you went about mapping the internal IPv6 addresses.  Did you assign a static IPv6 from the modem to your internal LAN interface?

Ok so I'm a bit confused.  If I'm going to use NPTv6 to NAT IPv6.  Essentially it's not really a NAT in the meaning that many internal IPv4 address map to 1 IPv4 global addresses, but that I assign an internal IPv6 block of IP addresses that get re-mapped as they leave the OPNsense server to a Valid Global address.

See attached graphic... (tried to include it in-line but had some difficulties having it render)

The modem is infront of my opensense server.   It provides the WAN interface on the OPNSense server an IPv6 address.  Thus any traffic leaving the OPNsense server itself the modem knows how to route it back to that opensense server interface.  The only way I would be able to have IPv6 traffic translate through the Opensense server would be to provide for every system inside my LAN to map an alias IPv6 global alias on the wan interface, and then map that global IPv6 address to an internal host....  That way the modem knows how to return the traffic back to the wan interface of the OPNsense server.   

In comparison to the IPv4 world you would assign a private network 192.168.X/172.16/10.    What IPv6 block to I use on the LAN side to be translated to the Global addresses on the WAN side?

Am I missing something?

@Maurice

@jnaughto, if there is no other way and you're really desperate, IPv6 outbound NAT should work...

I didn't really think there was such a thing as IPv6 and NAT I know this tends to be quite the viral topic.   

Essentially the modem is a Rogers Ignite Modem.  The modem has very little documentation to work with and the interface is quite simplified.  The modems LAN interface can be either configured for stateless ( SLACC  ) or stateful ( dhcpdv6 ) IPv6 deployment.  Yet the prefix is fixed to a /64.   The modem doesn't support any sort of static routing which I could subnet the IPv6 block either so yes routing could be a bit of an issue.

The reason why I've hesitated putting the modem into bridge mode, is that I have 2 ignite TV boxes also behind it.   The idea was to leave them outside of my LAN between the Router and OPNsense router.   I wasn't sure what was going on in these boxes.   I believe they are using both IPv6 and IPv4.  I also wasn't sure what exposure these boxes could be leveraged against the rest of my LAN.   It also helps when there's a TV related issue when dealing with the Rogers support group not to get into "Well my router is in bridge mode...." conversation.  If it's left as they like it and my internal LAN is invisible but functional that's my goal.   Right now everything is working on the LAN side except for IPv6 so at this point little impact. Yet as IPv6 becomes more predominant I'm going to have vast sections of the internet non reachable.

I've played with IPv4 NATs without any issues but I'm going to have to read up on the IPv6 outbound NAT.

Ok so I re-installed an Opensense box.   While configuring it I connected up the LAN side and set a gateway address so that I could do updates and such.  While the Opensense server wasn't in use the flowd was showing traffic and revealing the IP deligation that was passing through the opensense box.


Yet once I put the newly configured/installed opensense server in place all flowd information stopped and I no longer have any sort of insight traffic.  I did another update but noticed the update went from 22.7_4 to 22.7.1 but the issue hasn't gone away.  Still looking to see how to fix this issue.

OPNsense 22.7.1-amd64
FreeBSD 13.1-RELEASE
OpenSSL 1.1.1q 5 Jul 2022

Note that the opnsense server is sitting behind a ISP router.  I still haven't gone to bridge mode but didn't think this should have effected the flowd traffic analyzer

Thanks Maurice... I was concerned about that... there's no option on the dhcpv6 server on the modem for prefix delegation...   Yet I know the range of ipv6 addresses on the rogers modem and the ipv6 block is unlikely to  change.  Isn't there any way I could simply setup a ipv6 dhcpd server on the opnsense for the lan side to propagate the IPv6 block that the modem has?  The other idea would be to somehow allow the LAN dhcpdc requests to pass through the opnsense server to the modem that has dhcpv6 setup on it to respond to the requests.  Is that possible?  I assume there would have to be some sort of firewall rule to allow the modem to reach out to the LAN clients.

Tonight I was able to change rogers modem to do stateless IPv6 from DHCP to stateless for local configuration

Gateway > Connection > Local IP Configuration

I then changed the WAN configuration  on my OPNSense to be :

IPv6 Configuration type:  SLAAC
SLACC configuration
Use IPv4 connectivity: unchecked

The LAN card is still set to:

IPv6 Configuration Type: Track Interface

Track IPv6 Interface:
IPv6 Interface:  Wan Inteface
IPv6 Prefix ID: 0
Manual configuration: unchecked

LAN Interface still does not show up nor does any RA under services...

FYI Just so you know my opnsense server does have an IPv6 address, I can from the OPNSense ping with IPv6 address to www.google.ca (2001:4860:4860::8888).

Hello All,

I just realised today that my IPV6 block of ip addresses are not propagating.   Now I have a Rogers Ignite Modem setup which I cannot put into bridge mode.  Dealing with Rogers Guys can be frustrating all as hell at times.   yet behind it I have an OPN Sense box that sits in front of my home LAN.  The OPNsense box is:

OPNsense 22.7_4-amd64
FreeBSD 13.1-RELEASE
OpenSSL 1.1.1q 5 Jul 2022

latest update.  IPv4 is working fine.  My network configuration is essentially:

Internet <->  Rogers Ignite Modem (port forwarding various ports) <- ZONE A-> OPNSense <- ZONE B -> LAN

I can't bridge due to the fact that I may need to deal with the vendor to address a failed ignite TV box so I had them put in front of the OPNSense server.   The Rogers Modem has an IPv6 block and is doing DHCP for both 6 and 4 in Zone A.   My OPNSense server is showing a IPv6 address on the Wan port:

IPv6 link-local   fe80::XXXXX:814d/64
IPv6 address   2607:XXXXX:814d/64
IPv6 gateway   fe80:XXXXX:691f

Yet the LAN port shows nothing for IPv6. 

The LAN interface has been configured:

IPv6 Configuration Type: Track Interface
IPv6 Interface Wan Interface
IPV6 Prefix ID: 0

On the Wan interface configuration for DHCPv6 Client shows:

Configuration Mode: Basic
Request only an IPv6 Prefix: Checked
Send IPv6 prefix hint: checked
Use IPv4 conectivity: unchecked
Use VLAN priority: disalbed

Now I've noticed that some are referring to Router Advertisements showing up in the Services section but I see nothing there that reflects a RA.  My  DHCPv6 simply has Relay and Leases showing under it.  Yet I did notice on my main Dashboard  DHCPv6 Server is down and will not start.

Awesome thanks.  So I assume you have netflow working?   I was wondering if the 0's for the ksocket were the issue.

Can someone please simply open up their OPNsense server hopefully that is running:

OPNsense 22.7_4-amd64
FreeBSD 13.1-RELEASE
OpenSSL 1.1.1q 5 Jul 2022

click on Reporting -> Netflow -> Cache and tell  me if they see something like:

[Flow   Interface   Destinations   Sources   Pkts
ksocket_netflow_em0   netflow_em0   0   0   0
ksocket_netflow_em1   netflow_em1   0   0   0
netflow_em0   em0   86   3   138730
netflow_em1   em1   76   78   437655


You may have different network interfaces.  I'm using Intel 1 gig cards (em0 and em1) but I'm curious about the kscoket_netflow_em0 showing 0 for everything.   I'm not sure but I believe this maybe why I cannot see anything in Insight as there seems to be a disconnection from the actual em0 which does show packets collected to ksocket_netflow_em0