OPNsense Forum

English Forums => General Discussion => Topic started by: foxint on January 09, 2025, 08:05:55 AM

Title: When internet goes down network goes down
Post by: foxint on January 09, 2025, 08:05:55 AM
Hi. I have been using OPNsense for a few months. Please accept my apologies now.

I am total novice and did a basic setup using a You Tube video. I have nothing fancy, I used OPNSense because my modem broke and I live in the Aussie Outback and no one sells modems, so I made one from an old HP PC. (They filmed the latest Mad Max film here and yes there is nothing here. It is a mining town with rock and desert).

I have a LAN comprising of 5 PC's all with static IP addresses. When I am using the internet and it stops and pauses – that it does all the time, I cannot access any of my PCs on the LAN. I use programmes that must connect to the internet to function, and I access this via my workstation on another PC on the network. (This arrangement is the preferred arrangement from the software vendor – MYOB. It is a real princess of a programme and as such I had to buy a completely new PC as it does not like sharing).

I searched - when internet goes down network goes down. I could not find anything. I probably asked the wrong question. Please do not shoot me I am 66years old and need a young bloke to help.

Anyone????

Dan

Title: Re: When internet goes down network goes down
Post by: Patrick M. Hausen on January 09, 2025, 08:08:44 AM
How is your internal network set up? Do you have a switch? What do you mean by "Internet goes down"? Just the external connection and you can still ping OPNsense or is OPNsense crashing (possibly) and unreachable?
Title: Re: When internet goes down network goes down
Post by: blubern on January 09, 2025, 03:12:02 PM
I have the same behavior - with no internet i can't reach any other device internally.
But from the device i can reach opnsense.
what should i monitor to figure out what is the problem?
Title: Re: When internet goes down network goes down
Post by: Patrick M. Hausen on January 09, 2025, 03:15:12 PM
First describe how your devices and OPNsense are connected to each other.
Title: Re: When internet goes down network goes down
Post by: blubern on January 09, 2025, 04:49:03 PM
=> My OPNsense has three interfaces
1. connected to wan (pppoe)
2. connected to a NAS
3. connected to a switch - on the switch are all devices connected inclusive the wlan access points.

when wan goes down i cant reach pc1 over lan from pc2
and when wan comes back it need about 2 min that dns will start to work again
Title: Re: When internet goes down network goes down
Post by: Patrick M. Hausen on January 09, 2025, 05:19:40 PM
pc1 should be able to communicate with pc2 via the switch. That traffic SHOULD not pass through OPßnsense.

Please post your DHCP settings for the interface connected to the switch (LAN?).
Title: Re: When internet goes down network goes down
Post by: blubern on January 09, 2025, 06:24:56 PM
I switched today to the Kea DHCP

Could the issue be that there are some VLANs?
PC1 and PC2 are in different VLANs
Title: Re: When internet goes down network goes down
Post by: pfry on January 09, 2025, 08:07:55 PM
Quote from: blubern on January 09, 2025, 06:24:56 PM[...]
Could the issue be that there are some VLANs?
PC1 and PC2 are in different VLANs

Possibly. VLANs offer optional layer 2 isolation (it depends on your switch configuration). There are endless potential issues with VLAN configs - you'll have to describe your topology in much more detail for folks here to help out. On OPNsense, perhaps copy the Interfaces -> Overview text and edit it appropriately (or simply capture and post it - up to you) as a start. Don't forget switch config/topology details too.

Still, I would expect OPNsense to route whether the PPPoE session is active or not. Your config (and foxint's) should be interesting. Perhaps another VLAN tagged + untagged mix effect I haven't thought of?

[Edit:typos]
Title: Re: When internet goes down network goes down
Post by: foxint on January 09, 2025, 09:45:52 PM

Hi Patrick et al.

Nice to have a friendly post.

Connection: Internet modem to OPNsense box – OPN to switch – all PC's to switch via ethernet.

Internet goes down – I cannot get anything from the internet. The internet indicator (Win 10) on the task bar says no internet.

When I take OPN box out of the system the network operates. When it is in the system and the internet goes down, I cannot access my PC's on the network.

VLAN – not sure what this is and I do not think I have a VLAN.

Dan
 (PS I did not get a notification of the replies????)
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 09, 2025, 11:06:31 PM
@blubern - best if you open your own thread so the question and answers don't get all mixed up. Please.

@foxint - Please tell what are your DNS settings, so to rule them out.
a) Systems | Settings | General.
b) Unbound or dnsmasq - which do you have enabled, share the settings.
c) On your LAN interface settings. Share those too please.
Title: Re: When internet goes down network goes down
Post by: Patrick M. Hausen on January 09, 2025, 11:11:00 PM
Yep - we definitely have two unrelated problems here. With VLANs in place, of course local communication breaks whenever OPNsense breaks for some reason.

Without VLANs and an unmanaged or simple switch, local communication should not be affected by OPNsense failing. Unless e.g. DHCP settings (netmask!) are wrong.

@foxint please also show your DHCP settings.
@pfry please open a separate thread.
Title: Re: When internet goes down network goes down
Post by: foxint on January 10, 2025, 02:14:21 AM
Hi Guys – Thank you. I was just doing some testing without the internet. The OPN box and the network seem to work whether I have the OPN Box connected or not.

Can you give me instructions how to find the DNS and DHCP and on what device.

Thanks

Dan
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 10, 2025, 10:02:36 AM
> Can you give me instructions how to find the DNS and DHCP and on what device.
My post #9. Those are the pages to navigate on your OPN machine's UI to get to them.
Title: Re: When internet goes down network goes down
Post by: Patrick M. Hausen on January 10, 2025, 10:25:24 AM
Quote from: foxint on January 10, 2025, 02:14:21 AMHi Guys – Thank you. I was just doing some testing without the internet. The OPN box and the network seem to work whether I have the OPN Box connected or not.

So the problem is solved? That means we would probably not need to investigate your DHCP settings. I just suspected you might have got the netmask wrong which would explain why PCs cannot talk to each other if OPNsense is down.

Quote from: foxint on January 10, 2025, 02:14:21 AMCan you give me instructions how to find the DNS and DHCP and on what device.

See @cookiemonster's post - the settings you made on OPNsense, of course.
Title: Re: When internet goes down network goes down
Post by: blubern on January 10, 2025, 02:45:05 PM
Hi,

it looks like with the change to Kea the problem is gone!

@foxint sorry that i hijacked your thread
@all thanks for your help and support!
Title: Re: When internet goes down network goes down
Post by: foxint on January 10, 2025, 07:42:08 PM
Hi Guys,

I do not understand.

I still cannot talk with the PC's when the internet is down.

What is my problem?

What do I have to check? The OPN box or my network settings?

How do I find cookiemosters #9 post? What do I need to check?

Thanks
Dan
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 10, 2025, 10:14:23 PM
No problem. Here it is again:
Quote from: cookiemonster on January 09, 2025, 11:06:31 PM@foxint - Please tell what are your DNS settings, so to rule them out.
a) Systems | Settings | General.
b) Unbound or dnsmasq - which do you have enabled, share the settings.
c) On your LAN interface settings. Share those too please.
So for a), you log onto your OPN User Interface and navigate from the left. Click on "System" on the vertical left menu, then that opens on the right when clicked. There you expand still on the left the "Settings" and then again in "General".
We are interested in what settings you have there. Specifically those for DNS.
What I want to see is what DNS settings you have set there.
Then similar for b) you need to tell us which service you have enabled and with which settings.
Finally c).
It is not unusual for DNS settings for the network to be incorrect to cause what you describe. Your network clients are relying on DNS to function but that said, I'm a little unsure now we have other basics cleared.

Can you please explain what you mean you can not talk with the PCs, how do you try? And does is still happen when the internet is down (which I think you said is not a problem anymore?)
Title: Re: When internet goes down network goes down
Post by: foxint on January 10, 2025, 10:56:52 PM
Thanks

DNS servers
1.1.1.1
8.8.8.8

Unbound or dmsmasq – I do not know what these are or where to find them.

Lan interface - I do not know what these are or where to find them.

"Cannot talk with the Pc's". when the internet goes down/stops, I cannot access the other PC's on the network.

The network is OK when I remove the internet. So the problem remains when I re-insert the internet.

Thanks
Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 10, 2025, 11:25:23 PM
Quote from: foxint on January 10, 2025, 10:56:52 PMI cannot access the other PC's on the network
@foxint, what do you mean by this please? Are your other PCs acting as servers, or you mean you cannot use things like mail or social pages between them? How are you "accessing" other PCs?

Does Opnsense show the IP addresses of all machines when you disconnect WAN (/Interfaces/Diagnostics/ARP table in the menus on the left)?
Title: Re: When internet goes down network goes down
Post by: foxint on January 10, 2025, 11:35:22 PM
Hi

Thanks for asking.

I have 1 PC with a programme that I access from my main PC and the other PC's act as storage and back up for my data.

I usually access the PC's using Windows Explorer, Tight VNC (and more recently trying to make Remote desktop to work – but that is a challenge). Also, I have a synchronisation programme that "should" automatically synchronise the data (but that is also having issues).

"Does Opnsense show the IP addresses of all machines when you disconnect WAN (/Interfaces/Diagnostics/ARP table in the menus on the left)?"- I have no idea.

Thanks
Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 11, 2025, 12:05:50 AM
Thank you for your answers. I understand the problem better. I suspect that the DNS trail others are following will prove to be the issue.

Quote from: foxint on January 10, 2025, 11:35:22 PM"Does Opnsense show the IP addresses of all machines when you disconnect WAN (/Interfaces/Diagnostics/ARP table in the menus on the left)?"- I have no idea.
In OPNsense menus, if you follow the trail I gave you (/Interfaces/Diagnostics/ARP table) then you will see a list of recognised devices with their IP addresses (which you can ignore). If the number of devices listed approximates the number of devices you think you have, or you recognise device names, then Opnsense is not blocking their communication just because the internet is down.

To check Unbound or DNSmasq as cookiemonster requested, go to each of the menus /Services/DNSmasq DNS/Settings and /Services/Unbound/General. In each case, there will be a box at the top for "Enable". Is either one ticked?
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 11, 2025, 12:23:54 AM
Quote from: foxint on January 10, 2025, 10:56:52 PMThanks

DNS servers
1.1.1.1
8.8.8.8

Unbound or dmsmasq – I do not know what these are or where to find them.

Lan interface - I do not know what these are or where to find them.

"Cannot talk with the Pc's". when the internet goes down/stops, I cannot access the other PC's on the network.

The network is OK when I remove the internet. So the problem remains when I re-insert the internet.

Thanks
Dan

You are going to have to get down and dirty a little with your new box Dan, no escape. You installed it I presume, so please revise the installation instructions https://docs.opnsense.org/setup.html . You'll get help but go over those basics please. it'll get us faster to the solution.

let's go back a little:
QuoteI have a LAN comprising of 5 PC's all with static IP addresses. When I am using the internet and it stops and pauses – that it does all the time, I cannot access any of my PCs on the LAN. I use programmes that must connect to the internet to function, and I access this via my workstation on another PC on the network. (This arrangement is the preferred arrangement from the software vendor – MYOB. It is a real princess of a programme and as such I had to buy a completely new PC as it does not like sharing).
Static IP addresses could be a problem if you just insert a new network router. Where are the static ip's set, on the devices themselves? Describe it please. How are your machines connected to each other, a switch? where does it connect to OPN? Please describe the layout. We're interested in ports and the setup of those ports. In a router (OPN in this case) they become "Interfaces".

QuoteConnection: Internet modem to OPNsense box – OPN to switch – all PC's to switch via ethernet.

Internet goes down – I cannot get anything from the internet. The internet indicator (Win 10) on the task bar says no internet.

When I take OPN box out of the system the network operates. When it is in the system and the internet goes down, I cannot access my PC's on the network.
This is along the way what we need. Are you sure your modem is not also a router, otherwise you could be double NATing and might be part of the problem. How to check? Well we don't know your hardware so you'll need to find out a bit from your ISP or their forums, or their documentation (online I hope). If you can't tell, give us the url of your ISP, modem model and we'll see if we can ascertain it for you.

When "the internet goes down", could it be that actually your ISP has a problem, or that the modem is going offline? No much use looking in OPN for that. How to tell? Diagnostics by methodical elimination.
That said, when that happens, your machines connected to the switch should also be affected (no internet, so no access/connectivity to the outside) BUT they should be able to connect to each other. But here is the thing. They can only connect to each other by IP address (for simplicity, local name resolution we'll park for now). So you need to be able to diagnose that. Do you know how to connect by IP ? You mention windows explorer and I don't know much Windows but I imagine you can enter the IP in it to the machine you want to connect to. Otherwise a ping command from the command line.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 12:24:35 AM
Hi

I am glad you understand, I do not.

See attached. It has my PC's on the list.

""To check Unbound or DNSmasq as cookiemonster requested, go to each of the menus /Services/DNSmasq DNS/Settings and /Services/Unbound/General. In each case, there will be a box at the top for "Enable". Is either one ticked?

I could only get to "setting" – not sure where Services/Unbound/General is? But on my screen in General enabled is NOT ticked – should be it be ticked?

Thanks
Dan
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 11, 2025, 12:27:34 AM
Quote from: foxint on January 11, 2025, 12:24:35 AMI could only get to "setting" – not sure where Services/Unbound/General is? But on my screen in General enabled is NOT ticked – should be it be ticked?
It's the one called "Unbound DNS". Sorry, I should have been clearer. What's there? You are spot on, is it ticked.
Title: Re: When internet goes down network goes down
Post by: passeri on January 11, 2025, 12:34:54 AM
Quote from: foxint on January 11, 2025, 12:24:35 AM/Services/Unbound/General
From the top level of the menu, Services, Unbound, General.

You have already found DNSmasq under Services, so it is the same routine.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 12:43:21 AM
Quick question - ticked or NOT ticked for them both?
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 12:47:01 AM
Hi Cookiemonster.

Thank you.

I have set up static IP Addresses on each PC on the PC. I have connected the PC's to dumb switches.

The OPN box has a cord to one of the switches.

As far as I am aware the modem only deals with the internet. There is one cord to the wall and the other goes into the OPN box.

I have no hardware from my ISP. The modem box is provided by the NBN (National Broadband Network) people. I am not sure what is should be called we call it the NBN Box. I have fibre to the curb.

I have the NBN box ----- OPN Box ---- Switch.

I have been dealing with the continual dropouts with my ISP. The only thing the ISP refuses to do is test the NBN Box, because the NBN people deny any service failures.

Regardless of who is to blame (we have a very Donald Ducked system here), I get internet dropouts and then my network goes down.

I just tried to enter the IP Address of one of my PC's and it timed out. They all timed out.

Thank you.

Dan
PS – Thanks for the link to the set up (https://docs.opnsense.org/manual/interfaces.html). I do not understand any of this. I got lost at Virtual Machine, Trust Certificates, Change logs 2FA TOTP..... It was hard enough getting the programme onto a USB stick. I appreciate it is a great programme, but I have no idea its full capabilities, or what I should have or what I want.
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 11, 2025, 01:04:52 AM
Can you please show the settings on Interfaces | LAN. We're going to have to focus on device to device connectivity.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 01:10:01 AM
Thanks.

Interfaces/LAN - I think. See attched.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 01:18:23 AM
Photos
Title: Re: When internet goes down network goes down
Post by: passeri on January 11, 2025, 01:36:26 AM
QuoteThe only thing the ISP refuses to do is test the NBN Box, because the NBN people deny any service failures.
The ISP will not, because they do not own it. NBN Co will have verified connectivity to the box. Your problems lie elsewhere and right now it looks odd because all inter-PC communication should be passing through the switch without bothering Opnsense.

When you supplied the ARP table above, was the system connected to the internet at the time? If not, what is displayed after disconnecting?

You have not yet confirmed the answer to whether Unbound DNS is enabled or not. Please see the prior posts.

By the way, given it is your only LAN access, I strongly advise you tick the box "Prevent interface removal" in /Interfaces/LAN, and Save.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 03:29:54 AM
Hi Passeri

My apologies – the internet was down.

When you supplied the ARP table above, was the system connected to the internet at the time? If not, what is displayed after disconnecting? I did not notice.

Unbound DNS is ticked

DNSmasq DNS is NOT ticked

Are these correct??

Thanks for the tip - "Prevent interface removal" in /Interfaces/LAN, and Save. Done.

Is there a solution?

Thanks
Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 11, 2025, 03:52:45 AM
Is there a solution?Undoubtedly, but I have not found it yet :). Fortunately, there are more experienced people also trying. I cannot guarantee success.

Good to know Unbound is on. I believe it is now unlikely the problem is failure of DNS lookups, where I was first going.

Sorry to keep asking you to check things, but my next request is a follow-up to an earlier one from Patrick M Hausen and cookiemonster. If you already know whether you have configured DHCP via ISC or Kea then only one of the following is needed:
Please advise your settings (show screenshots) for /Services/ISC DHCP4/LAN and for /Services/Kea DHCP [new]/Kea DHCP4

While you are there you will notice in each of those subdirectories you can look at Leases which will confirm that you have everything properly addressed. You mentioned static mappings, we need to make sure those are set up and working correctly as well.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 04:40:15 AM
Hi Passeri,

Thank you.

I am bowled over by the kindness and the wealth of knowledge. It is overwhelming. Thank you.

Kea DHCP – no leases.

ISC DHCP – leases look good. A mixture of dynamic and static. The static ones are my PC's

Much appreciated

Thank you.

Dan
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 04:40:52 AM
Last photo
Title: Re: When internet goes down network goes down
Post by: passeri on January 11, 2025, 04:55:26 AM
Thank you for those. Things look good with one important caveat: your static leases are allocated in dynamic pool space. I see that your static leases are in the range 192.168.1.20 - 192.168.1.145 so change your Range, shown in OPN-2025-01-11-04.JPG, to start at 192.168.1.150. Renew leases on any dynamic clients -- this has to be done on the client -- to make sure they are in the new range. All static leases remain as they are.

If we are not nearing a solution now, and no-one else has comments on the configuration, then I would need to go back to look into your problem definition a bit more.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 05:53:23 AM
Hi Passeri,

Thank you.

Just to confirm.

"change your Range, shown in OPN-2025-01-11-04.JPG, to start at 192.168.1.150".

So – 192.168-1-150 to ???? 192.168.1.245?

"Renew leases on any dynamic clients -- this has to be done on the client -- to make sure they are in the new range" – How do I do that?

"All static leases remain as they are." So NO change to the static leases.

Thanks

Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 11, 2025, 06:20:20 AM
foxint, confirming each of your comments.

Renewing a lease depends on the client. I can tell you how to do that on many things that are not Windows. Look for words in your device settings like Network, DHCP, TCP/IP and Renew lease. Some AI thing in Google should have an answer too.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 06:52:13 AM
Hi Passeri,

Thank you.

Just to confirm.

   My static leases are in the range 192.168.04 to 192.168.1.149
   
   My dynamic leases "should be" in the range of 192.168.150 to 192.168.1.245

What will happen if my dynamic leases are Not in the new range - 192.168.150 to 192.168.1.245?

My phone is 192.168.1.150. So it automatically went into the new range.

I am assuming this I OK
Thank you and thanks to all the great guys.

Eventually when I become brave enough, I will do a VPN and a Firewall. Not sure what a Firewall is, but I think I need one

Thanks

Regards
Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 11, 2025, 07:33:55 AM
Quote from: foxint on January 11, 2025, 06:52:13 AMWhat will happen if my dynamic leases are Not in the new range - 192.168.150 to 192.168.1.245?
They will be, after renewal. You have told DHCP not to offer a dynamic lease outside that range. It won't.

If something not renewed has an existing lease in the static area, that is not a problem so long as it is not the same (stealing an existing static assignment). When it auto-renews at some point, it will be reassigned by DHCP into the dynamic pool.

Firewall? You have one. That is Opnsense. It's good.

There are many good options for a VPN if you have a good use for it, and different ways of setting it up for all or selected users, always on or not.

Now we wait to see whether your original problem is resolved or whether we need to pursue a new tack.
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 11, 2025, 10:43:43 PM
thanks for helping passeri. Good spot with the static leases.
@foxint - all good now?
As passeri says once you have re-allocated static leases to your clients on OPN OUTSIDE the dynamic range, which you seem to have done now, those clients will get reallocated when their leases expire UNLESS you have set them on the client themselves.
Explaination: say you have allocated your windows laptop an ip of 192.168.1.150 on the Windows Operating System, then it will not change to the one you allocated on OPN Sense.
Leases normally default to 2 hrs I think on DHCP(ISC) not KEA.
Title: Re: When internet goes down network goes down
Post by: foxint on January 11, 2025, 11:09:46 PM

You have been great, patient and very helpful.

I do not do this all all the time. I do not do it at all. It is interesting...but.

Yes, I set the static IP addresses in each PC and I am hoping they will stick.

Thanks again. I really do not know the full capability of OPNsense, but I hope I can slowly work my way through as I do things I think I need.

Thanks

Dan
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 11, 2025, 11:15:38 PM
No worries.
You need now to change the clients from static IP on them to DHCP. Then you assign them on OPN instead. There is a dance to do but is a one off. Do you know how?
Unless you have no requirement to have the same ip always and then they'll just get the next available from the pool if they reboot and there is contention for it.
Title: Re: When internet goes down network goes down
Post by: foxint on January 12, 2025, 03:12:33 AM
Hi Cookie monster,

Thank you.

"You need now to change the clients from static IP on them to DHCP. Then you assign them on OPN instead".

Why?

I think and that is only me the novice. I need static IP because I synchronise my data each night. (Not 100% happy with my synchronisation programme, but II am working with it)..

Thanks
Dan
Title: Re: When internet goes down network goes down
Post by: foxint on January 12, 2025, 03:58:49 AM
Hi Guys,

I have been moving files from one PC to another. The same problem: when the internet is down the transfer stops.....

I am confused.....

Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 12, 2025, 04:26:43 AM
I said earlier that if the problem was not fixed then we would need to look more closely at the problem definition.

Quote from: foxint on January 12, 2025, 03:58:49 AMproblem: when the internet is down the transfer stops.....

That statement assumes a sequence you have not demonstrated.

Quote from: foxint on January 12, 2025, 03:58:49 AM(Not 100% happy with my synchronisation programme, but I am working with it)..

What is this transfer software?

What if your transfer is breaking and that stops internet? We do not know.

When you are not doing a file transfer, does everything stop when the internet connection does?

How do you know the internet connection stopped prior to breakdown of the file transfer?

When the problem happens, are you able to access Opnsense to view connections and logs?

What steps do you take to fix the apparent internet failure?
Title: Re: When internet goes down network goes down
Post by: foxint on January 12, 2025, 09:26:04 AM
Hi Passeri,

Thank you.


What is this transfer software? – Windows Explorer.

What if your transfer is breaking and that stops internet?

When I move the file to another computer, the transfer stops when the internet connection stops, and Windows Explorer says that the location was not reachable (or something like that). When the internet connection is reestablished the file location was found, and the transfer continues.

This is what I have experienced for the last few months.

When you are not doing a file transfer, does everything stop when the internet connection does?

I do not know as I only have noticed it during file transfers.

How do you know the internet connection stopped prior to breakdown of the file transfer?

The file transfer stopes, then I check the internet logo (bottom right) and it says internet connection not available (or something like that).

When the problem happens, are you able to access Opnsense to view connections and logs?

I do not know how to view the logs and do not know what I would be looking at.

What steps do you take to fix the apparent internet failure?

This is a known failure of my internet; it is an intermittent drop out that happens each and every day. My ISP accepts its existence. I think it is a function of my remote location, but NBN refuses to acknowledge this. The frequency is recurring and can happen from a few seconds frequently, or a few seconds 4 – 5 times a day. Sometimes 45mins. Mostly a few seconds, but enough to stop data transfers from PC to PC. This is not the way I think the network should function????

Google Broken Hill NSW Australia on maps. This will give you an indication as to the remoteness of my location. It is a mining town of 16,000 people. As far as I know I am the only one with OPNsesne.

In writing this note the network has dropped out 2 times. But the last time I did not notice the internet indicator saying offline.

Thanks

Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 12, 2025, 11:14:51 AM
I know very well where Broken Hill is :). My avatar (https://en.wikipedia.org/wiki/Australian_king_parrot).

When you are quoting someone else's text, please use the 'quote' feature to distinguish it. It is the sixth symbol from the right in the toolbar above your message area. Hover over it to see the hint.

Quote... Windows Explorer
So, SMB file transfer, not rolling your own as seemed to be the case from your earlier comment.

Quote... internet connection stops, ... location not reachable ... internet connection reestablished ... transfer continues.
This simply will not happen if traffic is merely passing through the switch, all devices in the same LAN IP range. To be entering the router suggests a second network or a cloud is involved, which would explain loss of connection when you get a WAN glitch.

You also comment that you do not notice internet failures using your computer locally, other than file transfer, and that the system comes up by itself without intervention by you.

I will get to logs later. We may need to go there. For now, two questions:
Title: Re: When internet goes down network goes down
Post by: Patrick M. Hausen on January 12, 2025, 12:03:57 PM
@passeri they are diagnosing "Internet goes down" by looking at the network state icon in the Windows task bar. That might just indicate the entire network is failing, Windows doesn't make a difference here. When the link is down it will show "no Internet", too.

I don't have much time to participate today, just wanted to mention that.

@foxint You might want to start with "ipconfig /all" when the icon shows "no Internet" again.
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 12, 2025, 04:09:53 PM
Diagnosing more than that it seems.
- "Internet goes down". Symptoms could indicate actual ISP failure, modem failure, etc.
- "I cannot access any of my PCs on the LAN" . This is the one that will take time to unwrap.
And with MS Windows, that I am not good at. I might have to reduce my participation. Will help where I can.
Title: Re: When internet goes down network goes down
Post by: passeri on January 12, 2025, 10:15:29 PM
@cookiemonster, I am in the same or worse position, never having used Windows other than in passing, professionally or personally. @Patrick, I saw the ambiguity over what is in fact happening to the network, but I cannot interpret MS symbols as just observed.

A thing I want to test now is that in fact one end of this file transfer is in a cloud, or travelling via a remote route. Certainly Apple and quite likely Microsoft default to pushing data from their own apps to the cloud where you do not specify local. Although we do not have configuration detail, at this point it seems unlikely to me that Opnsense is involved at all. Logs may show whether that is correct.

Edit to add: Any MS users out there with spare time?
Title: Re: When internet goes down network goes down
Post by: foxint on January 12, 2025, 10:39:28 PM
Hi Passeri,

Thank you. And a big thank you to Patrick and Cookiemonster and the whole community.

I am glad you are knowledgeable. Even some Aussies have no idea where Broken Hill is. Thank you.

Due to my synchronisation issue, I have a lot of data I need to move to its soon to be permanent home. I have been slowly moving large amounts of data – usually takes (estimate from Windows) 1-2 hours. (Large for me. And it takes sooooo long.)

No cloud, Google Drive, OneDrive, Drop Box. The data transfer is all from my local PC's in my house connected via switches via ethernet.

I did a large file(s) transfer last night and it stopped. I had to push re-start, and it resumed. Presumably the internet dropped out overnight.
You have confirmed my worst fears. OPNsense users hate/do not use Windows. I looks like I have fallen down the rabbit hole.

Thank you.

Dan
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 12, 2025, 10:54:20 PM
@foxint . What about a suggestion: let's get you on a stable setup first and then we look if the problems go away?
If you are agreeable, can you list your requirements, and your complete setup. That is mostly to get a lay of the land (your IT land). We can propose a decent setup.
- I'd like to know the list of equipment and the ip addresses you have set on them, specify where they are set. Their operating systems where known.
The main part I'm puzzled about is you said you have set static IPs on the machines' OS, so they're unknown to your router possibly. I would suggest to set them as static leases instead (this would be explained in time).
Then you could continue doing your transfers as you know how so far.
- Can you draw a picture of the setup, lines to boxes with names is all that is necessary. Take a photo with a phone and attach it to a post. Are you able to do that?
- What sort of machine you have setup OPN on, what are the NICs? The thinking is if you have realtek NICs they can be really unstable and drop your connections. Saying that, it is expected and said before that the traffic between devices through a switch will be unaffected by this. We need to know how these transfers are happening and why they might be apparently unstable.
What do you say?
Title: Re: When internet goes down network goes down
Post by: foxint on January 12, 2025, 11:56:20 PM
Hi Cookiemonster

Thank you.

I have tried to keep it simple all Windows 10 Pro OS on all PC's.

192.168.1.1   OPNsense Box HP Elite Desk i5            OPN Box
192.168.1.115   HP i5                     Movies
192.168.1.105   GA-X99-SLI-CF i7–5829K       Self-built   Back up
192.168.1.118   GA-X299-UD4 Pro-CF i9-7900X   Self-built   Main Back up
192.168.1.124   GA-X79-UP4 i7 3820         Self-built   Tax
192.168.1.144   GA-X299-UD4 Pro-CF i9-7900X   Self-built   Main
Future                           Back up II

Dynamic
192.168.1.151   NetCom Wireless
(old modem/router modem part not working)            Wireless Hub for Phone
192.168.1.152   Samsung                   Phone

NBN Box
TP-Link 2.5Gbit Switch 8 port.
TP-Link 1.0 Gbit Switch 8 port.
Netgear 1.0 Gbit Switch 24 port.


Outside world – NBN Box – TP-Link 2.5Gb Switch
NBN box – OPNsesnse box – TP-Link 2.5Gb Switch
Main – TP-Link 2.5Gb Switch
Main Back up – TP-Link 2.5Gb Switch
TP-Link 2.5Gb Switch – TP-Link 1.0Gb Switch
Movies – Netgear 1.0Gb Switch
Back up – Netgear 1.0Gb Switch
Netgear 1.0Gb Switch – TP-Link 1.0Gb Switch.

I could possibly remove the TP-Link 1.0Gb Switch and rout this all through the Netgear.

Hope this helps.

Dan
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 13, 2025, 12:46:57 AM
Quote from: foxint on January 12, 2025, 11:56:20 PMHi Cookiemonster

Thank you.

I have tried to keep it simple all Windows 10 Pro OS on all PC's.

192.168.1.1   OPNsense Box HP Elite Desk i5            OPN Box
192.168.1.115   HP i5                     Movies
192.168.1.105   GA-X99-SLI-CF i7–5829K       Self-built   Back up
192.168.1.118   GA-X299-UD4 Pro-CF i9-7900X   Self-built   Main Back up
192.168.1.124   GA-X79-UP4 i7 3820         Self-built   Tax
192.168.1.144   GA-X299-UD4 Pro-CF i9-7900X   Self-built   Main
Future                           Back up II

Dynamic
192.168.1.151   NetCom Wireless
(old modem/router modem part not working)            Wireless Hub for Phone
192.168.1.152   Samsung                   Phone

NBN Box
TP-Link 2.5Gbit Switch 8 port.
TP-Link 1.0 Gbit Switch 8 port.
Netgear 1.0 Gbit Switch 24 port.


Outside world – NBN Box – TP-Link 2.5Gb Switch
NBN box – OPNsesnse box – TP-Link 2.5Gb Switch
Main – TP-Link 2.5Gb Switch
Main Back up – TP-Link 2.5Gb Switch
TP-Link 2.5Gb Switch – TP-Link 1.0Gb Switch
Movies – Netgear 1.0Gb Switch
Back up – Netgear 1.0Gb Switch
Netgear 1.0Gb Switch – TP-Link 1.0Gb Switch.

I could possibly remove the TP-Link 1.0Gb Switch and rout this all through the Netgear.

Hope this helps.

Dan

It helps a lot. Now please draw a logical (not physical) map of these connecting to each other. No need to go all fancy, draw with pen and paper and upload a picture. Please.
Title: Re: When internet goes down network goes down
Post by: foxint on January 13, 2025, 01:09:08 AM
Sorry for the delay.

Trying to re-size the image.

No matter how small I make the file, I cannot upload it

Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 13, 2025, 02:45:36 AM
That diagram is definitely needed. If you are unable to upload one, I will try to draw my interpretation so you can correct it. That would not be until later today but we are in nearby time zones.

The symptoms had fitted with a normal network doing cloud access. Now the network in text form does not make sense to me. It appears to say, for example, that you have at least one switch as well as a router connected directly to the NBN. I await clarification.
Title: Re: When internet goes down network goes down
Post by: foxint on January 13, 2025, 03:18:58 AM
Hi Passeri,

Thank you.

The NBN Box is only connected to the OPN Box and the OPN Box is connected to the 2.5GB switch.

Thanks

Dan
PS - looks like I managed to attach the map
Title: Re: When internet goes down network goes down
Post by: passeri on January 13, 2025, 06:03:29 AM
I just need some further clarification on that, because for me the words and the diagram do not marry up.

Is the first part this?
NBN --> OPN --> 2.5Gb switch --> other switches and devices

or this?
NBN -->  2.5Gb switch --> OPN -->  switches or devices
             |
             v
    switches or devices

How many ethernet ports does your box running Opnsense have? You show two devices directly attached and more through a switch or two. Are you able to answer cookiemonster's question about the brand of ethernet card? It should be on it.

By the way there are plenty of Windows users employing Opnsense, and on a variety of hardware.
Title: Re: When internet goes down network goes down
Post by: foxint on January 13, 2025, 07:34:20 AM
Hi Passeri,

Thank you.


NBN to OPN to 2.5Gb Switch to another switch.

OPN box has 3 but I could only get 2 to work. The PC had 1 Ethernet port and I bought a twin (2) port card, but only 1 worked. (Probably my mistakes).

Thank you.

Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 13, 2025, 11:06:38 AM
Thank you for clarifying that, foxint.

What is the 2-port card you bought please, name and model? Interesting that you could not get one port to work but I can make nothing of that as yet.

Some basic tests I should have requested earlier:

On the information to date, it ought not pass step 2. In that case, please
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 13, 2025, 12:40:58 PM
QuoteInteresting that you could not get one port to work but I can make nothing of that as yet.
I reckon foxint expects the box with OPN on it to behave as a consumer router, where ports are bridged by default. I think since there are switches available we leave that for now to keep it simpler. For now.

My interpretation of the landscape is attached. @foxint can you confirm or correct it. Make it easy for us, corrections make them graphic for now.
foxint1.png
Title: Re: When internet goes down network goes down
Post by: passeri on January 14, 2025, 01:39:22 AM
As side comment, the NBN box in question has a single ethernet port to the client and an RJ12 connector to a telephone wall socket with copper to a kerbside node where the fibre terminates. Some homeowners in the past would DIY additional splitters for other phone or fax services, passing the technically advised junction count. If these are still in place they may cause unreliability apart from the fact of copper lines. There may be a current plan to convert all connections to FTTP as was originally intended, depending on who wins elections. Mentioned as a possible driver of connection faults which one-off tests by NBN Co may not detect. End of aside.
Title: Re: When internet goes down network goes down
Post by: foxint on January 14, 2025, 03:58:23 AM
Hi Guys,

Thank you. I have been a little busy today, my apologies.

Basic test. I pulled the plug from the OPN-Box and the NBN Box and moved some files from various PC's and this was no problem.

It passed the test, so I am assuming I do not follow instructions in paragraph 2. I have reconnected the ONP-box and NBN-Box.

Cookiemosters diagram looks correct.

As for my house it is a 100 plus year old stone cottage in Broken Hill. So that means it is in need of total renovation. I have the original electrical wiring and all the joys that comes with an old house.

Internet arrives at the house via a copper wire. I have removed ALL additional cabling that the previous people had attached. So there is 1 wire into the house.

I have plans to re-wire this house and after that get FTTP. But it has taken me 1 ½ years to get 4 power points and I am not holding my breath for the remainder of the re-wiring to happen anytime soon. The NBN man wants me to run conduit into the roof and down into the house where I want the modem. The electrician has agreed to do this sometime.

This is the joy of living in the back end of nowhere.


Thanks

Dan
Title: Re: When internet goes down network goes down
Post by: cookiemonster on January 14, 2025, 10:25:00 AM
We never got to verify the settings, so don't be surprised if the issues re-occur. Problems that just vanish are just likely to come back.
Thanks for confirming though. At least now you can more easily visualise/describe the layout.
I'm moving on now.
Title: Re: When internet goes down network goes down
Post by: passeri on January 14, 2025, 12:01:04 PM
Good to hear that Dan. Keep an eye on your bits of equipment. We probably have not solved the problem, merely confirmed that Opnsense/WAN is highly unlikely to be involved.

What I appreciated was that you sought to answer our many questions or said plainly what you did not yet know. You now have a clear basis for future analysis, some improvements to your firewall config, and a bit of experience finding information in the Opnsense GUI. Something for your effort.
Title: Re: When internet goes down network goes down
Post by: foxint on January 15, 2025, 08:52:10 AM
Guys

you have been brilliant. Knowledgeable and patience.

I hope it is fixed. Crossing fingers.

Thank you.

Dan
Title: Re: When internet goes down network goes down
Post by: foxint on January 15, 2025, 09:05:43 AM
Still have the issue.

Sorry.

Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 15, 2025, 09:48:34 AM
As mentioned, that was pretty much expected. I believe though that you need to look at your other hardware or software, not Opnsense. We know that copy works without internet as was always most likely unless you were using some cloud service or intermediary, which you are not. Therefore it is a stretch to suggest that loss of internet interferes with a local file transfer -- what would be the mechanism, and why not when it is disconnected deliberately? The data on a local network does not even pass through the router.

Further investigation should relate to non-Opnsense network or PC trouble-shooting, so I think is not really appropriate for this board unless tests eliminate alternatives and some mechanism can be hypothesised. You know the shape of your network, so can look for critical points. I am happy to discuss plausibility of Opnsense related network issues that you think you may have found.

In my view it is time for you to move your testing focus until you can eliminate other possibilities.

Adding: Have you considered that an internal network or PC failure may stop access between systems and to the internet rather than vice versa? For example, if you are doing large file transfers then bits are going to warm up, and cool after they stop.
Title: Re: When internet goes down network goes down
Post by: foxint on January 15, 2025, 07:42:31 PM
Agreed.

You guys have been above excellent.

Thank you.

It will be sad to go...ha-ha.

Keep well and I wish you and all the guys on the board the best.

Dan
Title: Re: When internet goes down network goes down
Post by: passeri on January 15, 2025, 10:11:10 PM
I would look at your NICs and switches first. You may need to reduce the network for testing.

Everyone is still here for any question on Opnsense. Good luck.