Messages

At least you haven't been caught up with the whole GEA issue

I'm not sure we'll ever find out what happened.  I spent a while reading the thread you referred me to last night - specifically the epic 16 page thread.

Something definitely went wrong during the IP address provisioning process.  The timing was immediate on their man pressing the button and the consumer to business regrade took place 3 weeks prior.

5 Openreach engineers all insisted the issue was with Zen's rack at the Exchange, plus a sixth higher level Openreach engineer who had clearance for the Exchange (N27 level was it?) agreed.  Nothing was touched at my office, the ONT was never replaced and 3 different routers with new cables performed the same.  Even the workstation wired directly to the ONT with a dial up PPPoE connection was intermittent.

Interesting to learn about the wider issue on Zen's side though. 

Plus I learned a couple of things myself.  Next time I'll start with is it switched on? And if it's not that we'll know it's DNS, it's always DNS. Except when it's cache invalidation.

Thanks again.

Issue is resolved - embarrassingly it turns out I can't tell the difference between orange and green - see updated first post if you fancy a laugh at my expense.

> Assume that is a Typo, and you have performance issues, only getting 100Mbp/s down as per the title, with multiple different routers?
Yes, my original post had a typo in the body, downstream maxes out at 90Mbps and has done for weeks although it was worse (wildly intermittent from 0.8 to about 500Mbps) before stabilising at this level.

> I'm guessing your ISP is Zen?
At first I thought you knew Zen are handing out Fritzboxes now but from what you say it seems their issues are all over the tech forums now.

The reason I raised this on the OPNsense forum now was Zen advised the issue is resolved as they replaced kit at the Exchange.  So to verify I hooked the ONT up to their Fritzbox and connected a laptop by LAN.  It ran 800/100 so it looked like they did indeed resolve the issue.  However on connecting the ONT back to the OPNsense appliance and wider office network the 90/100 issue presented.

So suspicion fell on the OPNsense appliance as potentially I knocked a setting over the past 10 weeks attempting to figure this out.  I've now spent about 6 hours there which now seems like yet more time burned for nothing.

The original symptoms were always wildly fluctuating speeds.  Potentially I just got lucky with 800/100 when the Fritzbox was on.  I'll hook it up again at the weekend to see if runs that speed again or if it's still just fluctuating.

> I'm guessing the issues started only when you moved to the new business package?
The trigger was definitely when the 8x IPs were assigned, guy on the phone said "I'm doing it now, allow 15-20min for propagation, officially it's 24hrs but usually 20min.

The saga started 3 weeks earlier.  Zen regraded the line, adjusted billing and issued paperwork but the IP block never appeared. Called, was told 1 week, no IPs, called again, told 1 week, no ips, called again and this time your man could make the switch immediately.

Line speed was 900/100 throughout the saga, soon as he hit the button the problems begain.

Its been mental, 5 engineers on site in a two week period back at the start. Every time they called in advance to advise the remote test shows the fault as Ethernet Handover Failure which relates to Zen's rack at the Exchange which Openreach can't touch.

> I'm assuming when you moved to the business package, you were subject to a GEA migration - whereby you are moved from
> the BT Wholsesale GEA in your exchange to Zens own GEA - you might be able to see this if you log into the Zen portal,
> go to the 'old' portal, and look at order history.
Spot on, the customer portal shows an order for "Zen FTTP GEA Migration".

> Assuming all of the above is correct, it is a Zen issue. They have some kind of issue with their GEA links on
> connections >500Mb/s that 1. They can't work out, and 2. Largely refuse to accept, claiming
> it is an historic issue already fixed.
Appreaciate you pointing this out, I hadn't looked though the forums assuming it was largely isolated to myself and a few othe Zen customers.

One of the Openreach Engineers had searched the fault code on a closed group forum the Engineers use, at the time there were 6 mentions of the code and 5 related to Zen.  He said he'd never seen that particular fault code before.

> Go onto Think Broadband Forums, and Kitz.co.uk forums and do a search for 'Zen GEA' - you'll find reports of the
> same issue...On Kitz, look in the FTTP Issues section...there is quite a recent one with symptoms as bad as yours....
> ....good luck.
I'll take a look now.

Appreciate your insights.

[Plugins]

UPDATE - schoolboy error - the LAN connection from OPNsense to the switch had negotiated at 100Mbps.

You'd think I would have noticed the LED liot up orange (for 100) instead of green (for 1000).

Original post follows.

Can anyone help diagnose the issue described below please?

My gigabit FTTP connection was working at full speed, 900Mbps down/100Mbps up for 6 months without issue.  The line was regraded from consumer to business so that 8x public IP addresses could be added, at this point line speed became wildly intermittent and as low as 0.8Mbps down.

The ISP advised the NOC replaced some kit at the exchange and they are no longer seeing the original issue (Ethernet Handover Failure).

OPNsense now achieves a consistent 90Mbps down/100Mbps up

However with both a second OPNsense box and the ISP supplied Fritzbox line speeds are the full 900Mbps down/100Mbps expected.

I think a setting may have been changed accidentally when messing with the configuration, however can't rule out a coincidental hardware failure.  In relation to that the latest update to OPNsense 23.1.11-amd64.

Plugins

• Zenarmour (basic)
  Crowdsec

Hardware

• Dell R210ii
  Xeon E3 1270 3.4Ghz Quad Core
  32GB DDR 1333
  Dual Port Broadcom NetXtreme II BCM5716 Gigabit Ethernet
  Netgear GS324 and GS308 gigabit switches (both new)

I have an Intel E1G44ET Gigabit ET Quad Port Adapter but this is not fitted nor am I sure if it would help (although from what I've read Intel NICs are billed as the preferred choice).

Things I have tried

• read dozens of posts
  have power cycled
  replaced the cables from ONT to WAN port on R210ii and GS324 to GS308 with the same cables that work on the second OPNsense box

`$ ifconfig -a` returns

```
media: Ethernet autoselect (1000baseT <full-duplex>)
```

I am not convinced this is the WAN port OPNsense sees though because from the GUI two interfaces appear to be related to bce0, even though this is the port connected to te ONT.

Under Interfaces > Assignments the WAN (opt2) entry shows `pppoe (bce0) and the New interface dropdown shows `bce0` as available.  I'm unsure why `bce0` appears twice.

I tried deleting the WAN interface assignment and adding it again but the behavior remains the same.


`$ ifconfig | grep media` returns

```
media: Ethernet autoselect (1000baseT <full-duplex>)
media: Ethernet autoselect (100baseTX <full-duplex>)
```

I guess the two entries relate to `bce0` WAN and `bce1` LAN?

From other posts users looking at the same symptoms refer to the `Speed and duplex`. My GUI shows

• Interfaces > LAN has dropdown for `Speed and duplex` under the MSS setting
  Interfaces > WAN does not have the `Speed and duplex` option

I'm unsure why the dropdown is missing from the WAN interface or if this is relevant (may be because this is PPPoE?). The solution on the post I saw this under was to remove and recreate the interfaces which I've already done.

Any ideas welcomed.

Thanks,

Chris

[Objective]

Objective
To access the OPNsense GUI, using the Tailscale IP address assigned to the OPNsense appliance from any Machine connected to Tailscale.

Overview
After much trial and error this was working last night but I broke it by removing what I thought were irrelevant settings.  Despite restoring config files from `System: Configuration: History` I've been unable to reach the previous state and regain access the OPNsense GUI from Machines on Tailscale.

Can anyone help me figure the correct configuration please?


Steps taken
- Installed Tailscale following the instructions at https://tailscale.com/kb/1097/install-opnsense/
- Failed to gain access to the OPNsense GUI from Machines on Tailscale so in desparation removed Tailscale and reinstalled with
```
make deinstall
make clean
make install
```
- From memory this removed the `TLSCL` Interface so I added it back in from `Interfaes > Assignments` as `TLSCL (opt1)` on `Network Port taiulscale0`
- Created Firewall Rule for the `TLSCL` Interface allow traffic form the network to the interface address - as far as I understand this is the same as the default LAN rules except for net traffic on the TLSCL interface, and should allow traffic to Tailscale IP of the OPNsense appliance 100.11.22.33 on port 443
```
   IPv4 *   TLSCL net   *   *   *   *   *      Default allow TLSCL to any rule
```
- Discovered I need to update System > Settings > Administration: Listen Interface to include `TLSCL` alongside the default `LAN`
- Assigned the Tailscale IP of OPNsense Machine as a Static IPv4 to the TLSCL Interface at `Interfaces: [TLSCL]` - this seemed to be the key step in finally getting access to the GUI on the Tailscale IP of the OPNsense appliance
- Can ping a Tailscale Machine IP when logged into the OPNsense appliance via SSH (the first 3 responses are via DERP, the final response via the remote Machine's true Public IP)
- Can ping the Tailscale IP of the OPNsense appliance from another Machine on the Tailnet
```
chris@DO-XLR:~$ tailscale ping 100.11.22.33
pong from opnsense (100.11.22.33) via 81.82.83.84:37216 in 16ms
chris@DO-XLR:~$ tailscale ping 100.11.22.33
pong from opnsense (100.11.22.2339) via 81.82.83.84:37216 in 15ms

```
- this logs errors in the Firewall
```
TLSCL      2023-03-17T15:00:14   100.44.55.66:41244   100.11.22.33:443   tcp   Default deny / state violation rule
```
- Detailed output from the block is
```
Detailed rule info
__timestamp__   2023-03-17T15:18:53
ack   
action    [block]
anchorname   
datalen   0
dir    [in]
dst   100.11.22.33
dstport   443
ecn   
id   50907
interface   tailscale0
interface_name   TLSCL
ipflags   DF
ipversion   4
label   Default deny / state violation rule
length   60
offset   0
protoname   tcp
protonum   6
reason   match
rid   02f4bab031b57d1e30553ce08e0ec131
rulenr   4
seq   2736343943
src   100.44.55.66
srcport   47084
subrulenr   
tcpflags   S
tcpopts   
tos   0x0
ttl   64
urp   64480
```



Environment
- OPNsense 23.1.3_4-amd64
- FreeBSD 13.1-RELEASE-p7