Topics

[SPLIT TUNNEL]

v25.1.11 in DEC 850.
Wireguard macOS peer client app.

I have two working WireGuard VPN instances running ok, each with its own interface assignment and no peers with split tunnels:

• one for LAN only over vpn
 (its peer can access the file server on the LAN ok but can't browse the internet because of a related LAN-only firewall rule)

• one for internet only over vpn
 (its peers can browse the web and whatismyipaddress shows their public IP address is OpnSense's  instead of their ISP's and they can't access the LAN)

SPLIT TUNNEL

Today, I spent hours trying to create a split tunnel in several test peers so they can connect to the LAN but browse the internet over their own ISP instead of the vpn's.

The documentation seems simple, but I'm missing something.

For new peer tests with the LAN_only instance, I replaced Allowed IPs 0.0.0.0/0, :::/0
with the LAN network 10.1.10.0/24

Though the peer then browsed the internet over its ISP instead of the vpn, it couldn't access the LAN's file server anymore (10.1.10.99).

Huh???

Yet I can ping it -- 10.1.10.99.

But macOS won't connect to smb://10.1.10.99 as does fine with the default Allowed IPs in the peer.

By design I can't reach, including ping, other nodes on 10.1.10.0.

Then I changed Allowed IPs to 10.1.10.0/24, 10.25.25.0/24 (the latter is the vpn peer range) but the same thing happened.

How did specifying the LAN net(s) break the peer's access to the same LAN?

When I created a test peer with the default Allowed IPs 0.0.0.0/0, :::/0, the peer could again connect to the LAN's file server (smb://10.1.10.99) but by design couldn't browse anything on the internet because of its related LAN-only firewall rule.

I feel like it's gaslighting me. I'm sleep deprived and probably missed something obvious.
 

My local gigabit fiber ISP company was great for years but sold its residential business to another ISP about a year ago.

Unbeknownst to me, the new ISP uses CGNAT (Carrier-grade NAT: https://en.wikipedia.org/wiki/Carrier-grade_NAT). Its purpose is to conserve the ISP's IPv4 addresses to keep the ISP's costs down. The ISP should use IPv6 addresses instead.

CGNAT prevented my newly configured WireGuard VPN from working. This was my first time setting up WireGuard. I read that WireGuard is fast but difficult to set up, so I assumed I was botching the settings from the WireGuard Road Warrior Setup guide (https://docs.opnsense.org/manual/how-tos/wireguard-client.html#step-5-assignments-and-routing) and other guides, doc and video online.

My dynamicDNS address wasn't my DEC850's. It was the ISP's CGNAT address. The address difference was subtle because only the last octet was different (and just +-1) from my OpnSense WAN IP. I didn't notice that for a long time, dang it. I also tested using my WAN IP address in WireGuard instead of dynamicDNS but it wasn't routeable either.

I asked my ISP why my whatismyipaddress-dot-com address in my computer browser (and in dynamicDNS) was different from my firewall's WAN IP address.

They told they used CGNAT. I hadn't heard of it so read about it.

What a nightmare of wasted time. On and off for about 6 months, I slugged it out with OpnSense WireGuard, deleting all the configuration elements then adding them back using different advice online. I estimate I wasted 30-40 hours based on three to four sessions lasting 10 hours each.

The ISP's tech support said I could opt out of CGNAT for no extra cost if I submitted a webform explaining why. They couldn't guarantee it'd be approved and didn't give me a time estimate for their decision or implementation.

My ISP recently approved my request and took me off CGNAT, though they messed up their configuration and left me without internet for most of a day.

Today, after deleting all my WireGuard settings, including instance, peers, interface, firewall rules, I added them back cleanly from the WireGuard Road Warrior guide and everything worked the first time!

At least in my struggles I learned much more about WireGuard, OpnSense and CGNAT than I would have if WireGuard worked the first time.

This ISP has way more scheduled and unscheduled outages than the previous one. That's probably in part due to CGNAT, which is more complicated to maintain, I'm told.

I don't trust or like this small ISP. I'm switching back to my previous one as a business customer (they don't accept residential customers anymore). All their IP addresses are fully routable and they rarely had outages (maybe one a year).

DIRT ON CGNAT

From perplexity.ai...

The use of Carrier Grade NAT (CGNAT) by ISPs introduces several significant drawbacks for both providers and end-users:

## Performance Degradation 
- **Increased latency**: CGNAT adds an intermediary layer, causing delays in real-time applications like VoIP, video conferencing, and online gaming[1][4][8]. 
- **Network congestion**: Shared public IPs can lead to slower browsing and streaming during peak hours[2][4]. 

## Connectivity Limitations 
- **Broken peer-to-peer (P2P) functionality**: NAT traversal issues disrupt torrenting, gaming (e.g., Xbox), and direct device communication[1][3][6]. 
- **Remote access challenges**: Port forwarding restrictions prevent users from accessing home servers, security cameras, or NAS devices externally[2][4]. 
- **Hosting restrictions**: Self-hosted services (websites, email servers) become nearly impossible due to inbound traffic blocking[2][4]. 

## Security and Reputation Risks 
- **Shared IP vulnerabilities**: If one user engages in malicious activity, the entire shared IP may face blacklisting, affecting all users[2][4]. 
- **Expanded attack surface**: A DDoS attack on one IP can degrade service for multiple customers[5]. 

## Operational Challenges for ISPs 
- **Higher costs**: Implementation and maintenance fees, combined with IPv4 address scarcity, increase expenses[1][7]. 
- **Complex traffic monitoring**: Law enforcement and abuse tracking become difficult due to shared IPs[4][7]. 

## User Experience Issues 
- **Streaming quality degradation**: Buffering occurs on platforms like Netflix due to latency[2][4]. 
- **Smart home limitations**: Remote management of IoT devices often requires cloud workarounds[2][4]. 

These drawbacks highlight why CGNAT is often viewed as a temporary solution to IPv4 exhaustion, with IPv6 adoption being the long-term alternative[1][6].

Citations:
[1] https://ipv4connect.com/2023/06/pros-and-cons-deploying-carrier-grade-nat/
[2] https://ausgeek.net/viewtopic.php?t=284
[3] https://www.daryllswer.com/shortcomings-of-cgnat-and-potential-work-arounds/
[4] https://www.rapidseedbox.com/blog/cgnat
[5] https://www.corero.com/ddos-disadvantages-of-carrier-grade-nat/
[6] https://www.reddit.com/r/HomeNetworking/comments/s2ulh6/does_anyone_else_hate_it_when_isp_puts_you_behind/
[7] https://forum.universal-devices.com/topic/21946-ftth-fiber-to-the-home-isps-and-cgnat/
[8] https://www.snbforums.com/threads/pros-and-cons-of-static-ip.87208/

Newbie.

Configured the Dec 850 and updated to latest community edition (instead of Business ed it came with for a year).

In the web interface, instead of configuring igc0 lan0 and igc2 lan2 to communicate with each other just with rules, I tried to "bridge" them.

While bouncing between settings for igc0 lan0, igc2 lan2 and the new bridged interface, I messed up and disabled the igc0 lan interface Chernobyl style (I had bad feeling but kept going as I unchecked the box, saved and clicked apply). Late night config is too dangerous for me.

FAILED with CONSOLE

• I used the serial console to login, but "This account is currently not available" appeared. When I typed a wrong password intentionally, "Login incorrect" appeared instead.

• I also tried root with the password I had written down and the default password, but couldn't login ("Login incorrect").

SSH DISABLED
I recently enabled and used it to check the microcode version then disabled it again, so I know it's not an option.

Setting a manual IP for my computer and connecting to igc2, hoping the bridge might work, didn't help.

RESET HOLE
Apparently it doesn't restore to factory defaults (tried 60 seconds, but the DEC 850 just turned off). Maybe there's a button-press pattern that restores it factory defaults.

Somehere here posted in Oct 2024 that they might open their DEC 850 to reset the BIOS, which isn't my goal, but that makes me wonder if I can restore the DEC 850 to factory defaults by opening it and maybe replacing, erasing or rewritting its SSD, if removable.

I saved a backup config before I messed up the settings. But it won't do me any good if I can't get back in.

Fortunately, I have another firewall I can use.