Messages

Bump

Hi there!

I'm using an SFP module to connect my ISP's fiber link directly to the machine I'm running OPNsense with (Minisforum MS-01), applying the requisite VLAN tag directly to the bridge interface in Proxmox and connecting it as a VirtIO network adapter to take advantage of multiqueueing.

However, after starting the configuration of the firewall VM from scratch this time, it seems I might have changed a setting that prevents the PPPoE link from coming up immediately when the VM is started or rebooted, and as such I need to manually bring the interface down and up after a blackout thanks to the UPS shutting the computer off before running out of juice.

Considering that this might happen overnight or when I'm out of reach, it can make remote troubleshooting a little bit more difficult than what's ideal, but I'm not sure where to begin looking.

So far, I can tell that the behavior is consistent whether I'm using the SFP module or the ISP router in bridge mode, but not much more than that.

If y'all needed to look into this, where would you begin?

Thanks in advance!

I was under the impression from your original post that the cost could be x10 (probably with other benefits like higher bandwidth or better SLAs and support).

Eh, to a degree that's also true. The way the local ISP tried to sell it to me was that they offer three service tiers (Residential, Business and Enterprise).

What ended up being the same price was switching from the residential service to the business tier, but as far as I've been able to tell (they still haven't finished the arrangements on their end), the only practical difference is that the contract doesn't restrict me anymore from indirectly reselling the service (so if I were a restaurant with a guest network for example, I wouldn't get fined for conditioning internet access on the diner's consumption).

There's a slight chance I might get to talk to someone else at a different department when I call the ISP, but that remains to be seen still.

The part where things get really expensive is that according to them both the provisioning of PPPoE credentials and enabling bridge mode on the ONT are enterprise features, and unless they're forced by law, the only way you're getting a firewall on your network is with a double NAT from the get go.

They wouldn't give an exact quote, but supposedly enterprise contracts include a dedicated technician on their end instead of the regular call center, yet none of the ISP's offerings have any sort of legally binding warranty and/or SLAs (in fact, if you look them up on our FCC equivalent — the IFT, the contract only makes a best-effort promise to deliver the speed of the plan, but has a warranty of 0Mbps minimum service at any time).

So long story short, unless the service failing costs them more than it costs you, tough luck.

While I haven't had to deal with IPv6 yet (my ISP doesn't support it), I've done a little bit of reading on that subject, to be prepared (it's supposedly in the works).
RFC 6177 is the guidance.

While /64 is legit, it makes the case for smaller prefixes to allow multiple subnets.
That RFC was written in 2011...
GUEST or IOT subnets are even more prevalent today than they were back then.

Personally, I don't really understand the ISP pushback here.
It doesn't seem like allocating a /56 or /60 versus a /64 is costing them anything but the software costs of managing the smaller allocations (which is trivial).
I might understand a small add-on but nothing like switching to a business tier...

I also agree with you.
I've been looking into this since switching to fiber on some fashion or other, and it definitely seems that from the support standpoint there's a marginally better chance of getting anywhere with a business contract (lucky for me, this wouldn't even make any difference from a price standpoint at the moment).

Getting anyone to listen on their end is still a shot in the dark from what I can tell, but this thread has convinced me of at least trying to upgrade my contract for the time being. So, thanks everyone for helping me organize my thoughts on the matter!

Just one opinion: unless you have business class service, don't even bring this up with the support agent.  If you have options where you live try to find an ISP that you already know, from online research or word of mouth, that does prefix delegation with more than a /64.  For example here in the northeast US region, I know from experience that Comcast Xfinity honors a /60 PD request and Verizon Fios honors /56 at the time of writing.  You can call and ask before signing a contract, too.

Some ISPs maintain lists of supported 3rd party devices.  You'll typically find retail devices on the list, the likes of Motorola, Asus, TP-Link, etc.  Some Ubiquiti devices show up on the Comcast list now.  IMO, if the ISP lists retail devices that support VLANs and sub-netting as a core function, that's a good sign.  You likely won't find Deciso, Netgate, Protectli, etc.  That doesn't mean those won't work, just that the ISP doesn't officially support them.  You're on your own and if you mention it to the support agent they might escalate in an unhelpful way.

To be completely honest, I agree with you. Thing is, here in Mexico there's a single sided monopoly over networking that's pretty inescapable most of the time.

Be it for business or residential use, there are only four ISPs operating within the country (Telmex, Izzi, TotalPlay and Megacable), though in practice there's only one (Telmex).

Except for the last one (they still operate using Coaxial), every other provider works with FTTH, even though they all encapsulate traffic with PPPoE and none of them offer symmetrical connectivity for residential services (you're only able to ask for it with a business contract and just if you agree to an additional 60% markup fee).

Only Telmex gives out dynamic IPv4 addresses that change only when the link drops, all the others use CG-NAT and coverage is a hit or miss.

The final nail in the coffin is that the parent company of Telmex is also the majority owner of the biggest two cell carriers (Telcel and Movistar), while all of them fall under the umbrella of America Móvil (who operates several providers under different names for most of Latin America, be it Telmex, Telnor, Claro, or others), so even if you ignore them as an option to make the contract, sooner or later your connection to the rest of the world will go through their infrastructure in some fashion (be it as a gateway, ASN, or even a submarine cable).

For better or worse the service is decent as long as you only need an uplink, even their own IPv6 address blocks have been registered since around 2013. The main problem arises with implementation times (it took them till last week to enable IPv6 on the mobile side, and they still had issues routing DNS traffic over it). So as far as choice comes, in practice there's no other provider within the country no matter where you look.

If nothing else, let this be a decent cautionary tale of the end result of a monopoly.

Hello everyone!

So over the past week, several of my Homelab's projects started behaving erratically while on the go, since fortunately my phone's carrier finally started deploying IPv6 to regular customers (with a faulty DNS server, but at least they're working on it).

Turns out that now since my phone can try to use a dual-stack connection instead of only a CGNAT, anything I had configured with an AAAA record would freeze and timeout, only going back to the A record after several retries.

Thing is, my ISP only gives its customers a /64 from what I'm able to gather, and considering asking for anything beyond a reboot during troubleshooting with them tends to go sideways (I had to sue them to be allowed to use the bridge mode on my ONT), so I was wondering if anyone had any pointers on how to bring up the topic of prefix delegation with an ISP and successfully getting to at least a quote and/or reasonable answer.

I do work from home at the moment, and while I have a residential connection, upgrading to a small business contract is not out of the question if needed.

Under normal circumstances, even bringing up the topic of subnetting, using anything aside from their GPON ONT CPE in the network, or PPPoE to the personnel at their end tends to receive puzzled looks at worst, or them trying to justify their usage as Enterprise-only features that would make the bill increase tenfold at the best of times.

So if there's anything in particular I should try to mention, I'd love to hear from others in similar situations how they were able to get the point across.

Cheers and thanks in advance!

You dont need it if the provider has a route to you.

Its for setups where you only get an address via slaac without a route from the provider.

I should adjust the documentation a bit sometime.

Huh, funny that.

On the one hand I find it worryingly scary that such a scenario is probable enough to warrant development of a tool to handle it (why would anyone provide a networking protocol without the ability to natively route traffic‽), but on the other I find it great that you were able to facilitate a solution in the first place.

So I guess I'm back to square one in matters of subnetting IPv6. In any case, thanks for the prompt response! Now to figure out how to wrangle this with my ISP.

You cannot split a /64 further or SLAAC breaks.

If you get only a single /64 prefix you can use ndproxy for one internal LAN and thats it.

Hmm, perhaps I misunderstood the purpose of the tool then in the first place.

Before ndproxy was available, I already had IPv6 working on the LAN interface by following a similar method (WAN asks for a PPPoE link over IPv4 with PAP credentials to my ISP and requests a prefix to be assigned, ISP hands it out to the WAN interface and I track the assignment on the LAN interface).

However, neither SLAAC, DHCPv6 nor Router Advertisements proved difficult to get working under the previous conditions (regardless of if it was done using OPNSense, OpenWRT, or even Windows at some point).

So if everything supposedly worked as-is before I tried to integrate ndproxy into the network, does that mean I didn't need to in the first place, or did I actually have missing functionality and just never noticed?

Hi there!

I'm trying to follow the instructions outlined in the manual to configure NDProxy with an ISP that only delegates a /64 over an ONT CPE through PPPoE (the link is encapsulated with a VLAN tag and the ONT only has 1000Base-T connectivity for end-user devices).

At the moment, the LAN interface is able to connect over IPv6 successfully (same as the firewall), though I'm not yet sure what would be the proper way to implement this solution in my network given I segregate traffic using VLANs (four for the time being).

I assume that if I were to track the LAN interface (as it's untagged/VLAN ID 1) from any of the VLAN interfaces, the problem would solve itself.

However, I'm unable to select it in the list of trackable interfaces, and attempting to do the same with the WAN interface forces me to choose a prefix ID of 0, which given I'm on a /64 returns me to square one.

Is there perhaps something else I'm missing?

Thanks in advance!

Hello there!

I am trying to run OPNSense 23.1 within Proxmox 7.4-3 using a bridge adapter for both WAN and LAN, but for some reason I am unable to get the VM running for more than 5 minutes at a time, and I'm honestly not sure where to begin diagnosing the problem.

I'm using the same connection as with my previous router (a Raspberry Pi 4B running OpenWRT), and honestly, I'm pretty stumped on how to continue, given that even PPPoE dialing seems to prove difficult for the machine now that I've moved over.

While I'm aware I should be providing logs, I'm not sure which ones would be useful for diagnosing this issue, so please let me know what I need to provide so I can post it and begin troubleshooting.

As an aside, the very same instance of Proxmox is running a VM with Home Assistant OS, and so far, it's been running uninterrupted since installation, so I'm pretty sure the issue would lie with either how I configured the OPNSense VM or the guest OS itself.

In any case, thanks for the help!