OPNsense on Sophos XGS: drivers for every front port (XGS 136 / XGS 3300)

[abdelmonem]

Hi everyone,

Anyone who has tried OPNsense on a Sophos XGS knows how it goes: it installs, it boots, and there isn't a single network port to assign. That's where I started too. The reason is that these boxes are two computers in one case. The x86 board runs the firewall, and every front port hangs off a Marvell coprocessor sitting behind a PCIe link. Sophos only has drivers for its own OS.

I've been at this for close to two years now, first just figuring out how the two halves talk to each other, then writing FreeBSD drivers for the coprocessor and an OPNsense plugin around them. It's finally at the point where I'm comfortable showing it.

On my two boxes:

• XGS 3300 (Cavium OCTEON TX): all 12 front ports, plus the 4x10G expansion module
• XGS 136 (Marvell ARMADA): all 14 front ports

The part I'm happiest with is the offload. Once pf has passed a connection, the coprocessor forwards it on its own, and IPsec set up by strongSwan runs on the coprocessor's crypto engine (IPv6 and ESN included). pf stays in charge, so a block list, a JA4 fingerprint on a list or an IPsec policy that starts covering the traffic will still cut a connection that's already in hardware.

A few numbers from the bench. Every one of them has a page with how it was measured and a photo of the run:

• 9,837 Mbit/s through an XGS 3300 with 1,032 connections in hardware, CPUs at 7.8 %
• 16 IPsec tunnels between two 3300s, 9,019 Mbit/s together
• the XGS 136 doing a gigabit in hardware with its CPUs at 0.4 %
• 3,186 GiB of TLS through both boxes without a single byte changed

The other models in the two families (XGS 87, 107, 116, 126, 138, 2100, 2300, 3100) will install, but I don't own any of them, so they're marked as untested and check their own ports on first boot. Someone is trying it on an XGS 126 right now. If you have one of these sitting around, I'd really like to hear how it goes. There's a short form on the site for that.

Before you try it: it's alpha (0.1.0-alpha.5.4) and needs OPNsense 26.7, and you'll be wiping the Sophos firmware. The easiest route is the image builder. The image has everything the driver needs, so the ports come up on first boot without internet. Installing on top of a running OPNsense works as well, but the box needs its kernel sources first, because without the driver you have no ports. The docs walk through it.


One more thing I'd rather say up front: by default it sends problem reports and a once-a-day count of the appliance. They're scrubbed of anything that identifies the box or your network, there's a banner about it until you choose, and the same page lets you set reports to manual or turn both off.

Site and docs: https://npuforge.org
All the models with their ports drawn: https://npuforge.org/devices/
Image builder (runs in your own GitHub fork): https://build.npuforge.org
Source, BSD-2-Clause: https://github.com/AbdelmonemAwad/os-xgs-npu

This is a personal, independent project. It has nothing to do with Sophos or Deciso and doesn't ship any Sophos software. Questions are welcome, I'll be around.

[BoNetTech]

Hi abdelmonem,

Thank you for sharing this, and for being so clear about the alpha status and what the installation involves. A few questions that matter for anyone who would use this in front of real networks:

1) Intrusion detection. If the coprocessor forwards a connection on its own after pf has passed it, I assume Suricata no longer sees the packets of that connection. Is that right? If so, is there a way to keep chosen traffic out of the offload, so it still goes through the IPS, and what happens to a connection that is already offloaded when the rules change?

2) Telemetry. You mention problem reports and a daily count that are scrubbed of anything identifying. Could you describe exactly which fields are sent, where they go, and whether the receiving side is documented? For many users the default matters, and an opt-in would be easier to accept than an opt-out.

3) Trust in the image. Since the image builder runs in the user's own fork and the driver sits in the kernel, how can a user check that the image matches the published source? Signed releases or reproducible builds would help a lot.

I do not own any of these units, so I cannot test, but I wish you good luck with the project.

Sergio

[abdelmonem]

Hi Sergio,

Thank you, these are exactly the questions I'd ask too.

1) Intrusion detection. You're right. The flow maker hands a connection to the coprocessor after its first data, so Suricata sees how each connection starts (the TLS ClientHello, enough for a JA4 fingerprint) and nothing after that. Inline IPS mode hasn't been tested with the offload yet, so for now I'd treat the two as either/or.

To keep traffic out of the offload:
- XGS: Settings has "Never offloaded": addresses, networks and ports, up to 64 entries, and either end of a connection can match. Those connections always stay with pf, and Suricata sees all of them.
- Any single connection can be sent back with "Return to host" on the Offloaded Flows page.
- Or turn acceleration off, and every packet goes through the host like on any OPNsense box.
Connections the Traffic Shaper or the Captive Portal names stay on the host by themselves.

When the rules change: every ruleset reload empties the coprocessor's whole flow table before the new ruleset loads. Every connection comes back through pf, and the flow maker only offloads again what pf still passes. pf keeps existing states across a reload, as usual, so a connection that has to stop needs its state killed. That's what the block-list and JA4 cuts do, and both are measured: https://npuforge.org/docs/measurements/lists-cut-2026-10-19/

2) Telemetry. The full list is here: https://npuforge.org/docs/problem-reports/ In short:
- A problem report carries the problem's own log lines (or a panic's backtrace) and a description of the machine: model, board revision, BIOS, CPU and memory, the coprocessor and NICs by PCI id, the OPNsense and kernel versions, and the installed commit of each plugin. IP and MAC addresses, e-mail addresses, serials, the UUID, the host name and domain, keys and long hex strings are removed before the report is even stored. The appliance is named by 12 hex digits of a hash of its serial and UUID. "View" on the page shows the exact text that would be sent.
- It goes to a small relay, reports.npuforge.org, a Cloudflare Worker whose code is in the repo (https://github.com/AbdelmonemAwad/os-xgs-npu/blob/main/relay/worker.js), and it files the report as a public GitHub issue. The relay keeps the sending address for two days for its rate limit, and nothing else.
- The daily count sends three things: a random ID the box made for itself (nothing read from the hardware), the model as SMBIOS names it, and the installed release. No address is kept, and count.py --show prints it without sending anything.

On the default: it's on because there are eight models I've never had in my hands, and a report is the only way I see what breaks on them. But you're not the first to say opt-in would be easier to accept, so I'm weighing it seriously.

3) Trust in the image. It isn't signed or bit-for-bit reproducible yet, but here's what you can check today:
- The OPNsense image underneath is the official one, checked against the SHA-256 its release publishes. The builder refuses to build from anything it can't verify.
- The driver isn't in the image as a binary. Its source is, at a recorded commit, and the appliance compiles it at first boot against its own kernel sources, so what ends up in your kernel is source you can diff against the tag. Every other component is fetched at a commit listed in the build's summary.
- The build runs in your own fork, from a workflow you can read, and its log stays in your account.
- Releases are published immutable, with GitHub's release attestation: gh release verify v0.1.0-alpha.5.4 -R AbdelmonemAwad/os-xgs-npu. Updates from the GUI fetch that tag's commit and confirm it with GitHub directly.

A signed build attestation for the image itself is a fair ask, and I'll look at adding it.

Thanks again for reading it this carefully.

Abdelmonem

[BoNetTech]

Hi Abdelmonem,

Thank you for the detailed and honest answers. Knowing that the IPS and the offload are, for now, either one or the other is exactly what people need to know before they choose, so it may be worth stating it clearly on the project page. The same goes for telemetry being on by default: an opt-in would make it easier to adopt this in places where reports must not leave the network.

Good luck with the project.

Sergio