Messages

Thank you for your answers.
I have only one docker host; but now when i create a bridge network for a stack i bind it to a particular subinterface of the host as per default docker listen to all interfaces. For compose stacks i put the webserver in a macvlan (if it's more convenient) or a bridge in the dmz vlan and the others containers in another vlan network.

My authentik stack authenticates apps for lan or internet users and also apps only accessible from lan. For now its listening interface is in the lan but i wonder if i should move it to dmz. I'm also trying to force all apps through caddy and authentik to have an authentication and use the same hostname to access it whether i'm inside or outside.

The same goes for caddy. If i understood correctly caddy process is listening on all interfaces. But i think i don't really get the path of a packet.
Caddy listen on ports 80&443.
If the request comme from outside, it arrives on wan interface which caddy listens to. Then it processes the request to upstream server or forward the request to authenticating server depending on the authentication type (oauth or proxy).
For this request to be actually effective you need a firewall rule on wan interface to pass the pack to "this firewall".
If the request comes from the inside on the interface (LAN for now) where the user is. Then the request is processed as in the first case.
For this request to be actually effective you need a firewall rule on lan interface (or the interface on vlan where users resides) to pass the pack to "this firewall".
So if i'm correct when the reverse proxy is on the firewall itself you can't really put it in the dmz. The only thing i can do is move authentik to dmz and eventually change my users vlan from lan to another one.
Another thing i need to take care of is that the app with oidc provider need to contact the authentik server. For now as authentik is in the lan i have a firewall rule passing request from apps in the dmz with oidc provider to authentik server (not sure about that one as i don't which container of the stack actually makes the request).
For example i have a jellyfin docker in dmz which can't authenticate without such a rule (it's only one container).
But i have a nextcloud-aio (13 containers in a vlan bridge and the apache one in dmz bridge) which can authenticate without the rule.

Forget about that my authentik stack is still in a bridge network that is not bound to a particular interface so it listens to all. I consider this as a security hole and i have to modify my authentik bridge network so that it listens to only one interface.
I guess it's the danger to have a docker host with several interfaces.

When i setup opnsense i created several vlans.
My main server is unraid. It has its interface eth0 on lan and subinterfaces in vlans.
I host several applications (immich, nextcloud, authentik, homeassistant and many others).
The majority of these applications are installed through portainer stacks.
Some are accessible from outside, others are restricted to lan ip.
I connect to these apps through caddy plugin.
When i first setup caddy (previously i was using npm docker on unraid) i followed the documentation: (firewall wan+lan rule destination: this firewall dst ports: 80&443) 
I placed devices in according vlans when i setup dnsmasq dhcp but I never took the time to move the apps on another vlan.
Should i really do that knowing all apps are behind caddy?
 
Should i change the firewall lan rule to dmz vlan ? afaik the best practice is to put the reverse proxy in dmz.
Authentik is used to give access to almost all my apps (through oidc, ladp or proxy provider) should i move it to dmz ?
I think you would guess i'm a little confused.

 

Did you see the device actually reboot when you did the nvram update? It may have soft rebooted (re-booting just the OS)?
IIRC, The eeupdate util does not work on freeBSD.

Your device is the 1st reported one of such issue.

From all the util commands you posted, the devices do appear to be there, and not sure why one or more may have moved locations on PCI bus. Maybe perhaps you have other issue with that hardware. A bad solder joint can create the gremlin, etc.

I will suggest that the update procedure include removing power for 1min after a device shutdown. Then power it back up. But noted, you should not need to do this as long as the device actually is rebooted.

Now that you mention it, all my reboots (and there have been at least 5) prior to unplugging the power cord to move the device were done with the webgui and as i was not in the same room i didn't see it rebooting. Does a reboot initiated from the gui is a hard reboot ?

From what i posted you clearly see that device with @mac ending in 32:00 is igc5 but when i had the problem it became igc4. And at this time device with @mac ending in 31:ff was not detected (it's normally igc4).
All came back in order after power reset.
I mentionned eeupdate as it was part of the firmware package i found. I first intended to use it in EFI shell which i didn't managed to do. I am still struggling to make a bootable efi usb media with my mac.

Finally i managed to flash all the devices.
I guess a reboot is not enough despite what the nvmupdate manual says.
I went to the basement and took my appliance to try a bios update.
When i plugged it the missing interface was back.

For the record i didn't managed to do the update. Making bootable usb with efi shell for flashing with a mac is really a pain in the ...
I remember having struggled for flashing an hba.
Anyway in the iso file provided with firmware there was also eeupdate utility (like nvmupdate but for efi) and the bin files for the i226v devices (type 125c version 2.32)

Code Select Expand

dmesg | grep EEPROM
[1] igc0: EEPROM V2.32-0 eTrack 0x80000422
[1] igc1: EEPROM V2.32-0 eTrack 0x80000422
[1] igc2: EEPROM V2.32-0 eTrack 0x80000422
[1] igc3: EEPROM V2.32-0 eTrack 0x80000422
[1] igc4: EEPROM V2.32-0 eTrack 0x80000422
[1] igc5: EEPROM V2.32-0 eTrack 0x80000422

i have no more pci0:7:0:0 device.
previously it was igc5

Code Select Expand

pciconf -l igc5
igc5@pci0:7:0:0:    class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000

sudo pciconf -lbcv ix1@pci0:7:0:0
Password:
igc5@pci0:7:0:0:    class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor    = 'Intel Corporation'
    device    = 'Ethernet Controller I226-V'
    class      = network
    subclass  = ethernet
    bar  [10] = type Memory, range 32, base 0x80500000, size 1048576, enabled
    bar  [1c] = type Memory, range 32, base 0x80600000, size 16384, enabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks
    cap 11[70] = MSI-X supports 5 messages, enabled
                Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
                max read 512
                link x1(x1) speed 5.0(5.0) ASPM disabled(L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 a8b8e0ffff063200
    ecap 0018[1c0] = LTR 1
    ecap 001f[1f0] = Precision Time Measurement 1
    ecap 001e[1e0] = L1 PM Substates 1

i installed pciutils.

Code Select Expand

sudo lspci | grep 226
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
03:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
05:00.0 Non-Volatile memory controller: Silicon Motion, Inc. SM2263EN/SM2263XT (DRAM-less) NVMe SSD Controllers (rev 03)
06:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)

Yes the switch led lights on.
I initially had only 1 switch near my router and not enough free ports. So i decided to use 1 port fort lan and a lagg to 5 vlans. The first switch is 8 sfp ports switch with both fiber modules and rj45 ones. The RJ45 ones are 2,5G (to match my router)
I added another switch i had lying around but i want to replace it (unifi 48 port poe) it's hungry and noisy and only 1Gbps ports (except 2 sfp+)
i'm running opnsense 25.7.6. I think the igc5 update has been made within 25.7.5 version.

lspci : i suppose i need to install a package.

Code Select Expand

pciconf -lbv | grep -A 4 device=0x125c
igc0@pci0:1:0:0: class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet
--
igc1@pci0:2:0:0: class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet
--
igc2@pci0:3:0:0: class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet
--
igc3@pci0:4:0:0: class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet
--
igc4@pci0:6:0:0: class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet

Code Select Expand

ifconfig | grep igc
igc0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
igc1: flags=1008a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
igc2: flags=1008b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
igc3: flags=1008b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
igc4: flags=1008802<BROADCAST,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
laggport: igc2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: igc3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
vlan: 35 vlanproto: 802.1q vlanpcp: 0 parent interface: igc0
None of igc4 or 5 is actually not plugged.

Code Select Expand

cat nvm.cfg
CURRENT FAMILY: 1.0.0
CONFIG VERSION: 1.20.0

; NIC device
BEGIN DEVICE
DEVICENAME: Intel(R) Ethernet Controller I226-V
VENDOR: 8086
DEVICE: 125C
SUBVENDOR: 8086
SUBDEVICE: 0000
NVM IMAGE: FXVL_125C_V_2MB_2.32.bin
EEPID: 80000422
RESET TYPE: REBOOT
REPLACES: 80000284
i used this file for flashing both igc 5 and igc 4

For igc4 here is the .rollback.cfg:

Code Select Expand

sudo cat A8B8E00631FF/.rollback.cfg
;File generated automatically by NVMUpdate tool
CONFIG VERSION: 1.26.0

BEGIN DEVICE
DEVICENAME: Intel(R) Ethernet Controller I226-V
VENDOR: 8086
DEVICE: 125C
REPLACES: 80000422
EEPID: 80000284
NVM IMAGE: A8B8E00631FF/A8B8E00631FF.bin
IMAGE DOWNGRADE: TRUE
RESET TYPE: POWER
END DEVICE
and the flash log

Code Select Expand

cat flash_igc4.log
Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.43.20.0
Copyright(C) 2013 - 2025 Intel Corporation.

./nvmupdate64e -b -l igc4.log -m a8b8e00631ff -u -c nvm.cfg

Config file read.
Inventory
[00:005:00:00]: Intel(R) Ethernet Controller I226-V
Alternate MAC address is not set.
Flash inventory started.
Shadow RAM inventory started.
Shadow RAM inventory finished.
Flash inventory finished.
OROM inventory started.
OROM inventory finished.
Update
[00:005:00:00]: Intel(R) Ethernet Controller I226-V
Creating backup images in directory: A8B8E00631FF.
Backup images created.
Flash update started.
NVM verification started.
Shadow RAM verification started.
Shadow RAM verification finished.
Flash verification started.
Flash verification finished.
NVM verification finished.
Flash update successful.
Device update successful.
Update security revisions
[00:005:00:00]: Intel(R) Ethernet Controller I226-V
Skipping update minimum security revisions.
Update VPD with VPD template
[00:005:00:00]: Intel(R) Ethernet Controller I226-V
Skipping VPD update with VPD template.
Checking update availability for next tool run.
A reboot is required to complete the update process.
the same for igc5:

Code Select Expand

sudo cat A8B8E0063200/.rollback.cfg
;File generated automatically by NVMUpdate tool
CONFIG VERSION: 1.26.0

BEGIN DEVICE
DEVICENAME: Intel(R) Ethernet Controller I226-V
VENDOR: 8086
DEVICE: 125C
REPLACES: 80000422
EEPID: 80000284
NVM IMAGE: A8B8E0063200/A8B8E0063200.bin
IMAGE DOWNGRADE: TRUE
RESET TYPE: POWER
END DEVICE
and the log from flash:

Code Select Expand

cat flash_igc5.log
Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.43.20.0
Copyright(C) 2013 - 2025 Intel Corporation.

./nvmupdate64e -b -l flash_igc5.log -m a8b8e0063200 -u -c nvm.cfg

Config file read.
Inventory
[00:007:00:00]: Intel(R) Ethernet Controller I226-V
Alternate MAC address is not set.
Flash inventory started.
Shadow RAM inventory started.
Shadow RAM inventory finished.
Flash inventory finished.
OROM inventory started.
OROM inventory finished.
Update
[00:007:00:00]: Intel(R) Ethernet Controller I226-V
Creating backup images in directory: A8B8E0063200.
Backup images created.
Flash update started.
NVM verification started.
Shadow RAM verification started.
Shadow RAM verification finished.
Flash verification started.
Flash verification finished.
NVM verification finished.
Flash update successful.
Device update successful.
Update security revisions
[00:007:00:00]: Intel(R) Ethernet Controller I226-V
Skipping update minimum security revisions.
Update VPD with VPD template
[00:007:00:00]: Intel(R) Ethernet Controller I226-V
Skipping VPD update with VPD template.
Checking update availability for next tool run.
A reboot is required to complete the update process.

I didn't pay attention pci devices changed. So i guess after the reboot the appliance didn't list the same pci devices.

Hello,

I have a very strange issue.

My device is a generic N100 with 6 i226V.
I use 4 of them:
igc0 is wan
igc1 is Lan
igc2&3 are tied in lagg0 which is parent interface of my vlans.

igc4&5 are unused.
all i226 devices had 2.13 firmware.
I flashed igc5  with 2.32 and rebooted.
I didn't used it. I just plugged it in a switch and saw in log igc5 link state changed to up
Then i flashed igc4 and rebooted just like for igc5.
After the reboot igc5 is no more present and igc4 has the hardware address of igc5.
When i plug igc4 (as labelled on the device) nothing happens
When i plug igc5 (as labelled on the device) i see in the log :igc4: link state changed to UP

dmesg doesn't show any igc5 device and igc4 with address of the previous igc5.

i can't figure out why this would happen. When you flash the only reference to the device in the command line is the hardware address. You can't make any mistake.
After the flash a folder is created named after the hardware address. I have 2 folders as expected with the correct addressses.

Curiously in the bios i didn't anything related to hardware network devices.

What would you do in such case ?

upgrading from 25.7.5 and curiously i didn't have to apply the upgrade twice.

pkg was upgraded yesterday and don't remember having done this.

Code Select Expand

2025-10-23T12:10:19 Notice pkg-static bash upgraded: 5.3.3_1 -> 5.3.3_2
2025-10-22T18:27:36 Notice pkg pkg upgraded: 1.19.2_6 -> 2.3.1
2025-10-15T13:50:27 Notice pkg-static opnsense upgraded: 25.7.4 -> 25.7.5

thanks.
I should be ok i have a kvm plugged in my router.

I quickly read the topic and have not a clear idea of your whole config.
does your opnsense have the same ip as your former router ? if not  did you update the trusted proxies section in home assistant configuration.yaml
did you reboot all your machines involved in these communications ?

I flashed igc5 (unused) with 2.32
Thanks to you and meyergru ( i have the exact same interface and version) it went well.
Need to test. I guess the easiest way is to assign it to LAN.

Can you flash an interface in use ? My WAN need to be tied to igc0.
If flashing 2 interfaces tied in a lagg device i suppoose the best is to flash the 2 interfaces before rebooting.

I've not flashed for now. I don't even know what firmware version i have.
For now lan drops are solved since the change of my patch cables.

I have still an issue but i don't know if it's related.
My issue is with suricata and netmap. If i enable ips on lan and vlans interfaces i have frequent connections drops due to netmap errors.
I disabled ids and ips (currently using crowdsec and testing q-feeds).

edit: apparently i have 2.13
'dmesg | grep EEPROM
[1] igc0: EEPROM V2.13-0 eTrack 0x80000284
[1] igc1: EEPROM V2.13-0 eTrack 0x80000284
[1] igc2: EEPROM V2.13-0 eTrack 0x80000284
[1] igc3: EEPROM V2.13-0 eTrack 0x80000284
[1] igc4: EEPROM V2.13-0 eTrack 0x80000284
[1] igc5: EEPROM V2.13-0 eTrack 0x80000284'

I changed the patch cables and so far i had no more issues.
I try to move the cable and it seems better locked than the old one.
The switch is a chinese one (sks8300-8x). I found a new firmware.
On this switch i have several sfp+ modules some are fiber modules with lc connectors and some have RJ45 connectors.
The Lan is the one that had troubles. Speed is displayed at 2,5G
The two in lacp for vlans are displayed at 10G on the switch.

Code Select Expand

Ethernet1/0/1 usw48         Enabled fiber-auto 10G/Full Disabled auto
Ethernet1/0/2 vlans-router Enabled fiber-auto 10G/Full Disabled auto
Ethernet1/0/3 vlans-router Enabled fiber-auto 10G/Full Disabled auto
Ethernet1/0/4 lan-router Enabled fiber-auto 2500M/Full Disabled auto
Ethernet1/0/5 godzilla-eth1 Enabled fiber-auto 10G/Full Disabled auto
Ethernet1/0/6 godzilla-eth0 Enabled fiber-auto 10G/Full Disabled auto
Ethernet1/0/7 uplink-et3 Enabled fiber-auto Link Down Disabled auto
Ethernet1/0/8 uplink-et1 Enabled fiber-auto Link Down Disabled auto
Port-Channel1 vlans         Enabled Auto/Auto 20G/Full Disabled auto
1/0/1 is dac cable to unifi switch
1/0/2 is RJ45 module to i226v router
1/0/3 is RJ45 module to i226v router
1/0/4 is RJ45 module to i226v router
1/0/5 is sfp fiber module to sfp on server
1/0/6 is sfp fiber module to sfp on server

1/0/7 & 8 are down i moved these module on the unifi switch (links to other floors)

On router interfaces are ethernet 2,5G
SFP+ are on switch.

Thank for your answer.
Finally seems to be my patch cable.
Yesterday i moved my router to change a defective usb cable (for a kvm hid).
This morning i plug the new usb cable and noticed that led on the switch interface attached to lan (igc1) went off. And when i slightly move the cable it goes off. The interface are sfp+ 2,5G ethernet modules.
These patch cables are way to rigid and i guess not very good quality.
I ordered new cables for all my router interfaces.