I have the same issue happening. I am beginning to convert my home network over to opnsense. So naturally I dipped my toe in the water with a couple easy rules (pings) to make sure things work as I expect, before moving on to more complicated stuff.
And to my surprise, actually no, they're not working as I would expect - at least not with pings. As soon as any rule permits a ping to go through, that ping will remain in an allowed state until the opnsense system is rebooted, even after the original rule that allowed it is disabled or deleted.
And to my surprise, actually no, they're not working as I would expect - at least not with pings. As soon as any rule permits a ping to go through, that ping will remain in an allowed state until the opnsense system is rebooted, even after the original rule that allowed it is disabled or deleted.