...I dont want to disable the default allow rule for the LAN or nothing will work.
Well, like with the LAN rules, you would use the VLAN rules to regulate traffic coming from VLAN net