One thing I ran into that confused me is that the policy has a 'from' action (at the top) and a 'to' action (at the end). The top action is by default set to 'Drop'. So, if you change the bottom one to 'Drop' you are in effect saying: if the action is 'Drop' change it to 'Drop'.Make sure that you create a policy to change (Action, top) 'Alert' to (New Action, bottom)'Drop'.
Ok I missed that . If you go to Administration | Rules . There Filters drop down and chose Action: Action/Drop, do you get your rules there appearing?This is to verify, not to set. My policies are slightly different, maybe you can try that: Action (at top) is Alert, Drop.New action (at bottom) is Drop.
Any clues in /var/log/suricata/latest.log ?