The Kerala Police has more to worry about the growth of Left Wing extremism in Kerala after suspected Maoists attacked a private quarry-cum-crusher in Kannur district early on Friday.