Padma Shri recipient Adnan Sami claims he is being used as a pawn by people who have been attacking the government for conferring him with the honour.