Defence minister says Americans told to leave base from where they are alleged to have launched drone strikes.