A day after meeting Union Home Minister Amit Shah in Delhi, former Punjab Chief Minister Captain Amarinder Singh said that he is leaving Congress, but not joining the BJP.