Prime Minister Manmohan Singh is unlikely to go to Colombo for a Commonwealth summit later this month with a divided Congress core group on Friday not clearing the visit in view of strong opposition in Tamil Nadu.