A man accused of supplying drugs to Liam Payne before his death has insisted he "never" gave him narcotics but admitted they consumed such substances together on two occasions.