Sharon Osbourne has claimed she knew about Simon Cowell and Dannii Minogue's affair during their time on The X Factor, but insists the secret tryst is not the reason she quit the talent show.