The late-night host also says waiting to say something about the impersonation "was a mistake.