The actor quipped that he "can be kind of an a‑‑hole and people are like, 'Isn't that charming?'"