"I'm not trying to slut shame anybody or anything, but nobody ever got asked to move in unless they had slept with him," Madison said during the Call Her Daddy podcast Wednesday