Hilary Duff had the most relatable response to rumors that her marriage was “on the rocks”