The website design should strike a balance between visual appeal and robot-friendliness. Admittedly, this is a huge challenge but it can be done.