An outback Queensland town is losing its only doctor and aged care residents are being told to move out immediately.