Neighbors say the field has been vacant for years, but a company just started the work to build houses on the land.