Filipino workers in Taiwan are bearing the brunt of the anger after the death of a Taiwanese fisherman off Batanes last week.