The Federal opposition is accusing the government of not doing enough to protect the community after two former immigration detainees were charged in two separate incidents.