Shadow Treasurer Jim Chalmers has called it a "vote of no confidence" for the Morrison Government's handling of the recession.