Fitzroy defeat sees Rovers fall