Take the belts off anyways one at a time and see which one makes the noise go away. You could have bad waterpump bearings, bad power steering pump, bad alternator bearings, bad belt, etc.