DVD raises a very good point - stress. Don't forget to rough the part out, stress relieve and then finish. That way you stand a chance of keeping to tolerance.
The problem with machining rubber is its flexibility and resistance to transfering heat. In the rubber industry they use liquid nitrogen or CO2 to freeze the material to harden it and then machine it quickly before it looses that hardness. A polished surface finish on cutting edges is crucial...