I too have had flaky results using physical dynamics.
If you don't mind cheating, one way of creating a convincing, if not technically accurate, rack & pinion mate is as follows:
-Sketch a pitch circle on the pinion, and a pitch line on the rack.
-Sketch an arc on the rack, tangent to the...