Common design formulas for gear tooth bending and pitting limits are based on fatigue properties for the material and heat treatment used, which is not the same as YTS or UTS. Tooth bending fatigue is usually based on a mean tensile stress at the root fillet, and pitting is usually based on the mean contact stress at the working surface of the tooth flank, for a given number of load cycles. The tensile and surface compressive stress fatigue limits for gear materials are always well below YTS, so you should never experience yielding in a properly designed gear under normal operating conditions. You can find some detailed explanations in the AGMA gear design manuals.