what is the failure mode I should be expecting... i.e. reduced life, catastrophic bearing failure, tooth wear...
All of the above.
My friend Wayne spent much of WW2 rebuilding supercharger drives for B29 aircraft.. They used a planetary speed increaser; I think the ratio was around 1:15, but certainly not much higher. All the parts were machined to super precision standards, the best possible at the time.
Still, they had a substantial failure rate. Many/most of them just blew up, leaving no reusable parts. Wayne was kept busy salvaging what he could. If the housing survived, he could build a usable replacement, using a crate of brand new replacement gears, which he selectively fit at assembly, just as they were doing on the production line back home.
Since those days, gear manufacturers have made substantial improvements in tooth form generation, and auto transmission manufacturers have made great strides in load sharing with very subtle adjustments in flexibility of gear carrier structures and such.
... but you are asking a lot of even modern technology.
I'm wondering how much you could do about controlling the overspeed while/before it's happening, e.g. by a little DC injection into the motor coils while it's nominally 'off', and perhaps by direct mechanical braking of the inertial load without involving the gearbox at all.
You could get fantastic free help here, but there are obvious commercial reasons why you would not wish to share the details of your problems in a global public forum like this.
Given that, I suggest you engage in some private development with your gearbox supplier, using them as a partner, not an adversary.
Mike Halloran
Pembroke Pines, FL, USA