I'd recommend a radial shaft seal (rss) with elastomeric coating since they can follow the expansion of the housing better. Use bore tolerance d2 H8 or if your housing/recieving bore thin-walled is, then F8. The manufacturer are obligated, according DIN 3760, to produce a rss within tolerances...
you can also visit simrit.com and find some tools about how to choose the proper material and design a adequate groove for your application. The small "mud" particles are burned oil parts "thanks" to very high tepmerature or an inadequate oil-quality or just simple dirth...