Sigma delta clock drift

I'm have a sigma delta converter that is driven by a clock that is prone to drift with temperature change. This causes a change in the output rate of the conveter. I'm wondering if the readings it gives me are still valid? Does the timing of the circuit effect the integrity of the output?