More of the problem will be - as noted immediately above - the "calibration" of the partial joint "in the mind of the reader" who is looking at the ultrasonic test.
See, assume you have a full penetration "good" weld and pass a UT signal into that "good" weld joint. The reader - the person who actually is going to accept or reject the joint based on the signal that is NOT reflected from the "perfect" weld - is going to know what to expect, right? So, if one the second joint, there is a flaw, the reader is going to recognize the flaw - BECAUSE he or she sees he difference between a good joint and a flawed joint - and is going to respond as you expect. He (or she) is going to reject the "bad" joint reliably and is going to accept the good joints reliably, right?
Now, you are going to have an irregular, partially consumed, partially "open" unmelted joint that is going to be reflecting "weird" and "different" signals from what might be a flaw, and from what might be just an incomplete part of the weld where the two pieces of metal are not yet melted together. Remember, you are "planning" to be UT testing a 50% joint, but part of that joint is really going to be part 37%, 45%, part 50% and part 55%, and maybe even as much as 75%, so all of your weld is going to differ in different areas.
You might be seeing flaws, and have a good chance of cutting them out before you continue. But you might miss flaws. And you might be cutting out good weld as well.
If you are very, very concerned with inside weld quality, and have the budget, get a 50% X-ray. It will be more clear.
"Calibration" would in this sense, be a matter of taking identical weld geometry joints and doing a series of 10 50% welds, half with deliberate flaws and half with "perfect" welds. Then, UT testing each of the ten - twice with two different readers and two different UT specialists! - and see if you can tell the difference in the 20 readings between good and flawed welds.