Long ago, my employer had a crane with wrecking ball.
Mostly, they used that wrecking ball with trackhoes, busting up ground-level concrete.
Swinging side to side with the crane is fine, unless you miss the target, as that wrecking ball swings out but also swings back. (Loads swinging too far from a crane is not good, wrecking balls hitting the boom is not good!)
Dropping the wrecking ball sounds good. But the way it worked, if you just dropped it and that's all you did, it would spool out a bunch of cable that would then get tangled. So you had to engage a brake immediately after it hit to keep that from happening. And if you engaged that brake a 1/10 of a second too soon, you snagged yourself a wrecking ball prior to hitting the target, which was hard on both crane and operator.
Anyway, the whole premise sounds simple and looks simple in the cartoons but wasn't that easy to do in practice.
I expect most crane operators have never used a wrecking ball to any extent, either.