Even though the blank is square you can see from your stress distribution that the results are axisymmetric by nature. Most of the work carried out on the blank occurs within the die area and the back end of the blank (the square end) doesn't seem to do much, and also doesn't move much as the punch is moved down. That may be why changing friction doesn't have much of an effect as the blank seems to be effectively restrained within the holder.
Explicit does work better when you have contact and elastic plastic material as the stability conditions aren't rigorously enforced. In the static analysis I ran there were problems reaching the final time with not enough increments and so on, but if you include contact controls and the right time step then it runs and runs much quicker than an explicit analysis.
In your model you have shell elements. How these behave when you have stresses exceeding yield through part of the thickness I'm not sure of. In the 2D axisymmetric model I ran, I could model the actual thickness and non linear stress distribution through that. Perhaps changing the elements in the 3D model to brick elements might produce better results, though this would be expensive computationally if you include enough elements through the thickness.
If your experimental results differ significantly for the different size blanks then use a 3D model but as a first step, and as a means of understanding the general behaviour, I'd recommend using a 2D axisymmetric model first. This can be generated relatively quickly from your existing 3D geometry. Using a 2D approach means you can assess the sensitivity of the results to changing various factors, such as element types, friction,etc. and also get results in a matter of minutes. Then try running the full kit and caboodle 3D model.