I am not sure of the actual relay you have, so my comments apply to the 239 relay, as referenced by Jbartos.
The method of system grounding at your plant is very significant. The ground fault protection system that should be installed will depend on how much ground fault current is available. So, what sort of grounding system have you got and what is the available ground fault current?
If you have high resistance grounding then you will have very low levels of ground fault current available (and the fault current will essentially be the same irrespective of where the earth fault is). For this, you should use the GE core balance CT, which has a ratio of 50:0.025A.
If you have a system with low resistance grounding that limits the ground fault current to 400A then you can use a 5A core balance CT and a setting of 10% of the motor full load current would be reasonable. In this case, you could use a core balance CT of 100:5A or 200:5A ratio. If you can not get a 200:5A CT, then you could possibly use a 100:1A core balance CT, but in programming the relay, you would need to programme the ground fault CT ratio as being 500:5A and the ground fault setting appropriately.
You do not have to use a separate core balance CT if you can live with a ground fault setting that is 20% of the phase CT. For this, you connect the residual current from the phase CT's into the earth fault CT input.
What sort of ground fault setting do the other motors in your plant have? Are they all the same, irrespective of motor size or are they a fixed percentage of the motor full load current? It may pay to follow what is already being done (provided it is OK!).
Hope this helps.