GPGPU parallelization is disabled by Abaqus although GPU is OK

[stig911mah]

Hi,
When I open Abaqus 2017 and enter the job manager, I see that the "use GPGPU acceleration" is enabled. So, I can mark/unmark it. However, Following this tutorial, when I enter the job manager, that GPGPU parallelization is disabled and it is totally grey.
I don't know what happens?

Does anybody know that?

 

[Dave442]

I think GPGPU is only supported for Abaqus/Standard. That was the case in previous releases.

You should check the documentation.

 

[stig911mah]

OK. I ran a sample cae file which uses the standard solver. I have uploaded a 5MB video of what did happen. Please see the video at

http://www.mediafire.com/file/ja98qp6qmc8q9an/Rec_0002.mp4

As you can see, I set the GPGPU to 1 since there is a single M2000 (700 cuda cores) device is available. After submitting the job, the GPU core and memory clocks go high and that means the GPU is working, but the GPU load is 0!
After some seconds, the clocks become idle which is strange. Also, one CPU core is utilized as you see in the task manager.

So, what do you think about that?

 

[Dave442]

Your original question related to using GPU for an explicit analysis - I noted that GPU is only supported for standard (implicit) analyses. I think that explains your original problem.

I don't use GPUs so cant help you with your new problem.

If you are having difficulties using a GPU when running standard (implicit) analyses you should get in touch with Simulia technical support. You can definitely use GPUs with standard analyses so I would guess if you are having problems that something may not be configured correctly or there is some licensing issue. Do you need additional tokens/licenses when using GPUs?

 

[stig911mah]

Do you mean I have to obtain a separate license for the GPU acceleration? How can I apply for that. I didn't see such thing on the abaqus website. Maybe I missed something.

 

[Dave442]

I dont know, it was a suggestion.

http://deviceanalytics.com/abaqus-token-calculator

according to the information on that site you need 6 tokens to run an Abaqus/Standard job on one GPU.

 

[stig911mah]

OK. I know you haven't tried that but it seems that we are moving forward.
I asked our administrator and he said that in the license file, there are

FEATURE gpgpu ....
FEATURE gpus ....

However, there is no token number in those features. He said that in addition to some HOSTID and some dates, there are two ck parameters for them.

Any idea? He said that they had requested for GPU license but he doesn't know what is the token!

 

[Dave442]

You need a license to open and use abaqus/cae

Then you need analysis tokens to run analyses. 5 tokens are required to run an analysis on 1 core. To run in parallel you need to purchase additional analysis tokens. According to the site I linked in my previous post the equation is INT(5 X N^0.422). So, to run a job in parallel across 8 cores you would need 5 * 8^0.422 = 12 analysis tokens. Also, a GPU counts as 1 core so you need 5 * 2^0.422 = 6 analysis tokens to run a job on your GPU. This is explained better on the site I linked earlier.

When i check the license status on my license server I can see how many licenses and analysis tokens are available for use. However, the tokens are listed as licences.

 

[Dave442]

if you check the license status on your server you should see:

"Users of standard: (Total of X licenses issued; Total of Y licenses in use)"

You need X-Y to be >= 6 to run your job on one GPU.

 

[stig911mah]

So, I opened Abaqus and then opened my cae file. I perform a license inquiry and noticed that

"Users of cae: (Total of 32 licenses issued; Total of 1 licenses in use)"
cae v70,0, vendor ABAQUSLM
....
"Users of gpgpu: (Total of 16 licenses issued; Total of 0 licenses in use)"
"Users of gpus: (Total of 16 licenses issued; Total of 0 licenses in use)"

Then I opened the Job manager and went to parallelization and selected GPGPU and set it to 1. After submitting the job, I again checked the server status and saw

"Users of cae: (Total of 32 licenses issued; Total of 1 licenses in use)"
cae v70,0, vendor ABAQUSLM
....
"Users of gpgpu: (Total of 16 licenses issued; Total of 1 licenses in use)"
gpgpu v70,0, vendor ABAQUSLM
....
"Users of gpus: (Total of 16 licenses issued; Total of 0 licenses in use)"


Some seconds later, when the GPU core and memory clock becomes idle, I checked that one CPU core was utilized. Again, I checked the server and saw that still gpgpu license is used by 1 (me) and gpus is 0 and the cae is still 1.

What can be understand from that? I haven't ever tried to contact the Abaqus because I don't know if they answer to my question!

 

[Dave442]

Can you submit your job again and then perform a license inquiry and check the status of:

*users of standard

Also can you check the .dat file for any warning or error messages.

 

[stig911mah]

OK. I did what you said and saw the same thing as before.
I asked a colleague who works with abaqus. I am not an expert in designing with abaqus. He gave me another example and I can now see something new!
In this example, the GPU has been enabled and the core and memory clocks remain high during the simulation. It took 20 minutes for the GPU to finish the job. I ran it again with 4 cpu cores and it took 10 minutes!!!!!
A total catastrophe for the M2000!!

As I look at the GPU status during the run, I see that the clocks are high (not idle) but the GPU utilization is not always full. That means, at some point in the program, a kernel is shortly run by GPU.
So, I think it all depends on the problem. Maybe changing the settings, e. g. partitioning method and others thing can help GPU. But I don't see a setting to say Yup... this can dramatically affects the GPU performance.

See the picture here.
The cae file can be accessed from here. It is 1 MB.
Any thought?

 

[Dave442]

from what I've read, the benefit you see from running on GPUs is very dependent on both the type and size of your model.

For a relatively small analysis (10 mins on 4 CPUs) it seems reasonable that GPUs would take longer due to data transfer between system/GPU memory?

You should get on the phone to a Simulia rep and figure this stuff out.