Questions for the experts!! 1

[qwicker]

Hi all you expert Ansys users,

I was hoping to get some feedback on one of my models that I am running. I have a working model, and I am confident in the results. However, I have many models to run and the run times are quite long (in my opinion). I was looking for some thoughts, comments, and advice.

I have a very Large Deformation, nonlinear analysis with about a dozen contact pairs. I have a high level model for rubber (Ogden 3rd order). There are metalic parts encased in a large rubber boot (as a cable bend stiffener). The model has about 200,000 elements. My model takes approximately 2 weeks to solve. This is my hardware configuration:
2x quad core Xeon 3.2Ghz
32GB RAM (fastest available)
15,000rpm SAS drives in RAID 10 (is that a problem?)

I have the full blown Ansys license. Available is the HPC license, VT accelerator, SMP, DANSYS, etc.

I currently am using shared memory parallel with 6 processors (I found that 6 is better than using all 8).

Even thought the model *should* fit incore, the pivoting option keeps activating which (I believe) is causing an out of core solution. For the shared memory Sparse Solver I use the BCSOPTION,,incore. For DANSYS and the distributed solver I issue the DSPOPTION,,incore command. However this always fails in solution because of the pivoting and exceeding the amount of predicted memory!

Does anyone have input, comments, suggestions, quetions?

Thanks!!!!!!!!!

 

[qwicker]

I also forgot to mention, I keep seeing all the LN files being written, read, etc. Is it correct that this means it definitely is running out of core? Those files should never be written to the HDD if it is incore, correct?
Thanks

 

[cbrn]

Hmmm... strange...
It's not useful to mention that you're using a 64-bits architecture, but which Operating System? Are you sure there isn't some setting of the OS which prevents ANSYS from using a sufficiently large contiguous block of memory?
Try adding the ",performance" key in the BCSOPTION command, this will give you a really verbose output and perhaps it will help you finding your way out...
Also, before launching the solution, exit from Ansys and re-enter by manually setting a "database size" as small as possible, just enough to keep your database file into memory, and a "total workspace memory size" as large as possible.
I know the latest versions of ANSYS generally do a better job than the user in managing mem sizes dynamically, but I found that in some cases it does not.
If it doesn't become unstable because of too strong a non-linearity of the model and because of the materials' constitutive laws, you could also try to solve with one of the most evolved iterative solvers (most obvious is PCG): their memory management is globally far better.
As a "last chance", due to the huge memory you have, you could set up (from the Operating System) a "virtual drive" completely into RAM and run the whole ANSYS from there. I know, it's tremendously uneffective, in normal cases (as the whitepaper from Ansys Inc. correctly explains), but it seems to me that for some reason yours is all but "normal"... ;-)

Hope this helps in some way...

Regards

 

[qwicker]

I was reading through the many papers Ansys has available. It says that ALL models in which the pivoting option activates are forced to run out-of-core. It would appear that it has something to do with Ansys not being able to properly estimate the needed memory so it goes to the HD just in case...

Also, i have done the performance option; no useful info really. I have also set my memory as high as I can and it still goes out of core...

Also, I am running Vista x64 (I know, I know... but according to some benchmarks and stuff I have read, it is no different than XPx64). I am also using v11.0sp1.

I think I may try that virtual drive thing...

 

[Stringmaker]

What type of "pivoting" is going on exactly? I would consider the following:

1) Do you have an overly constrained model?
2) Can you substructure part of the model?
3) Perhaps the AMG solver is faster in this case?

I would look at the NLDIAG command and see if any of the outputs can be of help to you. Also, posting a snippet of your output would allow forum users to make better suggestions as well.

 

[qwicker]

Thanks for the response. I am near certain that I do not have an over constrained model.
Also, I am unfamiliar with substructuring.
I forgot to enable file output this time, so i had to take a screenshot of the output window. Please check that to see if there is anything strange going on.

 

[qwicker]

A second output screenshot

 

[Stringmaker]

Without seeing the entire output file it's hard to say what you could do to speed things up. Are you truly utilizing your HPC capabilities. Your output indicates that your stiffness matrix is unsymmetric. I don't think the unsymmetric solver within Ansys is multi-threaded. Is your problem ill conditioned? Do you get element shape warnings?

 

[qwicker]

I do get some element shape warnings. 88 of about 250,000 elements violate SWL... I didnt think it was a problem since its so few. Yes, my system matrix is unsymmetric (which i set for the contact). I have found that unsymmetric yields better convergence... Does it really matter? If my problem is ill-conditioned, I dont know how to make it better. I really thought about the simulation and how to define boundaries and loads. I do have a decent amount of contact pairs though and a few different hyperelastic materials.


For this analysis, i am using the sparse direct solver (not the distributed). I am running in a SMP mode also. I do not seem to have the parallel performance module installed (or the license is MIA... I have to check).

 

[qwicker]

Stupid question... I have the mechhpc license.. do I need something else to use the AMG solver? It isnt even listed as a solver in the solution controls area. I have distributed sparse, but not AMG or any of the others...

 

[qwicker]

And how do I change the matrix to symmetric? Sorry for the simple questions...

 

[Stringmaker]

You need to have Distributed Ansys installed.

Issue the following:

/SOLU
EQSLV,AMG
NROPT,FULL

I don't know why you would need to use an unsymmetric solver. I don't see your problem as being highly path dependent...but I could be wrong.

 

[qwicker]

I got the AMG solver (I actually am a moron because I did know how to activate it I just forgot...) Anyways though, how do I make the problem symmetric? When i initialize the solution, it says
GLOBALLY ASSEMBLED MATRIX . . . . . . . . . . .UNSYMMETRIC
which I assume needs to be symmetric. I have the friction stiffness matrix set as symmetric (is this what I am looking for?) Thanks

 

[qwicker]

OK, i got that symmetric thing fixed. However it turns out i cant use AMG anyways... i get an error that it cant be used with 180 series elements with mixed u-p formulations so it switches to the distributed sparse instead. I can can post my output later if need be. I will cross my fingers with this analysis! Thanks

 

[Stringmaker]

Friction is not always a symmetric phenomenon. Do you REALLY need friction in this model? Friction typically takes 1.5X the number of iterations to converge vs. without. Another option you may want to consider if needed is the "rough" contact behavior if you want no slippage to occur between two parts. I would be skeptical of the stress at the surface where this behavior occurs if used however. Beware of that.

Have you been to an Ansys nonlinear or contact class? If not and you feel it'd be beneficial I'm sure you could make a case to your manager based on this experience alone and the # of hours it could have potentially save you.

 

[qwicker]

Well i decided that i didnt REALLY need friction so I removed it. However its worse converging now! I dont understand really. I get good convergence with my old model which ran on the sparse solver out of core. It took forever, but it always converged fast. Now I get very fast iterations, but terrible convergence.

I wish I could just "submit" my problem and have someone look at it..

 

[Stringmaker]

If there is no IP constraints from allowing you to do so then upload it here. I'd be more than willing to take a look at it over lunch sometime. If posting for the public IS a problem then I'd recommend consulting your ASD. To obtain maximum file compression I would do the following:

CDWRITE,DB,,, ! Write database information only

Zip CDB file and post to site

 

[qwicker]

Thank you so much. I am modifying a few parameters right now. I believe my problems have to do with my cotnact, specifically the stiffness factors. I am running a solution now with low (0.01) factors and I am achieving quick convergence. When it is done I will check the results and adjust the factor.

Also, as a result of these modifications i am sucessfully running incore with the DSPARSE solver.

 

[Stringmaker]

Be careful when adjusting contact stiffness. You may see excessive contact penetration with such a low normal stiffness. If you feel excessive contact stiffness may be your problem in the first load step set keyopt 9 = 2 or 4 for the contact elements. This ramps stiffness and aids in convergence.

 

[cbrn]

Hi,
in addition, if problem is convergence and not calculation speed "per-iteration", then you'd better check that you do have "adjust contact stiffness -> at each equilibrium iteration" set.
If not, then the contact stiffness is evaluated at the beginning of the substep's solution and never updated through the equilibrium iterations, which can easily be a severe problem with hyperelastic materials.

In addition to Stringmaker's comment, sometimes I've achieved good results with low initial stiffness factor, strict penetration tolerance, and update at each equilibrium iteration. In this case, however, you may check that you don't spend too many iterations in order to respect the penetration tolerance ( = warning "XXX elements have too much penetration" in the solver output file, after the equilibrium iteration).

Regards