Best mem-disk-gpu configuration for large simulations

[ruimoreira11]

I have a lenovo ws P500 with 64gb ram, xeon e51620 v3 3.5GHz, with a M2-nvme disk for windows, another M2-nvme (in an PCIx4 3.0 adapter) for programs and 3 harddrives (1Tb each) for data.
And one nvidea quadro K4200 for display and 3D handling and a second K2200 just for GPU computing (i upgraded from this to the K4200 and left the k2200 in the second bay just because).
I need to handle large models and independently of the changes I make in the FEMAP settings (using the preferences ->Interfaces to enable ILP-64bit; defining number of available processors and GPU computing in analysis set) I realize that most of the consumed time is disk operation (and since my scratch file in in one of the 3 1Tb harddrives (which are not a fast drive) it takes time...a lot of time).
I am now waiting for a small ssd drive (240Gb) just for scratch disk but I wonder if there are any small tunes or parameter modifications that could improve solver speed in those cases (I mostly use Modal Analysis of large, single body, parts (usually free boundary conditions)).

 

[IRstuff]

Seems like a question to be posed to the manufacturer of your simulation software

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[ruimoreira11]

Sorry, this is not a question on how to do that (I think that no one can provide me a valid and general guide to speed up solver tasks) but rather a call for user experiences that once shared my issues and somehow succeed on getting better results with proper tune of parameters, or experiences with hardware changes (for example I am not shure if the speed of a ssd drive would provide better results than a memory upgrade - in fact my solver is using a lot of disc translation while not using the entire amount of ram available; why? I don't know; can it be improved? I hope someone already has and is willing to share with us some hints).
Thank you.

 

[IRstuff]

The people that wrote the software are likely the ONLY people that can tell you what's required to speed it up.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[GregLocock]

We run big models on clusters of 16 (or more) Linux machines. That's all I know, it's a detail.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[BlasMolero]

Dear RuiMoreira,
I run a powerful Windows 10 Pro workstation with plenty of RAM memory (128 GB RAM at 3000 MHz) and two fast SSD Samsung PRO PCie 1 TB at 3500 MB/w for Simcenter Nastran SCRATCH, this is the important key: a fast drive for scratch. Also, the i7-7820X processor is overclocked with the 8 cores running at 4.18 GHz.

Fast & plenty of RAM is critical when solving any normal modes analysis with FEMAP & Simcenter Nastran (SOL103), the reason is because only DIRECT SPARSE SOLVER is available, then when solving normal modes (SOL103) problems with millions of nodes be prepare for having a lot of free hard disc space and also a lot of RAM memory to run all problem in memory, if not all will go to hard disc, and this is where the fast SSD SCRATCH disk is important.

To learn more about the best hardware for FEMAP & Simcenter Nastran please visit my blog in the following address:

https://iberisa.wordpress.com/2014/01/03/requisitos-hardware-de-nx-nastran/

https://iberisa.wordpress.com/2016/03/21/requisitos-hardware-nx-nastran-ii/

Best regards,
Blas.

~~~~~~~~~~~~~~~~~~~~~~
Blas Molero Hidalgo
Ingeniero Industrial
Director

IBERISA
48004 BILBAO (SPAIN)
WEB:

http://www.iberisa.com

Blog de FEMAP & NX Nastran:

http://iberisa.wordpress.com/

 

[ThomasH]

Hi

I would start with the Resourse Monitor under the Task Manager in Windows. That can give you a hint regarding the bottleneck for your system with the type of analysis you run.

When you say large models, what does that mean? Depending on the budget, ssd disks could be replaced by large amounts of ram. That would probably speed things up. But doing that for very large models may be costly.
But I agree that the vendor has best knowledge on how to optimize things.

Good luck

Thomas

 

[ruimoreira11]

Yes, the bottleneck is the hardrive (at least, most of the solver time is applied in disk saving/loading.
Thank you for the help provided.
The vendor has in fact some hints regarding the best configuration but it is provided as general information since each solution type has its own requisites.
For example I realize that GPU computing is not a huge improvement for natural modes computation but it is reported along some webpages that distributed tasks can speed up when using GPU (for exaple in direct frequency analysis, since, once calculated the stiffness, mass and damping matrices, the frequency response is obtained from a repetitive calculation for each frequency step).
I am waiting for a ssd disk to be my stratch dir and I will take some conclusions on that.