FEA om multiple computers 1

[ThomasH]

Hi all

I little something I've been thinking about.

I have a friend who works with 3d Studio Max. When he does rendering work he runs over the company network and uses several computers working together.

Has anybody tried running FEA in the same manner? I routinely use more than one processor to solve a problem but all the processors are in the same computer, is that necessary?

Just a question ;-)

Thomas

 

[xerf]

Some of the FE software can run on computer clusters.
For example, ABAQUS can use MPI.

 

[Stringmaker]

Ansys and LS-DYNA can with certain solvers as well. However, I always found it funny that the license to in parallel on different machines is costwise approximately as expensive as buying a computer with adequate processing capability. How much processing power do you really need? Would a 2P computer with quad core processors and 16 or 32 GB of memory not be adequate? Anymore, you can build such a system for well under $5K (yourself of course...not through Dell). I believe that's in the realm of what the license fee is for what you're speak about.

Just one man's thoughts anyway...

 

[GregLocock]

Nastran runs on a Linux cluster. Now, whether it'll run under ClusterLinux is another question, but I don't see why not.





Cheers

Greg Locock

Please see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[ThomasH]

Stringmaker:

For the "how much processing power do you really need". Well you can never get to much can you? (A joke, although doesn't work that well in writing.)

I agree with you that a 2P quad core system will go a long way. But the reason for my question was a discussion we had at the office.

It seems like the days when we get higher and higher clockspeeds are if not over at least slowing down. Now it's parallel processing and I'm not sure that the softwares are running twice as fast on 2 cores instead of one. Some are good at it, others not that good. And then you have the licensing part. One processor one license, two processors two licences in some cases.

But like I said, It was just a thought I sent out to se what response I would get.

Thanks

Thomas

 

[pja]

Finite element formulations have an inherent problem with scalability as compared to, for example, finite difference solutions. The reason for this is that while the assembly procedure in FEM is basically what one calls perfectly parallel, the solution of large sparse systems of equations doesn't scale as well. On the other hand, finite difference solutions often involve only the solution of small tridiagonal systems and the exchange of information across parallel domain boundaries. Getting good scaling above 10 processors is rare for FEM, while CFD computations often scale well for hundreds of processors. Sparse solver technology and domain decomposition methods such as FETI will have to improve significantly in order for FEM to have comparable efficiency.

 

[cbrn]

Hi,
Pja explained well the reasons why FEA don't scale parallelly very well. To return to your question, the answer is definitely NO, the same solver scaled on 10 processors will not be as fast as if it was running on a SINGLE processor having 10x the power.
Some algorithms which are used to optimize the processes in a non-parallel architecture are even non-scalable: in fact, in ANSYS Parallel some features are not available, though you would expect the parallel solvers to be identical to their "mono-core" parents... On their turn, the "mono-cores" run well on a single physical processor but begin to get confused (i.e., not so efficient) if ran in MPI on more than one machine. I don't have any info, but I suspect any solver which "parallelizes" via shared-memory architecture (MPI and others) to be fairly unefficient. With Ansys, don't even know if you still can do that with v.11...
It is still a matter of fact, in my opinion (but there are some numerical reasons to comfort me), that in FEA you'd better use the most powerful processor (SINGLE-processor) you can afford, with the largest amount of RAM allowed by your system (for example, if you use XP-32 or even Vista, it's no use having more than 3 GB of RAM: the OS won't even "see" it and the really available memory will be even far less, around 2 GB if you are very lucky...).

Regards

 

[ThomasH]

Hi

I agree that there are reasons why FEA don't scale parallelly very well. The assembly procedure does but the timeconsuming solving of the equations don't.

I have seen benchmarks where certain softwares are supposed to scale well up to approximately 10 processors. By no means a 10x speed increase but still significant. One problem as I understand it is that the processors doesn't seem to increase their speed as they used to. We can't run the same code on a new computer and things automaticly go faster. Now they get more cores instead so things need to run parallell. The solution might be things like huge RAM-memorys and domain decomposition etc. There are numerical "tricks" but they only do so much.

It will be interesting to see what happens in the future.

Regards

Thomas

 

[cbrn]

Hi,
Thomas, you're right, although some solvers of the newest generations do parallelize the solution process, not only the assembly procedure. However, they are not able to do this at the same speed "as if" they could do it in "mono-thread" on a single processor.
Things get a bit confused nowadays, in my opinion, because the "multi-core" architecture is somewhere between a "multi-thread mono-processor" and a real "multi-processor". As a result, on the Pentium 4 platform even telling if, say, Ansys will behave better by disabling hyperthreading or by keeping it and asking for 2 "processors" in the program, was nearly impossible. I made this test and two different machines behaved in opposite ways!
Of course, building ultra-powerful single-processors is technically impossible, so the discussion about the "10x - the - power" processor is completely theoretical: you will have to do with 10 processors because you will never get the "10x" single-processor. With a little time, the algorythms will progress on this base.
From this point of view, things are much easier in the 3D animation / rendering field (for example, afaik, radiosity algorithms scale very well).

Regards

 

[Jordonlaw]

I've had some benchmarking runs performed on up to 8 processors using dual core technology (Woodcrest) on a PC cluster. The scaling was pretty good on both Win XP and Linux until you started to use more than one CPU per physical chip and then Linux won hands down. I now hear that the quad cores have a bottleneck and that the 'older' dual cores are better. Of course this is all solver dependant as has been noted.

The only thing to do is to put a true model (ie one of yours) on a suitable set of machines and run the tests. I was lucky in that I got this done for free with support from both software and hardware vendors.

In terms of using 'spare' capacity in an office environment this is often not very fruitfulas PC usually get used more than you think.

 

[crisb]

I have just bought a new workstation...larger non linear models run in about a third of the time taken on my 5 year old P4 desktop. I guess I was expecting more of an improvement than that.
My gut feeling is that the bottlenecks on larger models are now the limitation of 32 bit Windows OS to access memory, and the speed of access to swap files on the hard drive, rather than the amount of RAM, or number or speed of processors. However I need to do some more trials with different operating systems and drives, when I have spare time to investigate this.

 

[GregLocock]

One advantage of using a PC cluster is that you don't run out of RAM as quickly. That is, each single threaded CPU has 2GB to play with.


Cheers

Greg Locock

Please see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[OnlineFEASolver]

Here is a nice link to Abaqus's performance data for v6.7.

http://www.simulia.com/support/v67/v67_performance.htm

According to them:

Both the element computations and linear equation solution
can be executed in parallel. Parallelization tends to be
more effective with increasing problem size. Problems under
100,000 degrees of freedom (DOF) will typically not show a
decrease in execution time when more than 2 cores are used
a problem. Problems under 2 million DOF will typically
not show significant speedup beyond 16 cores. Problems with
a small number of iterations will also not show good
overall speedup with added cores as only the actual
iterations are executed in parallel. The input
file preprocessor and initialization code execute on a
single core.

Principal -

http://www.OnlineFEASolver.com

General FEA Consulting Services