Coding instead of using Math software 3

[surval]

I wonder if any Structural Engineers/Designers use software tools developed by them by coding instead of relying on calculations made in Mathcad (Prime or otherwise), or Maple (Maplesoft), etc.

I ran accross this book (Python for Civil and Structural Engineers -

https://www.amazon.com.mx/dp/1698951019/)

and it ignited my interest in the possibility of actually coding all my software tool kit in order to be independent of licensing fees and of compatibility issues (like the ones documented around these forums too) that can result in loss of these assets. It's promising in that, combined with a framework named Jupyter Notebook, it offers a page-format interface that allows for explanation of the process and documentation of the programmed routines, etc, much like you'd be able to do in Mathcad/SMath.

It seems that perhaps this approach would imply the ability to do more powerful programming but without the built-in mathematical methods that the third party software has, although it seems there are add-ons (like NumPy) that could help with this issue; with other modules, this extends to graphical representation and other funcionality that is missing from a pure-coding approach.

I'm currently re-freshing all my structural analyses from scratch (I've been using SMath for that purpose), so I'm not sure I have all the visibility I'd need to be able to know whether doing this (coding everything in Python instead of relying on a third party software) is a hole I want to get into that I will get out of with something good, or wether it's just going to keep me coding rather than doing analysis/design.

I know there's an engineering software room in the forum, but I though that it'd be better if I sought direct feedback from the structural fellows. Thanks in advance.

 

[dik]

I haven't done any real coding for over a decade... and used to use 6 or 8 different languages... My understanding that the current 'hot'one is Python... it's a scripting language (interpreted) and not compiled into machine language... so it makes it a tad slower (it can still be pretty peppy, depending on what you're doing. There are a couple of engineers on the site that seem to be extremely good at Python and there are a lot of 'freeware' Python programs out there. I briefly looked into it and it seems good and there is a lot of good 'free' documentation out there. Check Github...

Rather than think climate change and the corona virus as science, think of it as the wrath of God. Feel any better?

-Dik

 

[JoshPlumSE]

I think there are a lot of engineers that are using Visual Basic Macros in Excel and things along those lines. But, not many of them are doing real programming.

 

[GregLocock]

I use an ungodly mixture of Matlab, GNU/Octave and so on, mostly writing scripts for simulations. They can be anything up to several thousand lines long, but 250 is more typical. I'd actually use Mathcad by choice except they broke it for me.

Jupyter notebooks are a great replacement for Mathcad/Matlab, and if I had my druthers we'd switch over to them for my work. Sadly I have 40 Mb of custom toolboxes that would have to be rewritten, and revalidated.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[rscassar]

I am a big fan of python and strongly advocate the use of Python programming by structural engineers. Personally I have developed my own programs which can design and detail pad foundations, piles and pile caps, columns and walls and in the process of developing a finite element module inclusive of plate elements for further applications for core foundations and steel plate design. Plenty of great modules for download on PyPi and Github particular Numpy for matrix operations and ReportLab for PDF generation. Matplotlib also good for charts and graphs.

And python has a lot of interoperability with other programs, Revit has a pyrevit plug-in. Etabs and CSi programs have API which allow engineers to program their own applications and the forum802 is very frequently receiving questions about python programming using the etabs API. And on the Cad side of things Rhinoceros 3d and the parametric plug-in Grasshopper are also providing in-built python programming scratch pads.

Yes, I'm a big advocate for the use of python by Engineers. And I also find the engineers who are capable of programming an engineering applications are far more knowledgeable of the principles and science involved that the engineers who are 'black-box operators'.

 

[Mohanlal0488]

I am a huge fan of programming from school days, developing games to engineering tools and even my own Finite element tool box (both linear and nonlinear) for Matlab/Octave. It is a very underappreciated skill in engineering...

It is an amazing skill to have, currently I am busy with a project whereby I am performing signal processing for a structure I designed to support a vibrating feeder and jaw crusher. The knowledge gained by interpreting and processing data like this is so much more valuable compared to any information presented in a text book, this would be extended to development of any engineering tool.

There are many people who love Python, but I personally only use python for games and messing around, any work based code I prefer doing it on Matlab/Octave. The reason for this is that in the past when updating packages I found that code would stop working since many of the packages are independently developed thus creating compatibility issues. This being said, the issue only arose on very complex programs.

If you are interested in developing your own software you might also be interested in the development of CDF files which can be developed on Mathematica, this gives a nice interface and is better to use if other people are going to be using your code. I run my version through a Raspberry Pi as it is free.

https://www.wolfram.com/cdf/

 

[surval]

Thanks guys.

Dik - As for the speed thing, I guess you're right in that it can be slower, but I'll have to explore it more to learn whether or not this could be an issue bad enough to discard it as an option.

JohnPlumSE - I find VB to be pretty useful into doing repetitive stuff in Excel; these new tools though (Jupyter, etc) not only allow you to script repetitive stuff, but help with the presentation of the calculations too.

GregLocock - Good luck with all the rewriting and revalidation.

rscassar - Great to know it's being done in professional practice. I agree with the benefits of actually coding the engineering calcuations. I just wonder if it's something that is economically sustainable against using the available software. Plus, I also worry a bit about making an error and not noticing it, whereas commercial software is presumably checked/used by many people that can find these mistakes before they roll it out or soon after (although I'm sure they have their share of undetected mistakes too).

Mohanlal - Well, one of the reasons I was thinking of using Python within Jupyter was to eliminate the problem of compatibility. Since I don't fully understand what's involved, I still worry that this could be an issue and I may not realize it until I'm more invested in this solution; they do talk about Anaconda (the framework within which Jupyter can run) being able to control these types of compatibility problems by allowing you to run different versions of Python in a separate environment each (which is something I can't completely grasp); jupyter also let's you choos the version of Python you want to use with the notebook you're working on. I've been hearing over and over about Octave, but I didn't even consider it because I thought that the community was too small, but it has come up so many times already that I'm considering exploring it too.

After I wrote this thread's OP I found the following post:

https://www.eng-tips.com/viewthread.cfm?qid=473306

where this gentleman has already automated some of the things that have to be done to make things a bit more straightforward within Jupyter, and making it work with Latex and providing symbollic and numerical calculations. It all seems pretty neat, but still a bit daunting and less straightforward that using SMath.

 

[IRstuff]

Has anyone tried Numerit Not symbolic, but possibly a little less daunting than Python from the looks of it. Haven't tried it though

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[Mohanlal0488]

surval, I haven't tried Jupyter as yet, however I have heard it often. The Octave community may not be as large as some of the others, but it does seem to be well run. One of the biggest benefits in my opinion is the compatibility with Matlab. The Matlab help and forums are by far some of the best I have ever seen and I often find myself using them when working on Octave.

 

[GregLocock]

I had a quick look at Numerit and it looks fine, but the company no longer allows you to buy it. So you are stuck with the demo version. (!)

Another option is Julia

https://julialang.org/benchmarks/

, very fast compared with the other alternatives discussed here, and the syntax looks simple enough.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[rscassar]

rscassar - Great to know it's being done in professional practice. I agree with the benefits of actually coding the engineering calcuations. I just wonder if it's something that is economically sustainable against using the available software. Plus, I also worry a bit about making an error and not noticing it, whereas commercial software is presumably checked/used by many people that can find these mistakes before they roll it out or soon after (although I'm sure they have their share of undetected mistakes too).

Yes, it can be. Developing a fully integrated program likely won't happen. Not impossible but will probably take a career to achieve. I see the strengths in interoperability. My preference is Etabs for analysis, but not so much the design modules. And not all design aspects are covered in a standalone software package are will often require interoperability with another software package. And there should be more importance on innovation in structural engineering. People are always looking for something already created to speed up their design. Not as many people are willing create something themselves. Not every software package is free from errors, and given how people like 'black-box' programs, if the software does have an error or bug they are likely oblivious to it. I don't like the GUI of etabs and not completely satisfied with Ram Concepts calculations or the way it "trims" and reinforced a cross-section.

Finite Element in python isn't too difficult. Just starting to get contour plots for floors starting to work. Still have a bit of work to do on it. Still can't work out how to derive shape functions for general-form quadrilateral plate bending element. Just using closed form solution for rectangular elements.

 

[IRstuff]

Thanks, Greg; I had wondered about lack of pricing on the order page. It looks like they haven't done anything to it since 2012.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[Michael L]

I believe Python is fantastic for structural engineering.

It allows you to be productive right from the start, without knowing about classes, types, or gnarly syntax such C# Java having you enter "public static main void". However, it sort of draws you in as you hit extra levels of complexity, you can uncover these features slowly however by learning about functions, or classes, or the web side of things. The guys at Talk Python podcast call it a "full spectrum" language for its broad reach from beginner to advanced.

Software developers may have the time to learn 3 or 5 programming languages, but where your primary role is a structural/mechanical engineer, you want the investment to carry you through at least a good decade.

My experience of the above has been starting from simple procedural code, basically writing out equations in a python editor similar to Smath/Mathcad. Then I wanted to present nice output to internal reviewers, so I found Jupyter and a package "handcalcs" (written by a structural engineer in Canada) to make that good. Then I thought I want to share my code, so I used Streamlit for super quick websites which you can have up and running. Now I'm thinking about constructing a suite of tools, so looking at Django. All those steps and I haven't had to abandon Python to find a domain-specific language for web or for numerical computing, or for GUI generation (although distribution of desktop GUI's is a weakness in Python). On the way I discovered things that I wasn't expecting, such as use of Python in Dynamo/Grasshopper, ability to write scripts for FEA software using their COM interface, parsing tables out of PDFs, how amazing version control and GitHub is etc.

 

[Enable]

I find the combination of R and Python to be extremely complimentary. R for statistical analysis and Python for the heavy lifting when intensive computation is required.

 

[dik]

A couple of my programs had source code over 1MB in size...

Rather than think climate change and the corona virus as science, think of it as the wrath of God. Feel any better?

-Dik

 

[dik]

Greg... looking at the Julia charts... things have changed a bunch... I used to do a lot of programming in BASIC (MS PDS, not QuickBASIC) and with heavy math issues used to beat my C Programmer buddy... he didn't realise that (at the time) BASIC used MS's Fortran Math library which was faster than the C Math library. Since beyond the ..386, the processors are so fast, there's little need for speed... I go back to the 8088 times where there was a big difference...

Rather than think climate change and the corona virus as science, think of it as the wrath of God. Feel any better?

-Dik

 

[IDS]

I think there are a lot of engineers that are using Visual Basic Macros in Excel and things along those lines. But, not many of them are doing real programming.

I'm not sure where you are drawing the "real programming" line, but in my opinion anyone who creates an application that can be re-used is doing real programming.

I have done a lot of work with python in recent years, but I don't use Jupyter Notebooks, and I don't see any reason to switch from Excel as the user interface, which I use in conjunction with xlwings, and more recently pyxll. (Xlwings is free, pyxll is a commercial package, but reasonably priced).

The main reasons for me for using python rather than VBA are:
Python has a huge selection of free library packages, providing high performance matrix functions, integration, curve fitting, solvers, etc.
Ensuring that code is compatible with different bit versions of Excel is easier, especially when linking to compiled code.

Python fans go on about how easy it is to use, and how it is much better than VBA. It does have some advantages, but overall I still prefer VBA, and if I need to do something quickly I will still do it in VBA rather than python.

On the speed front, VBA is actually faster than plain python, but:
Linking to compiled libraries in python (such as numpy, scipy) makes the maths intensive work much quicker than pure VBA (although you can link to compiled code from VBA of course).
Python has jit compilers, such as Numba, which speed everything up substantially, probably to something close to Julia, although I haven't actually compared them.


Doug Jenkins
Interactive Design Services

http://newtonexcelbach.wordpress.com/

 

[IDS]

Reading Michael L's post reminds me:

Another thing I do with Excel + VBA or Python is link to the Strand7 FEA package through their API. The API has interfaces for both VBA and python, but the main advantages with python are:

Manipulating matrices is easier; e.g. converting 1 1D results matrix to 2D for display in a spreadsheet.
I find it easier in python to create matrices of the right size for receiving results from the API.
Python code can be saved as a stand-alone package that can be called from any file, rather than linked to a specific spreadsheet.

Doug Jenkins
Interactive Design Services

http://newtonexcelbach.wordpress.com/

 

[goutam_freelance]

I have used MATLAB for writing FEA solution of 3D trusses and found it very convenient.I do not know about python or others but matrix manipulation in MATLAB is very handy.

Engineers, think what we have done to the environment !

https://www.linkedin.com/in/goutam-das-59743b30/

 

[wcfrobert]

Looking at the current trend, all new graduate engineers will know programming in one form or another. in a few years, knowing how to program would be equivalent of knowing how to use Excel.

I do see wide adoption of python in terms of accessing software APIs and scripting to minimize repetitive work. But I do not believe it will replace calculation tools like MathCAD or Excel (at least for my use case).


Reason 1: Presentable Calculation Package
I have actually implemented several of my calculation sheets in Jupyter Notebook to see how it works. There are several downsides. First of all, your calculations will eventually be combined into a presentable calculation package. If you want to make the jupyter notebook pretty and easy to understand, you have to load it with print statements. It just looks ugly. If you want to put less print statements, or put them at the end, or make your code prettier with custom output, the notebook becomes hard to debug and like a black box. Alternatively, you can use IDE like spyder which shows you all the variable. But again you are trading off "debugability" with "presentability". Here is an example of a script I made. I actually played around with the custom CSS file to make it look prettier, but it just cannot compare to Excel or MathCAD.

In essence, yes it is fast to program a calculation sheet in python, but it takes longer to make it look presentable.


Reason 2: Debugging And Work Checking
I showed some of my scripts to some other engineers at my office. It turns out no one wants to debug or check a print-out of python code. Even if they are fluent in programming, it is just easier to read an Excel print-out or a MathCAD sheet.


Reason 3: Structures first, software second
I made a wind load calculation module once that outputs .txt files of all the relevant results as well as some plots as .png files. I even put widgets like sliders and toggle buttons into the Jupyter notebook. It took me about a day and a half to make and takes into account many different edge cases.

You end up being a software engineer worrying about UI/UX rather than a structural engineer concern about the actual calculations and designing structures...

I could have made the same thing in about two hours in Excel. There is a reason why people say Excel is the most popular programming language. I think people take for granted just how great, easy-to-use, and powerful Excel really is.


Reason 4: Unnecessary for most use case in structural engineering
At least in building design, the calculations we still do by hand are related to code-checks, dimensions and detailing, free-body diagrams, or at most some energy method like virtual works or yield-line. The difficult part comes from understanding various material behavior and physical concepts, or getting accustom to the code documents (getting thicker every year). The math usually boils down to algebra you learned before Calculus 100. Anything more mathematically complex gets done in commercial software like ETABS or RISA.

Python works best when you can take advantage of the programmatic nature of it. in other words, you can take advantage of loops, flow control, increasing levels of abstractions with object-oriented programming. If you are using it for a series of algebraic manipulation that you could have done by hand, that would be akin to killing a fly with a nuclear bomb.

In my case, it's hard to fully take advantage of the power of python. Most of the times, I've used it to access the ETABS/SAP API to streamline some workflow. It is hard to generalize. I suppose the ability to think programmatically is most important. Structural engineering is unique in that most building are not optimized like an airplane or an automobile. If you can reduce 1 unit of weight in an airplane design, due to the coupled nature of aircrafts (aerodynamics, propulsion, stability, etc), the net weight reduction will be several times greater than just the structural weight reduction. So it is worth it for them to employ complex computational methods. Maybe we will go down that path, who knows.


Reason 5: Maintenance
I know some structural engineering firms are hiring software engineers just to develop their own internal tools. That's pretty cool to see. If it's for some highly specialized tool, or to streamline some particular workflow, then I think that is a great idea. I don't know if the cost-benefit is there for developing your own calculation toolkit. you have to take into account the labor cost of developing, debugging, and maintaining your internal software vs. purchasing some external commercial software. Software don't last forever, you have to maintain it. Plus you have to debug it yourself vs. using something that is time-tested.