-
2
- #1
testing
Aerospace
- Jul 19, 2013
- 127
So recently, I've spent a good deal of time trying to get external python modules (that do not come with the Abaqus installation) in order to cut down on some of my file I/O time extracting Abaqus results for use in other programs (currently MATLAB since that's what most of my current tools are written in or interfaced with). You probably know that Abaqus's version of Python comes with NumPY, though the version may be a little outdated (I think the NumPy version with Abaqus 6.13 was too low for the last major rewrite of SciPy). My motivation was to be able to save in MATLAB readable .mat files rather than humongous ASCII text files (or reverse engineering NumPy's compressed format). For that, I needed the hdf5storage module (which relies on h5py) in order to write MATLAB version 7.3 .mat files (SciPy can do earlier versions) due to the size of my data files. I dabbled with trying to get Abaqus's ODB API modules imported on my system Python install, but I didn't get too far with that. I'm guessing they've compiled their own Python interpreter with their modules linked as shared objects rather than the usual Python package setup. So here's what I was able to do as far as getting more functionality within Abaqus's Python version...
Environment Details:
Abaqus Version: 6.14-1 (though I believe the approach should work for earlier versions as well, you'll just need to make sure you're using the correct version of python for your external installations). This version of Abaqus comes with Python 2.7.x as it's interpreter.
OS: Red Hat Linux 6.x, which comes with Python 2.6.x I believe. I separately installed Python 2.7.x as an alternate install (called using python2.7 rather than python in order to not break any OS python scripts). You'll want to make sure you have a similar Python major version installation as your Abaqus Python version.
Process:
The way python module installation works on Linux (if you're using pip) is basically, you run "pip install package-name" to get additional modules. Essentially, what pip does is download the tarball for the package you want, extracts it, then runs "python setup.py install" from the folder it extracts. This builds and deploys that package to your python's site-packages/ folder. On my system, I believe this was located in the /usr/local/lib/python2.7/site-packages folder.
In order to install a module in Abaqus, the process I used was to install the module for my system python folder for the same version of python that Abaqus uses, then copy the folder from the system python site-packages folder to the Abaqus site packages folder, which is located at $ABAQUS/6.14-1/tools/SMApy/python2.7/lib/python2.7/site-packages (replace the 6.14-1 with your system's version and the $ABAQUS variable is the location of your Abaqus installation). For some packages this was pretty straightforward. For others, which rely on external libraries or additional dependencies, YMMV. For dependencies that are python packages, I basically installed the package I wanted, then opened up an abaqus python session, tried to load it and added packages based on the missing module errors that were generated. I was able to get most of the packages I wanted working using this approach. Note, it appeared that sometimes, installing a module put the needed folder nested one deeper in the site-packages folder than Abaqus liked. The folder containing the package in the Abaqus folder should have the __init__.py and _version.py files in it's root.
Packages I Tried:
NumPy: Yes, Abaqus comes with NumPy, but it is a much older version of NumPy. I think 6.14 comes with Numpy 1.4.x, which might be new enough to support SciPy, but I'm pretty sure the 6.13 version is older and has limited/no support for SciPy. If you want/need to upgrade your NumPy, do not use the latest version (1.9 as of writing this), but instead use the newest 1.8.x release. NumPy 1.9 removes support for it's oldnumerics submodule, which I found was needed for Abaqus/CAE. I believe the install for NumPy was straightforward and didn't have any issues with dependencies.
SciPy: This was again straightforward, but I haven't really used it other than loading the module without errors, so I can't 100% say it's working correctly, but I don't think there's any weird external dependencies, so I'd expect it to work.
matplotlib: So, I could barely get this to install on my system python due to libpng issues. I managed to get it importing fine in Abaqus, but it gives libtk/libtcl problems when I actually try to make a plot, so this isn't working yet. I didn't really have a huge need for it, so I didn't spend too much time trying to get it to work
Numba: This is a neat package that does just in time compilation to speed up your code. It has an external dependency of LLVM 3.5 and I had a hell of a time getting it to work due to the llvmlite package it uses to interface with LLVM. I ended up having to download a prebuilt version of llvmlite from (which was not easy to find in the first place). I was able to get it to work, but I didn't see any speedup since it doesn't support list comprehensions (which the main bottleneck of my code). If you have a lot of numpy operations in an inner loop, it should help speed things up though.
line_profiler: This was a nifty package that gives line by line data on where your function is spending time. Once you get the module installed, you can invoke it with 'abaqus python -m kernprof -l -v scriptname.py args' (the -m is python for running the module, -l -v are kernprof arguments with -v giving verbose output, you can also view results with 'abaqus python line_profiler profresult_file). This worked pretty well once I figured out how to invoke it. You do need to add @profile to just before the function you're trying to profile to get results (make sure to remove it once you are no longer running the profiler).
h5py and hdf5storage: Installed pretty smoothly and was able to successfully write to MATLAB v7.3 .mat files. They end up a bit bigger than if you were saving the same data within MATLAB, but it definitely cut down on my I/O time (and removed a step of reading the ASCII file into MATLAB and resaving as .mat for storage and future I/O speed).
Conclusion:
Hope this helps! I imagine you can use a similar process to get things working on Windows, but I make no guarantees or promises with that.
Environment Details:
Abaqus Version: 6.14-1 (though I believe the approach should work for earlier versions as well, you'll just need to make sure you're using the correct version of python for your external installations). This version of Abaqus comes with Python 2.7.x as it's interpreter.
OS: Red Hat Linux 6.x, which comes with Python 2.6.x I believe. I separately installed Python 2.7.x as an alternate install (called using python2.7 rather than python in order to not break any OS python scripts). You'll want to make sure you have a similar Python major version installation as your Abaqus Python version.
Process:
The way python module installation works on Linux (if you're using pip) is basically, you run "pip install package-name" to get additional modules. Essentially, what pip does is download the tarball for the package you want, extracts it, then runs "python setup.py install" from the folder it extracts. This builds and deploys that package to your python's site-packages/ folder. On my system, I believe this was located in the /usr/local/lib/python2.7/site-packages folder.
In order to install a module in Abaqus, the process I used was to install the module for my system python folder for the same version of python that Abaqus uses, then copy the folder from the system python site-packages folder to the Abaqus site packages folder, which is located at $ABAQUS/6.14-1/tools/SMApy/python2.7/lib/python2.7/site-packages (replace the 6.14-1 with your system's version and the $ABAQUS variable is the location of your Abaqus installation). For some packages this was pretty straightforward. For others, which rely on external libraries or additional dependencies, YMMV. For dependencies that are python packages, I basically installed the package I wanted, then opened up an abaqus python session, tried to load it and added packages based on the missing module errors that were generated. I was able to get most of the packages I wanted working using this approach. Note, it appeared that sometimes, installing a module put the needed folder nested one deeper in the site-packages folder than Abaqus liked. The folder containing the package in the Abaqus folder should have the __init__.py and _version.py files in it's root.
Packages I Tried:
NumPy: Yes, Abaqus comes with NumPy, but it is a much older version of NumPy. I think 6.14 comes with Numpy 1.4.x, which might be new enough to support SciPy, but I'm pretty sure the 6.13 version is older and has limited/no support for SciPy. If you want/need to upgrade your NumPy, do not use the latest version (1.9 as of writing this), but instead use the newest 1.8.x release. NumPy 1.9 removes support for it's oldnumerics submodule, which I found was needed for Abaqus/CAE. I believe the install for NumPy was straightforward and didn't have any issues with dependencies.
SciPy: This was again straightforward, but I haven't really used it other than loading the module without errors, so I can't 100% say it's working correctly, but I don't think there's any weird external dependencies, so I'd expect it to work.
matplotlib: So, I could barely get this to install on my system python due to libpng issues. I managed to get it importing fine in Abaqus, but it gives libtk/libtcl problems when I actually try to make a plot, so this isn't working yet. I didn't really have a huge need for it, so I didn't spend too much time trying to get it to work
Numba: This is a neat package that does just in time compilation to speed up your code. It has an external dependency of LLVM 3.5 and I had a hell of a time getting it to work due to the llvmlite package it uses to interface with LLVM. I ended up having to download a prebuilt version of llvmlite from (which was not easy to find in the first place). I was able to get it to work, but I didn't see any speedup since it doesn't support list comprehensions (which the main bottleneck of my code). If you have a lot of numpy operations in an inner loop, it should help speed things up though.
line_profiler: This was a nifty package that gives line by line data on where your function is spending time. Once you get the module installed, you can invoke it with 'abaqus python -m kernprof -l -v scriptname.py args' (the -m is python for running the module, -l -v are kernprof arguments with -v giving verbose output, you can also view results with 'abaqus python line_profiler profresult_file). This worked pretty well once I figured out how to invoke it. You do need to add @profile to just before the function you're trying to profile to get results (make sure to remove it once you are no longer running the profiler).
h5py and hdf5storage: Installed pretty smoothly and was able to successfully write to MATLAB v7.3 .mat files. They end up a bit bigger than if you were saving the same data within MATLAB, but it definitely cut down on my I/O time (and removed a step of reading the ASCII file into MATLAB and resaving as .mat for storage and future I/O speed).
Conclusion:
Hope this helps! I imagine you can use a similar process to get things working on Windows, but I make no guarantees or promises with that.