I believe this is within the capabilities of 3D FLAC, which is actually finite-difference, rather than FEM. We've used FLAC for stability analysis in 2D, and it seems to work well, at least benchmarked against limit equilibrium methods and "gut checks." For 3D, FEM/FDM may be much better than limit equilibrium because of the difficulty in getting the boundary conditions on the sides of the failure mass, and trying to find the most realistic failure surface. I think there is a lot of potential for FEM/FDM methods there; 3D limit equilibrium results are commonly viewed with suspicion. At least in theory, FEM/FDM should take care of strain incompatibility among different materials (when the stiff materials reach post-peak strength before the soft ones have been fully mobilized).
I just want to add that you may want to start with:
1. Which problem you want to do: stability of plain slope or those with reinforcement material; linear analaysis or non-linear analysis.
2. Which analysis you want to do: using homogenation / linearitation method or you consider every material, such as soil and other reinforcement material, separately then you also want to know the connection between reinforcement element and soil element, whether using zero thickness element (Goodman) or thin element (Desai)
3. Then finally the solution of system equations which you want to use --> implicit or explicit. Both will have advantages and disadvantages.
