This will sound "not so good", but trying to help you however I can. First of all I didn't do this in Matlab before, so I can't help much with that. But I sure did natural frequency and frequency response analysis. So I'm familiar at FEA level only.
Your first 4 modes actually correspond to:
1) x translation,
2) y translation,
3) z translation,
4) x rotation.
Always remember, your natural frequency gets better (higher) as you move your center of gravity of the structure closer to the boundaries (constraints). As you change your design and modify your structure's design such that you move the center of gravity closer to the constraints, you will have better (higher) natural frequencies - which will make you design stronger under vibrational fatigue and any other frequency related phenomena.
Now you know my study's outcome. As you look at those frequencies and remember the directions given above for the first 4 modes, (and good thing you put a good snapshot with the coordinate system) you can observe that your ANSYS and Matlab results are changing by; (approximately)
1) +30% (the deformation will look like a translation in x-direction. so your cog-center of gravity- change with the addition of the support is having a reverse effect compared to below results. the deformation in this direction is not affecting your cog a lot-meaning cog is shifting to the right with the addition of the plate and luckily your deformation is in that direction. So you are not gaining extra moment behavior by this deformation)
2) -40% (the deformation is translation in y-direction. the deformation is affecting the cog a lot - meaning the deformation direction is perpendicular to the "cog" shift. that's where problems are starting)
3) -40% (same as 2)
4) -40% (same as 2)
All above feels like you need a transformation matrix for the boundary of that plate only. I might be wrong, but I just wanted you to know how I feel about it knowing the FEA side of it. Hope it helps and you understand what I'm getting at. You may not understand it perfectly, but "cog shift" and "direction of the modes" are the most important subjects I wanted to introduce you on this subject. Now hopefully you'll have a more mature way of handling this problem. My gut feeling says it is the transformation matrix. Because of that, you might be seeing a different behavior in modes 2, 3 & 4. Please let me know the solution as well. This is a nice interesting case. Not that it would be applicable everywhere, but just knowing about the transformation matrix will give me more insight to the Matlab solution of this. Good luck!
Spaceship!!
Aerospace Engineer, M.Sc. / Aircraft Stress Engineer
