Close tolerance machining on aluminum is a tricky thing. First, was the material cut from sheet or bar? Was it cut with the grain? Is the material drawn or rolled stock?
I've stress relieved 6061 by putting it in an oven after roughing at 375 for 6hrs. Sometimes machining processes are enough...
