I think that the washer is twisting slightly as well as being pressed into the shaft. This is due to the fact that you've got a separation contact between it and the shaft which has no friction so it's free to rotate.
The software then scales the deformation (your setting is on "Adjusted x1" in the display tab) which makes it look as if the washer has grown, but it hasn't. It's just a case of the fact that each vertex of the mesh has moved (rotated) from its original position. This movement is then scaled in a straight line which makes it appear as if it's grown when it hasn't.
Assuming that the parts have symmetry, I would quarter the shaft and washer along their length, and add frictionless supports to the sectioned faces to remove the degrees of freedom. That would stop the twist as well as reducing your solve times.
Googling for "FEA symmetry" should give you some useful links that explain what I'm on about in more detail.
Here's a few links, that aren't all Inventor specific but should help:
http://vrblog.info/2010/02/25/stress-analysis-of-quarter-half-and-cyclic-symmetry-models/
http://www.mcad.com/solidworks-topics/solidworks-simulation-using-symmetry/
http://usa.autodesk.com/adsk/servlet/ps/dl/item?siteID=123112&id=16960466&linkID=13806469
Hope this is useful
Rory
Attach your assembly here.
I was going to submit the assembly, but I ran the simulation one more time and now it is behaving the way I would expect. Whatever the issue was, it seems to have disappeared. Inventor just doesn't make sense some times... Thanks for your help, all!