Thanks @daniel_lyall for jumping in 🙂
I agree with your assesment, and I'd like to add on a few things:
1) I second what Daniel has said about your modeling skills improvement. Truly such an improvement! Let's take it one step further, for sake of discussion, shall we? Right now we have all these surfaces, but we really don't have a "solid body". We need to select all the faces in a setup. What if we could convert them all into a "Body"? Well, we can! Go into the "Patch" workspace and select the "Stitch" command. Crank down the tolerance and select the faces to stitch. Boom! You now have an actual body and not three separate faces:

2) Your 2D Contour is cutting through your part:

Depressing the A:T key (or Command on a Mac), you can then select a single edge. Click on it again and you will be able to make an open chain around the part:

And generally speaking; Finish passes, I typically avoid "Rest Machining" when possible, simply for the sake of generation time. Of course, if you have two tools finishing one certain area, that does not apply, as you want to use rest machining.
yes, I prefer to use "As Computed" with a zero offset:

You had your Adaptive cuts set to "Tool Inside Boundary". This will cause the tool to leave a small cusp around the edge of the part. I prefer to set it to "Tool Center on Boundary"
And lastly, Smoothing is your friend. I set your roughing tolerances to .0005 and smoothing to .001, the Parallel I set to .0001 and .001 respectively. Your code was literally cut in half. That was half the movements required to achieve the same result. File has been attached, and I hope this helps!
Seth Madore
Owner, Liberty Machine, Inc.
Good. Fast. Cheap. Pick two.