I Fixed the model by removing all the vertically patterened groves. The casting now is smooth without grooves. Made the mesh size higher ( So it could mesh quicker..). Also further increased the wall of the extrusion (realistically we wont have it this thick, but I was hoping to see if this improves)
Here are the results.
Safty factor : I am sorry I bring this up always. To me if model has acceptable stresses at safety factor 3 & load of 100 lbs means that realistically the model can actually take 300lbs load and still be safe. Correct me if iam wrong.
Stresses : See the bar shows so low scale numbers. Showing that the casting is having really very very low stresses. The large stress is deep in side the pole on the extrusion.