Lathe Cross Slide Problem

[EdK]

I was planning on knocking this bushing out but I noticed it's knurled. I don't know how they could have pressed a knurled bushing into the hole and expect the milled slot to stay at a right angle to the cross slide. It looks to me like the slot is not at a right angle to the cross slide and that might be causing some of my problems.

I need to get the bushing out so I can mill that raw cast iron surface flat for the nut to rest on.

Ed