Now that I kinda understand your application, how are you going to determine the hole size, taper and straightness?

While pressing in a sleeve that size, that long with an interference fit in aluminum is going to be a challenge in itself given the fact you most likely will not have the greatest surface finish nor a true round thru bore (I imagine peaks and valleys like a vac hose....exaggerated example, but gives you an image) that will not be tapered.

Using the quill on your knee mill will get you the result you desire as far as a 1.5" dia. thru the material, but as far as taper and surface finish go, it's going to be difficult at best, UNLESS you have a very tight machine, that is trammed in dead square.

Your best bet is following Johns suggest of boring in the lathe using the cross slide to hold your part.

When making your boring bar be sure to use a fine pitch set screw to push the bit out of the bar, that way you will have better control of your size as you just need to know how many thousands per inch each revolution of the screw is, and you can dial it in pretty damn good in that manner.

That 3" boring head sticking out of your quill with a 5 or 6" boring bar sticking out of that (whether it be carbide or not) is not going to be doing you any favors in achieving a decent surface finish, which will translate into a tapered diameter that will be all over the place in size as you measure it thru.

When you do get your hole bored, and your sleeve made, (not knowing the wall thickness here but this will help nonetheless) place that sleeve in your freezer for the day. Keep the mating piece room temp. Do not heat it up! When ready, pull out the sleeve (using gloves to hold....no warm hands) and get it into your bore ASAP and pressed in (again depending on the wall thickness determines the shrinkage) and you should be good to go!

Good luck!
Russ