Can a vise be too big?

[Wawoodman]

For my Rockwell mill (table size about 6 x 26), I was looking for a 4" Kurt, with a swivel base. The only thing I came up with (at a price I could afford) was a 5". I have it mounted, and it's a monster! Should I continue my search for a 4, or will I be happier with the extra heft of the 5? Remember that the Rockwell is a medium duty (compared to a BP) and so I won't really be doing anything that I would figure would be too much for either vise.

BTW, if anybody has a 4 on a swivel and wants to move up, let's talk!