Can I use movement commands to refine a mouse-defined region?

[Nick Helm]

I usually define a region by firstly setting the mark with C-SPC, moving
point with some combination of C-f, C-b, C-n, C-p etc, then acting on the
highlighted region.

Occasionally, I use the mouse to define the region using click and drag.
This, I presume, sets the mark on button down and moves point with the
drag. 

Sometimes though, my mousing is not as accurate as it could be and I
would like to refine the region by moving point with C-f, C-b etc. 
However, any movement keys immediately deactivates the defined region
and removes the region highlight. 

Any ideas how I might resolve this? 

Thanks,
Nick