I would like to be able to control the font size for the labels produced by
plot file u 1:2:(sprintf(\"%4.3f\", $1)) with labels offset char 0,2 t \"\"
In the case of the latex terminals the font choice and size are under the control of LaTeX rather than gnuplot's internal enhanced text processing. You would have to format the label as a TeX command.
plot file u 1:2:(sprintf('{\\tiny %4.3f}', $1)) with labels notitle
Note that the number of escaped backslashes depends on whether you place the format in single or double quotes.