Plotting at 1/4"=1'-0" tells me that you are plotting in model space.
As a go-by, we use a 4 inch text height for our "regular" text, but plotting at a different scale (for instance 1/8 scale) will result in text being too small.
I have seen "Regular Text" as large as 6 inches. Establish a standard and stick with it.
Explore "Strong Text", "Small Text", "Small Drawing Title" and "Large Drawing Title" as basic text object types for your template (5 in all).
Is the STYLE of the text in question assigned to a height greater than 0 (zero)?
If so, the SETVAR TEXTSIZE does not come into play.
You might need to tweak your dimension variables so as align to the current text style. The name of this variable is DIMTXSTY.
(setvar 'DIMTXSTY (getvar 'TEXTSTYLE)) should do it.
To fix existing dimensions, command -DIMSTYLE "Apply" and select the objects.
Is the STYLE assigned a height equal to 0?
If so, play with the SETVAR DIMTXT.
(setvar 'DIMTXT (getvar 'TEXTSIZE)) should do it.
To fix existing dimensions is also command -DIMSTYLE "Apply".
As a reminder, text styles can either be assigned a height equal to zero, or assigned a fixed height greater than zero. Each has it merit, however TABLES do not like the assigned height greater than zero text styles.
To fix existing single-line text, highlight and open the properties floating window or command SCALETEXT.
[untested for those who want to challenge]
Hope this helps.
Scot-65
A gift of extraordinary Common Sense does not require an Acronym Suffix to be added to my given name.