The label associated to a grading point when is moved when is moved changes its size automatically and a leader appears. After this happen no control over the grading point and its associate label. Please refer to the image below.
Look in the LabelStyle. The dragged state text can have a different height than the normal label text height.
