@pmartin, If it’s really bugging you and you’re not using shared coordinates; you could always build your project from sea level – starting with a very deep earthen foundation – and then place the Survey Point and building levels on top of it. 
For me though: I build from the ground up and anything placed below the pad elevation is relative to my building level – not to sea level. That includes Toposurface points.
I model the site in Revit (usually from the Civil Engineer’s CAD base for the Rough Grading Plans) in order to calculate the finish grade elevations that need be shown on Plot Plans. “True” point elevations don’t concern me because I’m usually sloping earth away from and around the house to meet drainage, driveway and other slope requirements. Having the pad points at 0.0 makes the math a lot easier. Still, my Spot Elevations accurately report the “true” elevations on Plot Plans because they are set relative to the Survey Point or the “Sea Level” Level.
So, I have no problem with moving the Toposurface down once it’s created. If I have to add “true” points to it later on, and I don’t want to bother doing the math; I simply unpin and move the Topo up to its origin point, edit the surface and I then move it back down and re-pin. It’s no biggie to me.
TIP 1: If your Topo is dependent on a duplicate for cut and fill computations, I suggest you move both at the same time to avoid breaking that association.
TIP 2: Have a Level at the base of your Topo. When the Topo and its Level are at the correct elevation for your project; pull a dimension between the Topo’s Level and the Project Base Point Level (e.g. the Topo’s origination level), and then lock that dimension. This Level will serve as a “Relative Base” for Spot Elevations, should your Survey Point be needed elsewhere in the project.