Slow Export DEM surface

[Mike_D_G]

Hello.   I'm exporting a surface to DEM.  The surface has about 1000 points covering 28 acres.  I tried exporting with 1 foot spacing, sampling surface at grid points only.   It has taken over an hour to get to about 10% done.   This is not a very detailed surface, many triangles are 50 ft long so maybe 1ft spacing is overkill.  However, there are a few important breakline type features to catch, so I've kept the sample spacing tight.        Exporting this surface would take my GIS software a few minutes tops to complete.   Any pointers?   Or is this one of those things that C3D just doesn't do well, and I should be using a different product in the C3D family?  

[Pointdump]

Hi Mike,
Welcome to the Autodesk Forums.
1 Acre = 43,560 Sq Ft
28 Acres = 1,219,680 Sq Ft
No wonder Civil 3D is choking.
I gotta ask. What was the original data used to create your Surface, and why do you want to create a DEM? (I'm hoping by DEM you mean GeoTiff.)
Dave

[fcernst]

We need to be able to handle on the order of 15,000 acres for Metropolitan basin studies in Civil 3D. 

[rl_jackson]

@fcernst 

That amount of data is totally unnecessary, Q=CIA, if we were building a rocket ship to Mars, then that precision would be completely required, hence the reason machine parts are to the .005" and greater. Just think of how often you round values in your daily calculations. Honestly, I'm all for the technology, its another tool in the toolbox (should we be able to handle the tech, Yes!), I don't think the overall dataset is needed to do what needs to be done, when you look at the grand picture.

[fcernst]

We are quickly moving past this train of thought that we all used to have regarding large scale Terrain datasets using LiDAR. 

Civil 3D is falling way behind now..

[fcernst]

Surface flow modeling is primarily done in 2D now with LiDAR data supplemented with survey bathymetric data … with perhaps industry standard survey data in focus areas. 

Final site grading and transportation corridor design is done with survey and typical industry survey accuracy standards. 

[Mike_D_G]

I'm well aware how much land area is represented by 28 acres.  It is also a very small surface compared to the ones I've dealt with in the past, which GIS  has worked with handily.  I'm also aware that my sample spacing is very tight, compared to the source survey data.   I've chosen tight spacing to better preserve the surface detail when exporting to a grid surface, DEM or Geotiff.  I'll be doing hydraulic modeling with this surface, Fred is on the right wavelength.

So much for what I'm exporting and why. Back to the original question: why does civil 3D not handle this situation better?

Another item:. When importing a DEM surface, Why does civil3D many times import it in the wrong location, horizontally?  It will be off like 5-20 ft in X and Y coordinates.  Others have pointed this out over the last 10 years or so, myself included.  It must be a low priority to fix, because several have submitted this to Autodesk for fixing.  

[ChicagoLooper]

---

@Mike_D_G wrote:

<< I'm exporting a surface to DEM.>> 

---

Are you export your DEM using Toolspace?

Would it be possible to upload a screenshot of the Export to DEM window? Example shown below.

[Pointdump]

Fred,
What does a Metropolitan basin study involve? Do you use Grid or TIN Surfaces? What C3D tools?
Dave

[Pointdump]

Mike,
"I'm well aware..."
Ouch! Didn't mean to offend you. Sorry. I was just wondering why if your original Surface had only 1,000 points you needed to interpolate all those extra points. I'm not casting asparagus at your work flow, I'm trying to learn.
So hydraulic modeling requires a Grid Surface? Why do you need to export your Surface if you already have one? Do you use another software for the modeling?
"When importing a DEM surface, Why does civil3D many times import it in the wrong location, horizontally?"
Agreed, Civil 3D needs to be able to distinguish between Lower/Upper Left and Center Pixel. It utterly fails in this regard.
Dave

[fcernst]

Use just about all the tools really..

Lots of surface analysis, surface comparisons, profiles, sections.....

No grading objects..

[Anonymous]

I have had issues with export to DEM as well.

For whatever reason if I exported it to LandXML, import into a new blank drawing, then export it would work.

I hope that will work for you as well.

Thanks,

[ChicagoLooper]

Why no screen shots? 

[Mike_D_G]

Dave,

Sorry to be touchy on this.  I was a little grumpy when CAD wasn't doing what I wanted it to do.

[Mike_D_G]

Thanks for the replies.   I wound up doing something suggested by the Autodesk person in my double posted thread.  Here's what worked for me:

 The surface contained a large area - 27.5 Acres of the total 28 Acres, which is the only area I wanted to export.   The remaining 1/2 acre of the surface included several smaller disconnected chunks of surface.   I cropped the 27.5 acre piece to a new drawing, and then was able to export the DEM in about 5 minutes, with my 1 foot sample spacing preserved.   Not sure why that worked better.

I should have explained a bit more earlier:  This is a survey of a creek bottom which was a lake bed until the dam broke in 2018.   This survey surface is essentially (dry) bathymetry which supplements the lidar surface below the previous water surface elevation (when lidar was flown).   

I have worked with lidar DEM surfaces in C3D going back to 2010 - they don't work so well due to 1) C3D chokes on large surfaces with millions of points 2) aforementioned horizontal shift issue when bringing DEM surfaces into C3D.  Thus, I prefer to export the survey surface and deal with pasting surfaces, etc. in GIS or GeoHECRAS.     

I'm assuming the horizontal shift issue is still present in C3D 2022.   Can anyone substantiate this?   Also, are there any plans to actually fix this problem?

[TerryDotson]

---

I'm assuming the horizontal shift issue is still present in C3D 2022.   Can anyone substantiate this?   Also, are there any plans to actually fix this problem?

Unfortunately there are two ways to look at grid data, is it a grid line intersection or a pixel (with width/height).  Attached is a PDF from Michael Goodchild on the topic.  See section 1.1 for:

• values are averages for cells
• values are samples at cell centers
• values are samples at the grid nodes

For example, ESRI ASC Grid uses the second method (pixels).  Our Grid Import & Export drivers handle each file format differently based on the best information we can obtain about the format.  If it's not documented we comply with GDAL/QGIS.

[Mike_D_G]

Terry, 

Thanks.   Yes, cell center vs. cell corner is one issue at hand.  The other factor at play here is survey feet vs. international feet.   That explains the fact that the shift is not uniform (as opposed to the uniform shift in square cell center vs. cell corner), for instance 5 ft X shift, 30 ft Y shift.    In my case, the coordinates for the point are X 1774779.68 and Y 15877180.12.

To test this, I opened a lidar DEM surface in GIS this morning.   I then loaded surveyed points, which fit where they should.   I exported a portion of the (very large) DEM, and imported into CAD.   The same surveyed points were off about 33-34 feet.   Using an online US Survey Ft to International Ft calculator, the shift for the coordinates works out to about 3.55 ft X and 31.75 ft Y distance.   The shift for the grid center is 1/2 of 3.28 feet in both directions, or an additional 1.64 feet of difference.   If I move the DEM surface in CAD 5.19 ft X , and 33.39 ft Y, the surface fits pretty good.

It appears this is a bug in need of fixing, as others have pointed out.   Has anyone found a more precise workaround?  

[ChicagoLooper]

---

@Mike_D_G wrote:

I exported a portion of the (very large) DEM, and imported into CAD.   

Has anyone found a more precise workaround?  

---

There are different ways to 'import' a DEM into CAD. What procedure or command on the ribbon (or whereever) are you using? The whole premise of a shift might be due to the appropriateness of the command.

[TerryDotson]

Sounds like you have it pretty much pinned down but no way of fixing it for the future.  Always interested in tests to confirm valid results.  If there is any way you can zip and post the GridSurface file, source coord system info and a drawing with 4 points of where it should hit (target system), it would be appreciated.

[Pointdump]

Mike,
In QGIS there is a function called Raster Pixels to Points which creates a point in the center of each pixel, which you can save as a CSV or SHP.
Dave