DEM Raster to Surface

[izzybabur]

Hi,

Does anyone have suggestions on creating surfaces from USGS NED DEM data?

My current workflow looks like this:

1. Download 10m or 30m DEM data online
2. Import the data into ArcGIS and transform it to a projected coordinate system
3. Clip the data (my alignments are over 350km long and usually go through at least 4 tiles)
   • I find it easier to clip the data in ArcGIS by creating a polygon and shapefile around the corridor
4. Export the data
   • I tried GRID and TIFF files (can't find other options that Civil 3D will read)
5. Open Civil 3D, set the coordinate system, and click "create surface from DEM"
   • I tried GRID and TIFF files and then Civil 3D either hangs or says "unable to read file" or some sort of error like that

I feel like this workflow is quite long and frustrating - also it doesn't work. Does anyone have suggestions or alternative workflows? I want the surface data that I'm pulling to work well with Infraworks when I eventually bring in a corridor model from Civil 3D to Infraworks. Unfortunately, I can't pull the surface data from Infraworks because of the 200 sqkm limit in Model Builder. 

Any help or suggestions would be greatly appreciated!

Thanks,

izzy

[Pointdump]

Izzy,

Here's a pretty good <<<Workflow>>>.

Dave

[neilyj666]

Over a 350km length even with a 10m or 30m DEM that is a serious amount of data that will cause Civil to choke - have you considered splitting it into more manageable areas?

[L3GO-Ed]

Izzy,

We also follow part of your workflow, except that after clipping the DEM, we automatically create points within the clipped boundaries using ArcGIS, then we use the join tool to attach elevation values to the points from the DEM - still in ArcGIS.
We export the points to a CSV and then we add the points to a created surface in C3D.

Ediriisa.

[L3GO-Ed]

I haven't seen Dave's video but I think there should be a way in Map 3D.

I also have a colleague who adds the DEM to infraworks and exports it to IMX into C3D

[JamesMaeding]

What we do s this:

1) Set coord system to desired projected system (CA83-VIF for us)

2) make blank surface

3) add a data clip boundary to spatially limit the data that will be read in (critical, do not skip)

4) add dems to the surface, using UTM27-11 as the system the dem is on.

I never make surface from dem, that just has not worked for me.

Now that workflow has two (at least) flaws:

1) adding every dem one at a time is cumbersome if many. Why didn't the c3d team let us do multiple!!!

2) no thinning out of data if you don't need density of points in the dem, only after and that will likely take forever in c3d with lots of data.

My solution would be to write a .net prog to read in the points, limit with some polyline I picked, then thin out somehow, maybe my making mini surfaces and resampling at some grid spacing. People that know other tools like the SAGA, ESRI, or Manifold can do that with those.

Either way, combining and downsampling the dems outside of c3d will save tons of trouble IMO.

[izzybabur]

@L3GO-Ed thanks for that suggestion - I actually had converted them to points over the weekend and imported them into Civil 3D, later to find out that no Z values where imported  .. Do you have a quick ArcGIS workflow (or link) to map the Z values to the points? 

@Pointdump that's a pretty awesome video - I did not know that you could create a point cloud from a GeoTIFF. I have tried the method from that link, but for some reason, Civil 3D is still unable to read my GeoTIFF. Essentially, my file was made from:

• 6 ArcGRID rasters (tiled) that are merged (using the mosaic to raster tool in ArcGIS)
• The merged raster is then clipped to a boundary shapefile (I made the boundary in Civil 3D as a 500m wide right of way which is about 350km long)
• I then export this clipped raster to a GeoTIFF using ArcGIS
• I tried importing this many different ways, including the link suggestion that you sent (great link by the way) - but keep getting errors in Civil 3D that it can't read the file. I also tried it with a GRID format as well. 

@JamesMaeding what format is your DEM in? I like your workflow and @Pointdump's workflow, since it stays in Civil 3D. However, I always seem to have some sort of issue. Is there a way I should try importing each of the original ArcGRID tiles (i.e. "USGS_NED_n35w117_1_arc-second_2013_1_x_1_degree_ArcGrid")? I believe this would mean importing the ADF file and try clipping it to a boundary. What do you all think?

@neilyj666 I am trying to split it and clip it in ArcGIS to avoid the hanging issues in Civil 3D. Or use a boundary initially in Civil 3D. 

Thank you all for all your help so far - I am chipping away at some workflows to resolve these issues today and if I make any breakthroughs I'll let you know. In the meantime, any additional suggestions would be greatly appreciated!

Thanks,

izzy

[JamesMaeding]

The DEM's I have are straight from USGS, not made by myself.

That workflow of mine is rock solid for USGS California DEM's at least, using CA83-VIF for drawing and UTM27-11 for dem system.

Make sure you are setting that dem system in the dialog, you have to click once in the field to get the ... button, then click the button to choose system.

Its one at a time though like I mentioned.

If you cannot get those in, keep on it and reply until it works, that is fundamental.

Then you can talk about pulling in other data formats (IMO at least).

I did use the SAGA gis tools once to combine 25 10m dem's into one, and resample at 100m grid interval.

It was fast, while other tools I tried were too slow. Use its "Grid-Tools" items, its all free.

Its like you need to get to  a tool without heavy GUI to do the heavy lifting. I'm not a great ESRI guy though so...

[izzybabur]

Hi @JamesMaeding,

I tried your method with the native USGS files (.adf), and added them one-by-one. Here's my workflow, which is pretty much the same as yours:

1. Create new drawing and set projected coordinate system (in my case UTM Zone 11N)
2. Import alignment polyline (shapefile created in ArcGIS from a KMZ) using the mapimport command
3. Offset my polyline 250m on each side (using offset command), add two polylines lines (one at each end) to join the ends of the offset polylines, and close the polylines (using the pedit command)
4. Create a blank surface
5. Add a boundary to the surface by selecting the closed polyline in Step 3, and setting the type to "data clip"
6. Add the first DEM to the surface
   • Set the coordinate system of the DEM to LL83 (my data set is USGS NED 1 Arc-Second and comes in Lat Long Decimal Degrees by NAD83, with Z values set to NADV83)
   • Current drawing coordinate system is UTM Zone 11N (as specified in Step 1)
7. Add the second DEM to the surface, and specify the coordinate system of the DEM
   • Continue this until all 6 DEMS are added
   • This takes a quite a while, but it works
8. Now, I have quite a curvy alignment for 350km, so although my data clip is applied to the imported DEM, I had to create another surface boundary definition (since triangulation occurs outside my data clip boundary on the curved portions)
   • I right click boundaries under surface definitions, and add an "outer boundary" by selecting the same polyline created in Step 3
9. I then rebuild the surface, which takes about 10 minutes on my machine
   • Question: Would this run significantly quicker on a server cluster machine? 

I think that my main issue was the inter-operability of ArcGIS with Civil 3D. Creating the GeoTIFF or ArcGRID in ArcGIS works with other GIS software (such as SAGA), but Civil 3D is unable to read it. I guess I should stick to the native metadata files and work with Civil 3D.

I am still exploring other workflows that have been mentioned in this post, and I would love to see if we could get a faster, non-gui intensive, and more efficient workflow ironed out. If not, I'll write a C# .NET add-in when I get a chance!

Thanks again everyone! Feel free to post any other suggestions - otherwise I'll close out the post 

Cheers,

izzy

[Kalhab]

Hi izzy,

I use DEM raster to create surface for preliminary engineering design and it works fine but I prefer global mapper.

Workflow

>open your centerline alignment as a polyline from dwg or line from Google earth as kml or kmz as well as your dem raster using global mapper software.
>transform to projected coordinate system.
>select the line and create buffer at offset distance to the right and left then it create a mesh
>select the mesh and export to elevation grid format (since you've many format option select DEM)
>create your surface using C3D from DEM.
try this one it'll work.

[tcorey]

I second your recommendation of Global Mapper for processing DEMs before you feed them to Civil 3D. It's a great program, well worth the money.

[JamesMaeding]

Also manifold GIS is a great medium cost option - $500 range and less:

http://www.manifold.net/info/manifold8.shtml

I've used a few but need to look at global mapper again.

[dsmapping]

Hey Tim, I use GM to process DEMS but I cannot figure out how to get C3D to read them correctly. After I import the dem as a geotiff C3D tells me that no z value was imported. Im importing by right clicking on "create surface from dem" in the toolspace menu. Is there a better way to do this?

[dsmapping]

Is there correct way to import the DEM into C3D from GM? I import by right clicking on "create surface from dem" and import the geotiff I processed in GM that way. But after I do this it tells me that no z value is there. Why would this be? It looks awesome in GM but I need to make a tin surface from the dem...it is what the client has asked for.

[Pointdump]

Hi Dennis,
"create surface from dem" is fraught with peril. Instead, create a TIN Surface and add the DEM to the surface definition.
Dave

[tcorey]

Make a TIN surface and then add the DEM as a data source.

[dsmapping]

I cant figure out why after I import a dem to make a surface in C3D it tells me there are no Z values. Is there a better way to import it?

[JamesMaeding]

@dsmapping 

If you mean a .tif (geotiff), I pull those into QGIS first to see them.

QGIS is free and ultra fast at dealing with large images. You did not mention how large your file is.

I also generally "launder" my geotiffs by then resaving them as geotiff of slightly different name.

I think it can also read .dems but not sure why your file is having issues.

[Pointdump]

Hi Dennis,
"no Z values".
Let's see what you've got. Please post a link to the original DEM and attach your GM modified Tiff to compare.
Dave