Extract a small section of data from a larger shapefile

[FJankovsky]

I have searched through the forum and found different approaches to solving this problem.  After combining methods from various posts I have come up with this workflow which seems overly complicated.  I am posting this with the hopes that someone has a more direct solution to my end result. 

I have a map that covers hundreds of square miles and has data connections to numerous shapefiles.  For field work I need to extract the various Map Features within a smaller area, let's say one square mile, and then send these smaller shapefiles to third party software on an android tablet.  Unfortunately, the third party software is not able to handle the larger shapefiles and this is the reason for the need to work with only a portion of the original files.

My reading has lead me to the following process:

1. Open a new map and assign proper coordinate system.
2. Turn on an online map of the area.
3. Use Data Connect to connect to each shapefile and then instead of using  Add to Map, use Add to Map with Query.

4:  Select Filter features by locating on map

5: Then use the desired shape and condition to select an area on the map to bring in only those Map Features in the desired area.

6:  In the Display Manager highlight the shapefile and right click and select Export Data Layer to SDF.

7:  Either start a new map or deselect the shapefile in Display Manager

8: Use Data Connect to connect to this SDF file

9:  To create a new shapefile from the imported SDF data that includes all the Object Data, first create an empty shapefile by the following method: Data Connect>Add SHP Connection>Folder-Not File, then click Connect.

10:  Go to Task Pane>Map Explorer>SDF>Then select the Data, in my case Handholes, then select Tools>Bulk Copy

11:  In the Bllk Copy window on the left pane pick the SDF file as the Source and check the Data that you want to copy.  In the right hand pane select the "Empty" SHP connection previously made to the Folder as the Target.  Click Copy Now and if successful the Bulk Copy Results window will pop up.

12:  The resulting shapefile now has all the data in the data table and is only for the smaller area selected on the map.

This is a lot of steps and if there is a better way to reach this end I'll be very appreciative of any suggestions.

[ChicagoLooper · Accepted solution]

Your method works. Good job explaining! Your screen grabs, they’re good too! Kudos to you! All other posters, you should bookmark this page. This is how you thoroughly explain yourself.

 So, is there a better way? That’s hard to answer. Better is a relative term. Better than what? Better than a root canal? Better than bacon on a Sunday morning? Better than your 12 steps?  IDK if this is better than your regimen but it’s probably more streamlined.  Give it a try.

Setup: I have a street shapefile (Trans_RoadSegment_NTDNOFS.shp) for the city of Chicago. I don’t want to bring in all streets. I only want the streets for a small area adjacent to Navy Pier. To define my small target area, I have drawn a green rectangle.

1.       Open a brand new, clean template. Assign a projected coordinate system applicable to your site. Turn on Bing Hybrid aerial map and zoom in to your area. Draw a Rectangle around your target area. Draw your rectangle slightly bigger but not too big, making it both taller and wider, than your desired target area. See Image-1.

Image-1.

2.       Use MAPIMPORT command. If you prefer the ribbon go to Insert Panel=>Import Tab=>Map Import Icon.  See Image-2.

Image-2.

3.       Change ‘Files of type’ ESRI Shapefile (*.shp) then browse to and select your shapefile. Click OK. See Image-3.

Image-3.

4.       In the Import dialog box, make your selections as shown. The key to limiting the shapefile is the DEFINE WINDOW function in step 4. See Image-4. Note: Depending on the size of your shapefile, the import may take more than a minute.

Image-4.

5.       The Mapimport command converts ESRI entities into AutoCAD entities during the import procedure. Mapimport does NOT require you to use a Data Connection to bring the big ol’ shapefile. Once the streets are imported to modelspace you may, if you wish, control the appearance of the imported lines in the Layer Prop Manager, i.e. color, layer name, linetype, line weight, etc. The Layer Prop Manager shows the new layer name has adopted the shapefile's name by default. The layer color defaults to white during import. You can change the default color to yellow in Layer Prop Manager. See Image-5.

Image-5.

6.       If you wish to proceed further, you may continue and use the MAPEXPORT command which will convert this subset of line work back to shapefile. Name the subset ‘TargetStreets’ and save it in a folder. See Image-6.

Image-6.

7.        The Mapexport dialog box has three tabs. The settings of each tab is shown. You may, or may not, adjust the settings based on the ‘object type’ you are exporting to shapefile. See Image-7.

Image-7.

This concludes the ‘Extract a small section of data from a larger shapefile’ procedure. I feel it is less clunkier than yours. Your procedure involves Bulk Copy, this one doesn’t and IMHO the above procedure is more intuitive too. Don’t get me wrong, I’ve used bulk copy and fully understand what it achieves, I just don’t think Bulk Copy is needed to create the smaller subset of what you want.

Finally, test your new shapefile by bringing it into a drawing using a Data Connection. In the Task Pane, view the data in Table View and turn on Bing Hybrid imagery to verify geospatial accuracy. Add street labels and play with street color and street line thickness. 

[braudpat]

Hello

Sometimes I am working with Huge 2D GIS DWGs (300-850 Mb) ! 

My present record is a 850 Mb 2D DWG file coming of course from SHP files ...

Sometimes I have to query these GIS files mainly with geographical parameters

and sometimes with attribute parameters ...

With these huge files , I almost always import (MAPIMPORT) GIS files into DWG + Object Data Attributes ... 

Why ? ---- Because query from drawing group of MAP is very FAST !! 

----  A few infos and query results with ACAD MAP 2017 x64 ---- 

-- Total size in SHP format = 3 900 Mb --
Town = 13 Mo , Suddivision = 49 Mo , Parcel = 3 130 Mo , Building = 677 Mo

-- Total size in SDF v3 = 1 000 Mb --
Town = 15 Mo , Subdivision = 48 Mo , Parcel = 811 Mo , Building = 145 Mo

-- Total size in DWG 2010 Format (from MAPIMPORT) (with Layer + Space Index) = 530 Mb -- 

-- Number of entities (2D LWPLines with ODs) per Layer -- DWG = 530 Mb (with Layer + Space Index) -- 
Town = 288 , Subdivision = 4938 , Parcel = 1 279 777 , Building = 449 151 

The bounding box is about : Height = 138 km , Width = 128 km

I have done a query with 2 geographical parameters : 2 capturing circles with a radius = 5 Km (with OR Mode)

... circle1 OR circle2 ... 

MAP Query on the 530 Mb DWG (1 file) = 17 secs !!!

MAP Query on the 3 900 Mb SHP Files = 4 mn 30 secs

MAP Query on the 1 000 Mb SDF v3 Files = 40 secs

I have done tests on a Dell notebook (32 Gb) with a classic (SLOW) Hard Drive on Win 10 Pro

I imagine that with a FAST M2/SSD drive the result would 2/3 times BETTER !?

Good Luck with your huge data ... I hope these Infos will help you !? 

THE HEALTH (Stay Safe, Stay Home, Stay Live), Regards, Patrice (The Old French EE Froggy) 

[FJankovsky]

Your solution is much simpler and more like bacon on Sunday morning where mine was more like a root canal.

Thanks for the input.

[FJankovsky]

Thank you for your reinforcement of this process.  Much simpler.