Shape file attributes won't list

[Neilw_05]

I'm trying to filter a shape file to show only features with a certain attribute. When I select the option to list the attributes in the filter dialog, the list doesn't populate. I know there are features because they show up in the attribute table. If I type in the attribute vs. picking from a list, the validation fails. So I can't type it and I can't pick it. I can't do anything with it.

What is going on here?

[ChicagoLooper]

Hi @Neilw_05 

Based on your uploaded images, it's hard to tell what you have done or have not done. So, it's not easy to offer a definitive answer.

Try doing it like this:

Alternatively, you may also try replacing [value] with 'FLOODWAY' so you don't have to use that clunky green arrow.

In other words, when you manually create your query, it should look like this:

ZONE_SUBTY = 'FLOODWAY'

Click the Validate Button and 'FLOODWAY' will turn green.

[Neilw_05]

I was using double quotes. It works with single quotes. I knew better. Still doesn't answer why the filter list doesn't work. It's just one of those aggravations that make you waste time troubleshooting.

I have wasted days this week doing troubleshooting in C3D, thus the very foul mood to hit this on top of it all.

At least I am able to move forward.

[ChicagoLooper]

Since you mentioned  ....Still doesn't answer why the filter list doesn't work.....

Would it be possible to share the shapefile for evaluation? That way we can determine whether it's something in C3D or the data itself.

[Neilw_05]

The shape file is from the FEMA FIRM GIS data for Arizona, 2022 version. The shape file is over 350 MB, so too big to post. If you decide to get it, let me know what you find.

[Pointdump]

Neil,

I'd like to check out your data. What's the link?

Dave

[Neilw_05]

Here you go Dave. Thanks for looking. See my snapshot to see what shapefile I am accessing.

https://msc.fema.gov/portal/downloadProduct?productTypeID=NFHL&productSubTypeID=NFHL_STATE_DATA&prod...

[ChicagoLooper · Accepted solution]

@Neilw_05 

The attribute ZONE_SUBTY is strange or sluggish. The option to 'Get values from a list' doesn't seem to work when you click the green button with the white arrow.

If I run a query with the AND operator, then the green button works. Woohoo!

Here's a query that LIMITS my area of interest to a small area, specifically the area defined by a rectangle I drew in downtown Phoenix.

The AND operator evaluates the attribute named ZONE_SUBTY and isolates only the FLOODWAY value that falls inside or intersects the defined area.   

1. This is a query that uses the 'operator' named AND.

Below is the result of the query. The query only searches the area defined by the red rectangle and at the same time, it also evaluates the attribute ZONE_SUBTY and isolates FLOODWAY. It doesn't search the entire dataset because it doesn't have to. Its search for FLOODWAY is limited, which to me means, the program doesn't have to work very hard.

2. Red rectangle is area of interest. The query applies to objects either fully inside or intersecting the red rectangle.

My theory: 

The query is forced to limit its search to the red rectangle. Items returned from the search (the query) must fall completely inside the rectangle or intersect the perimeter of the rectangle. I'm guessing, of course, when I say if it wasn't for this 'defined area' the query would work but it would take a long, long time. Anyhoo-hoo-a-choo, that's my best guess. Haha! 

Try running the query overnight. Will the green button display a list after you wake up?

[Neilw_05]

Thank you for figuring it out Rick. I had a spatial filter applied to my query as well but I didn't have any other parameters. I let the filter list process for at least 15 min. and gave up when it didn't populate.

FYI, I was planning to use this as an example for a demo to our engineers to learn about Map's GIS capabilities. I have long known Autocad/Map and FDO sucks when working with large datasets, and this exercise brought that problem clearly back to mind, not just for this reason only, but the overall performance to display the graphics as well. I am not going to present the FDO option to them knowing it will just lead them down a rabbit hole figuring out how to deal with the issues.

They will have to utilize the MapImport and Query tools.

[ChicagoLooper]

@Neilw_05 

<<…I was planning to use this as an example for a demo to our engineers to learn about Map's GIS capabilities.>>

If it were me, I’d still do the GIS demo because it gives credibility to the power of data. Instead of using FEMA flood data use parcel data. It’s readily available and free to download. Most of all though, you can run  queries that’ll be easy for your audience to understand. That alone will increase your wow factor.

Examples:

Query parcels to find…..

1. Owner name is ‘like’ Creators ‘R’ Us. 
2. Area is greater then 45,O00 square feet. 
3. With street address on Presidential Blvd. 
4. Properties sold  by ‘QRS.’
5. Properties in a rectangular area defined by you. 

Queries can also do this: 

• create labels displaying both parcel IDs AND owner name.
• isolate parcels located on this street OR that street.
• put yellow hatch in parcels below 22,000-sq ft and green hatch in all other parcels.
• color code neighborhoods according to the name of the subdivision then demonstrate how the hatch color automatically updates when the neighborhood expands with newly added properties .

MapImport can work but shapefile data might be easier to manage and faster to modify. Be careful not to shortchange GIS data. 

I wish you good luck. 

[Neilw_05]

I totally get what you are saying. My audience is civil engineers who already have the field topo survey with property boundaries. They need to evaluate the flood boundaries, base flood elevations, etc. in context with the surveyed boundary. We have local parcels in a GIS, but they are too crude to be useful for a real world evaluation. So GIS parcels are irrelevant in this scenario.

Granted when doing a preliminary review, the GIS parcels can be useful, but we are beyond that level of detail when the engineers get involved.

[Pointdump]

Hi Neil,
Thanks for posting your data.
I didn't even get as far as you and @ChicagoLooper. It was taking forever just to load that big dataset. (I tried to load all 218 files.)
I drag-n-dropped everything into QGIS, queried "FLOODWAY", and Save Selected. Zero to shapefile in under a minute.
Dave

[Neilw_05]

I don't have an issue with loading it into a dedicated GIS app. I need our engineers to bring the data into autocad in context with our grading and drainage projects. If we can't do it with Map then they will have to go out and learn about all the GIS tools and apps on their own. That's not a bad thing if Map isn't up to the task. As I said earlier,  I'm not going to lead them into using FDO connections. Either MapImport or a round trip via a GIS app in my book. If they want to deal with the issues with Autocad, that is their choice.

[Pointdump]

Neil,
"...Map isn't up to the task."
Sadly, in this instance, it isn't.
"...they will have to go out and learn about all the GIS tools and apps on their own."
Don't coddle them too much. They need to bring their own nap blanky and cookies and milk to work. 🙂
Dave

[parkr4st]

Just took a look at this one.  Don't stay up all night like some  do:) 

Get the shp into the map and save it as a SDF file to start.  

SDFs work better in map3d.

attach the SDF and use the query to get the rows of data where FLOODWAY is.  Then only FLOODWAY data is in the layer.

Save that layer to a SDF.

Now the map will redraw quickly as you theme the layer which seems to be the goal aside from the data itself.

The original data as SDF is not attached - file is to large

This could be done with all .shp files of course if need be.

D &D the .layer files into the map.  To prep for a demo, do the theming in various ways and save to the .layer files.

[Neilw_05]

Granted there are workarounds. They won't know what to do though. I would have to get involved when these issues arise. Lucky me 🙂

Actually I've already done something similar by importing the shape files into a dwg. They can query what they need and trim with map trim after. They won't have to feel with the doggy performance and they can manage the data in the cad environment, which is ultimately how it ends up anyway.

[ChicagoLooper]

Here's how you can do it in under 30 minutes, that's a conservative estimate. (This does NOT include the time it took to download the data.) The link in Post #7 of this thread downloads a geodatabase (Gdb), not shapefile.  

Since Map3D is capable of connecting to Geodatabase using the data Connect Palette (FDO), I revisited this exercise and developed a more streamlined workflow.

1. Connect to the FEMA geodatabase like this. See image-1.

Image-1

HINT 1: Explore the Data Connect Palette. When connecting to the Gdb it'll be a more direct approach. Pulling the needed feature class out of the Gdb and converting it to shapefile or some other format might be an unnecessary and time consuming step. 

2. Scroll down to S_FLD_HAZ_AR. That's the feature class I know I want because it's the same file in the image of Post #1. This feature class has attribute ZONE_SUBTY which is the attribute I want to query (or isolate) for the value known as FLOODWAY. This 'connection' took me less than 1 minute, your actual time may vary.

Instead of clicking Add to Map I chose Add to Map with Query. See image-2.

Image-2

HINT 2: If I didn't know the feature class within the Gdb then this part of the procedure might, or might not, take longer. Since I did know, I went for the jugular. I'm not bothering with other feature classes because bringing them into M3D may bog down the program. Streamline the workflow, remember?

3. Here's the query. The green button that didn't work in Post #1 works now. Buya! Writing this query took less than 30 seconds. See image-3. 

Image-3

HINT 3: You can add the Gdb file to your drawing and delete it later. Adding it to your drawing, even if only temporarily, gives you the benefit of 'viewing' the Attributes and their values. When you compose your query, you'll have better focus and improve your success getting a valid query. 

4. Here's the result of the query. The geodatabase returned 1,404 rows (or instances) of FLOODWAY when evaluating ZONE_SUBTY. And it did it using the green button. Compared to the method in Post #1, the green button didn't work....and it appears it was only querying 26 rows of data and not the entire dataset. See image-4.

Image-4

HINT 4: Run the query from the Data Connect Palette so you only connect to the part of the geodatabase you need (refer to image-2). Don't run it from the Map Task Pane because running it from there means you have to bring in ALL the data then decide which data you want to display and what data you don't. If you have no intention of displaying the data, why are you bringing it in? Extra baggage, leave it out.

5.Here's the image from Post #1, the one where the query didn't work. The Table indicates only 26 rows of data, not the 30,000+ rows in S_FLS_HAZ_AR. The green button doesn't appear to work (or might work if the program is given enough time). See image-5.

Image-5

6. BONUS. If you want to make it easier for dabblers of Map3D, you can do this. No, no, no it does NOT improve the workflow above, it simply presents the data in a more user friendly way.

Pull the required feature class out of the downloaded Gdb. You can save it as shapefile in shp format (this means export the feature class to SHP format).

Using Google Earth Pro menu bar go to File=>Import or Open, either will work=>Browse to your newly exported (saved) shapefile=>OPEN. This will add the shapefile to GE Pro. Clicking inside a polygon will automatically reveal a 'bubble' containing the exact same data attributes of the feature class. 

HINT 5: Establish your area and zoom-in. Be sure to give yourself some 'cushion' by zooming to a slightly larger area. This importing of shapefile process can be time consuming. This exercise however, took less than 3 minutes, your actual time may vary. Of course, larger datasets will take longer. Just before adding the linework, you'll be asked if you want to import the entire dataset OR just what fits in the current view of your monitor. Choose the current view, always choose the current view. Choosing the entire dataset will take longer which is why you want to establish your zoom level ahead of time.  

If you're ambitious, you can export your Civil parcels to shapefile too. That way those dabblers can zoom-in and -out and play with geospatial stuff that contains data.

Dabblers shouldn't have the option to manipulate data. Read the data, yes, definitely yes. But manipulate data, be careful if you say yes. Sure, they can access all the Cad and Civil stuff and even modify that if they want, but the data and the workflow, especially the workflow, it's too overwhelming and it doesn't benefit anyone when they're lost and not up to speed. The benefit is in the way a qualified user 'presents' the data alongside the Cad stuff, of course, and how creative he is at doing it.

Attachment: The FEMA Floodway in kml and kmz formats.

[Neilw_05]

What I meant about manipulating the data was they can stylize, label, change display order, etc. They would have no reason to physically edit the features considering they want it to represent the true source data.

I was in a bad mood after a week of software hassles. Having Map misbehave as well was the last straw.

Now that I have cooled down I can see where they would benefit from FDO connections as it allows them to easily view shape files without having to convert, query, etc. if they start to have performance issues with a dataset we can then look at conversion.

By the way, we do have parcels in shp format. I have imported them into a DWG for easy queries as well.

[Neilw_05]

I have them in kmz, kml as well. I imported the shape files in GE and then saved as KMZ. Is that how you did it?

Oops, I re-read your post and saw that is what you did.

[Neilw_05]

I read through your post again and I hadn't noticed your notes about connecting to the GDB. In the past, Map had limitations with GDB so I never even gave a thought to connecting to it. This is great information in many ways. You are the new Murph for the Map forum.