Problem with Tif images in an unmanaged datasource

[jacklloyd]

Has anyone had a problem with imagery shifting randomly or not displaying in the proper location when it comes from an External Alias Folder? I have about 1300 TIF images connected by an Alias. When I pan around, it looks like the image tiles randomly shift. One time they all line up, then I pan and some of them shift. Pan back, and they are okay. These are the same images that are used in MG 6.5 with a RIC file, and they work perfectly. Is the unmanaged datasource (alias) supposed to be the option to be used in place of a RIC file in MG 6.5?

[dswilson]

Can you specify more details please? Is this MGE 2008 or 2009 for example. Are you in a flexible web layout or a basic layout? Are you using the AJAX or the DWF viewer? Can you attach some jpg images showing the issue please?

The unmanaged source is the corrrect approach.

Dave

[jacklloyd]

Thanks for getting back to me Dave -

MGE 2009
Happens in Studio on a "Map" before even creating a layout

I've attached 2 images. Sample1.jpg shows an area with a street network overlayed on an orthophoto layer. The area in red shows where the change will take place. Sample2.jpg is zoomed into that area with the red outline basically the same area outlined as in sample1.jpg. In the upper left part of the image you can see how the image has changed content. If you look at them side by side, it's easier to see. This is just one area, it happens in other areas as well.

Aside from this, performance is really poor with the Tif images. A single ECW mosaic performs better. A WMS connection to ERMapper's Image Web Server (IWS) is the fastest. With the ECW and IWS this problem doesn't happen because the mosaic was done outside of Studio.

Is there anyway to import or reference the existing 6.5 RIC file in MG Studio? Let me know if you need anything else.

Thanks - Jack

[jacklloyd]

Sorry - Forgot to attach the images. Here's 1.

[jacklloyd]

Here's 2.

[jacklloyd]

Dave -

Here's the URL if you want to try it yourself.

http://gaston.dot.pima.gov/mapguide2009/mapviewerajax/?WEBLAYOUT=Library://WebLayout/PCDOTMGE2009-1.WebLayout&LOCALE=en

Username is Anonymous. There are 2 problematic layers:

2005clr02ft
2005clr02ft-tfws

The first one was created with the tfws in the directory. I thought that they might have been causing the problem so I created the "NoTFW" layer with the tfws removed from the directory.

Another odd thing that's happening is that not all of the images are showing up, and it's different for both layers. I guess they get dropped from the list (where ever it's located) when the layers are initially created?

The area I sent as jpgs were around TRS 121234, which is located in the upper center of the County. If you turn on the TRS layer it will help you find it.

Jack

[Anonymous]

Hi Jack, not sure what's wrong with your tiff image shifting problem.

Based on my experience in using Autodesk FDO Provider for Raster with MGE
for good performance, here are some suggestions:

1) A single ECW file can give you the best performance in MGE, which is
different from MG6.5. So whatever possible, convert your multiple ECWs or
TIFFs to a single ECW file (size <4GB) for a MGE image layer.

2) For better performance of a TIF image layer in MGE, you need to 1) reduce
the total size of the files by creating subsampled (and tiled) TIF files
(using Raster Workshop) and/or 2) reduce the total number of the files by
merging your TIF files with a tool (e.g. FME or FWTOOLs). In your case, 1300
of TIF files is defintely too many. MGE is different from MG 6.5 and it
will not read your MG 6.5's RIC file. In my case, I subsampled my 347 TIFF
files of 2 GB to the same number of 36 MB (about 1.5% of the original), and
use the subsampled files for the image layer from infinity to 1:6000, the
original TIFF files for a detail scale (>1:6000). This way, the display time
is under 10 seconds.

3) You have to use your TIF files together with their TFW (a ESRI world file
for georeferencing info) files under the same folder, unless your TIF files
are GEOTIFF files which mean the georeferencing info is incorporated within
the TIFF files (as a file header info).

Meng

wrote in message news:5921372@discussion.autodesk.com...
Dave -

Here's the URL if you want to try it yourself.

http://gaston.dot.pima.gov/mapguide2009/mapviewerajax/?WEBLAYOUT=Library://WebLayout/PCDOTMGE2009-1.WebLayout&LOCALE=en

Username is Anonymous. There are 2 problematic layers:

2005clr02ft
2005clr02ft-tfws

The first one was created with the tfws in the directory. I thought that
they might have been causing the problem so I created the "NoTFW" layer with
the tfws removed from the directory.

Another odd thing that's happening is that not all of the images are showing
up, and it's different for both layers. I guess they get dropped from the
list (where ever it's located) when the layers are initially created?

The area I sent as jpgs were around TRS 121234, which is located in the
upper center of the County. If you turn on the TRS layer it will help you
find it.

Jack

[jacklloyd]

Thanks Meng. I have no idea what's causing the shifting either. Like I said before, they work fine in MG 6.5.

I'm very familiar with subsampling tiffs to reduce the file sizes, that's how we do it on 6.5. We use lo-res ECW mosaics at high scales and hi-res tiffs from a RIC at low scales for best performance. I guess one difference between your site and ours in the number of images. There's no way I could do a <4 GB ECW and still have somewhat decent resolution.

I also find using the WMS connection to Image Web Server out performs raw ECWs using the raster FDO at least 2:1. Our users are spoiled by very fast imagery. If they had to wait close to 10 seconds for an image to display I'd never hear the end of it. I was hoping 2009 would have better performance. One step at a time I guess....

I'll keep playing around to see if I can figure out what the shift problem is. Maybe someone from adesk will figure it out.

Take care - Jack

[dswilson]

Hey Jack.

Even in the map preview the default viewer with MG 2009 Studio is set to use a Flexible Web Layout. If you could change your viewer from the Tools\Options menu within Studio to the AJAX Viewer and retest the map preview that might tell us if the problem is specific to the viewer or generic to the server.

The RIC equivalent is saved when the feature source is created. You can view it by doing the following:

Navigate in your browser to Http:///mapguide2009/mapagent/index.html

From there you can EnumerateResourceData by providing the path to your feature source in your Repository. You can easily grab the path by right clicking and getting the properties of the feature source from the tree in Studio.

There should be a config.xml or similar file associated with your feature source. Then you can execute GetResourceData by supplying the path and the file name from the prior request. This xml file should contain what is roughly the MGE equivalent to the RIC used in MG6.5.

Regards,
Dave

[jacklloyd]

Dave -

I switched to the basic Ajax viewer in Studio and still have the same problem with the shifting images.

I am able to see the xml created from the initial image load. Do you think the shifting is coming from this file? I find it odd that the images tile fine at one scale and at another scale or location (from panning in the view) they shift. Wouldn't the image positions all be coming from the same xml file?

Maybe I should compare the coordinates in the xml file to the coords in the RIC file for the same images. I'll do that just to see if there are any differences.

Thanks - Jack

[jacklloyd]

Dave -

If you want, I can send you the xml file for the images in MGE and the MG 6.5 RIC file. I did some spot checking of insertion points and bounds and some were identical, and some were not. I'd think they should be the same if they were generated from the same set of imagery. Let me know.

Jack

[dswilson]

Ok so we can at least determine the that issue is server side and likely specific to the provider and not related to the Web Tier and the viewer. 1300 files not that large in comparison to some datasets we have either encountered or heard of.

Unless some of the points in the file are grossly off I might expect some differences due to precision of the values. If you can find some values that are completely off then please identify some and provide the files for comparison.

Dave

[jacklloyd]

Dave,

The coordinate differences were not that far off for the ones that were different. I'm starting to think it's not with the coordinate values in the xml file. If it was, why would the images display correct one time and then display in a different place when zoomed in closer or by panning around. Here's an example of a difference in the xml and the RIC file for the same image:

-
-
-
-
1005961.5
429580.5
2
2
0
0

-
1005961.5
429580.5
1012051.5
435562.5





"E:\images3\Ortho2005\Images\clr02ft\14142802.tif",1005962.000000,429580.000000,1012052.000000,435562.000000

They are generally off by 0.5 to 1.0 units.

Do you know Carsten Hess? I gave the Geospatial team this full imagery set last summer. When I was in San Rafael last November they had the external hard drives that had this imagery. I gave it to them for testing purposes.

Jack

[dswilson]

I'm beginning to think this is a data issue. We have a copy of the PIMA data in house and this problem only appears to happen with the 2ft data in the TRS you specified. It doesn't happen with the 4 foot. I also notice in the same section that the 2 ft data skips a tile or 2, but the 4 ft doesn't.

Dave

[jacklloyd]

That's a possibility. Maybe MG 6.5 is more forgiving? It would be nice to know "why" it's happening though, in case it happens with other imagery sets. Will you be able to determine why if it is the data?

Thanks,

Jack

[dswilson]

If you haven't logged a support call on this one you might want to. That increases its chances of getting looked at sooner. It's also possible it might be specific to the tif format. I'm curious if the same thing happens with tiled tif. Do you know what if any compression is used on the data?

Regards,
Dave

[jacklloyd]

Dave,

For this imagery set, the 1ft, 2ft and 16ft came directly from the vendor. the 4ft, 8ft, 32ft and 64ft was re-sampled by us. I'm going to see if the 1ft and 16ft imagery has the same problem. There was no compression used on this imagery.

Jack

[jacklloyd]

Dave - some information about the imagery

For the 4, 8, 32, 64ft imagery, we stripped the GeoTiff header out of these images by turning the image into a Grid in Arcinfo, and then saving it out as a tif.

For the 1, 2, and 16ft imagery, these are GeoTiffs.

When MGE creates the XML file of the image coordinates, what does it read first, GeoTiff header or world file, if one exists? I've also heard that the anchor point in a GeoTiff is the center of the upper left pixel and where in a world file, it's the upper left corner of the upper left pixel. On a low resolution image, this could cause a pretty big shift in the position of the image.

This imagery came from Sanborn. I'd guess there might be other locations out there that are using imagery flown by Sanborn.

[dswilson]

Another thing of interest which may or may not be a factor since I would have thought an exception would result in MGE 2009, but when I refresh the spatial context list on the feature source in Studio I get 3 entries for the 2 ft, but only 2 for the 4 ft. The additional one in the 2 ft is AZHP-C (which is meters) whereas the others are unknown. I assume it's getting this from the GeoTiffs or at least some of them. I assigned the unkown entries to AZHP-CIF (which is in feet) based on the document we got with the data. However MGE 2009 won't transform the raster data and it will generate an exception if the coordinate systems don't match. It may be that it fails to pick up tiles or it is misinterpreting the data and positioning it incorrectly. That doesn't explain why it might work in MG 6.5 and it doesn't explain the difference at different scales in MGE.

Dave

[dswilson]

We have done some more investigating and it definitely appears to be a bug, whether it's data related or not we don't know for sure. However none of the images in any of the stuff we have appear to be Geotiff.

The shift doesn't always occur. You can pan and sometimes it's ok.

At least it's reproducible.

I would still urge you to log a support call on it if you haven't done so already.

Dave