Faster primitive data extraction

[Anonymous]

Hello,

So I was trying to export the coordinates of models in a nwd file using the guide on the aec blog. However, when I try to do that, the execution is very slow. It extracts the coordinates in triangles. Does anyone have an idea as to what I might be doing wrong? Or maybe how to make it faster?

The blog post in question is this: https://adndevblog.typepad.com/aec/2012/05/get-primitive-from-solid-of-navisworks.html

[Anonymous]

This is the sample code of what I'm doing: I'm getting the coordinates using this code and simply writing them at the end of the Execute function.

using Autodesk.Navisworks.Api;

using Autodesk.Navisworks.Api.Plugins;

using ComBridge = Autodesk.Navisworks.Api.ComApi.ComApiBridge;

using COMApi = Autodesk.Navisworks.Api.Interop.ComApi;

 

#region InwSimplePrimitivesCB Class

class CallbackGeomListener : COMApi.InwSimplePrimitivesCB

{

    public void Line(COMApi.InwSimpleVertex v1,

            COMApi.InwSimpleVertex v2)

    {

        // do your work

    }

 

    public void Point(COMApi.InwSimpleVertex v1)

    {

        // do your work

    }

 

    public void SnapPoint(COMApi.InwSimpleVertex v1)

    {

        // do your work

    }

 

    public void Triangle(COMApi.InwSimpleVertex v1,

            COMApi.InwSimpleVertex v2,

            COMApi.InwSimpleVertex v3)

    {

        

    // do your work

 

    // get the first value of the vertex

    Array array_v1 = (Array)(object)v1.coord;

    float fVal_nw2013_VS2010 = (float)(array_v1.GetValue(1)); 

    
    Class1.Coords.Add(fVal_nw2013_VS2010);


    }

 

 

 }

 #endregion

 

 #region NW Plugin

[PluginAttribute("Test","ADSK",DisplayName= "Test")]

[AddInPluginAttribute(AddInLocation.AddIn)]

public class Class1:AddInPlugin

{
    public static List<float> Coords = new List<float>();

    public override int Execute(params string[] parameters)

    {

        // get the current selection

        ModelItemCollection oModelColl =

            Autodesk.Navisworks.Api.Application.

                ActiveDocument.CurrentSelection.SelectedItems;

 

        //convert to COM selection

        COMApi.InwOpState oState = ComBridge.State;

        COMApi.InwOpSelection oSel =

                ComBridge.ToInwOpSelection(oModelColl);

 

        // create the callback object

        CallbackGeomListener callbkListener =

                new CallbackGeomListener();

        foreach (COMApi.InwOaPath3 path in oSel.Paths())

        {

            foreach (COMApi.InwOaFragment3 frag in path.Fragments())

            {

                // generate the primitives

                frag.GenerateSimplePrimitives(

                    COMApi.nwEVertexProperty.eNORMAL, 

                                   callbkListener);

            }

        }

        //Over here I write that data to a file.

        return 0;

    }

}

  #endregion

So please do let me know how do I make it faster? Or is there something here that I'm doing wrong?

Please let me know,

Sami

[alexisDVJML]

Nothing wrong in your code I would guess.

Extracting primitives in Navisworks via the API is F...ING **** slow to the point of ridiculous.

Many developers have faced it, including myself.
Calls (by post here) to Autodesk, with the exception of best efforts by Xiaodong Liang, seemed to have ended in a dark matter black hole.

Since we can't rely on a sudden breakthrough in inter-galactic space worm travel, your options are:
- give up on your project/idea (which I would guess has been the most common choice here)
- cache the parsing so at least you incur the 'cost' only once per object (my current implementation)

- find someone who had/will reverse engineer navisworks file formats and let us know or even better give us an implementation so you/we can access this data directly as a block of bytes and parse it ourselves (let me know and I will send you/him a nice bottle of french Champagne) while still using Navisworks API for all other stuff...

[Anonymous]

@alexisDVJML thanks for the reply. The thing is, I've seen the implementation for the extraction being used by one of the plugins and it is much faster. Thats the only reason I picked up the project. Cuz I thought that since this was the only way to access the primitives and extract them, this might be how they did it. It was until recently when I compared the timings their plugin and mine. 

What I would also like to know is that is there another way of doing this? Say using automation app or a hybrid of the two?

I want to pursue this because the plugin I've mentioned is doing so and very quickly. If you'd like I can send a private message to you with the plugin name.

But please let me know what you guys think.

Also, could you please explain and elaborate your implementation of caching? It'd be helpful.

Regards.

[alexisDVJML]

Sorry for late reply, busy past days on other topics.

Yes, would be great to know which plugin is doing this fast, maybe can give clues, you can email me at alexis@idigo.com.sg.

Re: caching, my code is not clean/independent enough to be shared, but the concept is simple:

- I have 2 levels of primitives cache: memory and file

- I have a class (named Tesselation for historical reasons) that store the equivalent as native C# types, aka list of fragments with points, snap-points (and should also contains lines and points to be a full equivalent but I do not need these for now) and able to be saved/load from a file

- I have 1 method that build a file path from a model item (document folder / model file path '.peekee / item path  '.prm'

When I need an object primitive I look in my in-memory cache (a dictionary) for a Tesselation for this object. If I have I use it. If not I look for a saved Tesselation file, load it and add it to my in-memory cache. If not, I extract the primitives using the standard method you use, store them in my Tesselation object, save this Tesselation data to file and add it to the dictionary, then I use it 😉

I'm planning to replace individual files in the future by some kind of simple hard coded database to avoid multiple file access and bloating the hard disk.

Saving to disk in the same folder as the original Navisworks document allows this cached info to be persistent, which make a big difference in user experience in my plugin due to the specific use case where user add items for later processing.

Of course, all this complexity and disk access would probably be useless if we can extract the primitives efficiently.

[Anonymous]

Hey @alexisDVJML,

Sorry for the late reply, I was tied up in a separate project.

I've sent you an email regarding the same a while back. Was hoping if you had received it or not. I'll be happy to forward it to you again if you'd like. I'd really like your insight on the topic.

Regards,

Sami

[awmcc90VZTT2]

The provided API isn't actually all that slow - it's just ridiculously confusing.

Here are some ways to increase your performance by a significant margin:

First, array.GetValue is incredibly slow, use the following extension instead:

[MethodImpl(MethodImplOptions.AggressiveInlining)]
 public static T[] ToArray<T>(this Array arr)
     where T : struct
{
    T[] result = new T[arr.Length];
    Array.Copy(arr, result, result.Length);
    return result;
}

Array.Copy will provide almost native speed and it handles the indexing differences. The result array will be 0 indexed with all the vertex values.

Second, the result of selection.Paths(), and subsequently path.Fragments() yields every duplication in the entire model. So if you have multiple objects in your model with the same geometry (which is perfectly common) then every instance will return all the default fragments and a different transform. The solution is to pre-filter all the fragments and paths so that when you generate the geometry you only yield unique fragments. This can be done as follows:

public Dictionary<int[], Stack<ComApi.InwOaFragment3>> GetSortedFragments(ModelItemCollection modelItems)
{
    ComApi.InwOpState oState = ComBridge.State;
    ComApi.InwOpSelection oSel = ComBridge.ToInwOpSelection(modelItems);
    // To be most efficient you need to lookup an efficient EqualityComparer
    // for the int[] key
    var pathDict = new Dictionary<int[], Stack<ComApi.InwOaFragment3>>();
    foreach (ComApi.InwOaPath3 path in oSel.Paths())
    {
        // this yields ONLY unique fragments
        // ordered by geometry they belong to
        foreach (ComApi.InwOaFragment3 frag in path.Fragments())
        {
            int[] pathArr = ((Array)frag.path.ArrayData).ToArray<int>();
            if (!pathDict.TryGetValue(pathArr, out Stack<ComApi.InwOaFragment3> frags))
            {
                frags = new Stack<ComApi.InwOaFragment3>();
                pathDict[pathArr] = frags;
            }
            frags.Push(frag);
        }
    }
    return pathDict;
}

Then iterate over them:

foreach (var kvp in pathDict)
{
    var frags = kvp.Value;
    while (frags.Count > 0)
    {
        frag = frags.Pop();
        frag.GenerateSimplePrimitives(
            ComApi.nwEVertexProperty.eNORMAL,
            <call_back_listener>
        );
    }
}

Using these tricks will significantly speed up your export. Also, consider filtering model elements that are geometrically large but overall insignificant like dimensions, text, mtext, etc.

[alexisDVJML]

Dear @awmcc90VZTT2,

Agreed and thanks for posting these very useful details.

I actually implemented very similar tricks a few while back, sorry for not reporting these here, and the speed becomes more reasonable, on par with some plugins (I did quick benchmarks to confirm so).

1/ Use of Array method

Using things like

float[] coords = new float[3];
a.CopyTo<float>((Array)v1.coord); // v1 is a InwSimpleVertex

significantly speed up the process.

However that means 3x float[] allocations and Array copy for each triangle, not to mention:
- the calls back and forth between Navisworks and our plugin
- the difficulty of preallocating correctly sized arrays for each primitive type per fragment

2/ Fragments filtering
I will look at your implementation. On my current implementation, to determine what I call ownership, for each fragment in item.Fragments, I check that its path starts with the same as the item. It seems to do the trick but is different than your approach. One issue with this is I still need to keep a reference somewhere so I know where to look these "shared' fragments, not done yet.

3/ Actual geometry stored by Navisworks vs primitives

I'm not sure yet how Navisworks actually store geometry, but based on nw_create API, the options to use parametric primitives etc, I would assume that Navisworks internals are actually much more semantically rich than just a bunch of triangle. And if this is the case, having access to these would be of tremendous value.

So, long story short, we can "survive" with tricks like you suggested but still far from being optimal.

[Anonymous]

Hi @Anonymous,

I'm new in the forum.

I'm facing some difficulties to extract geometry dimensions from Navisworks and searching in goolgle I found this topic and I hope you could give a hand. I have no experience in programming and most of my search results leads me to perform a class library in c# using visual studio. Unfortunatelly my lack of knowledge doesn't allow me to do it myself.

I really appreciate if you could give some directions.

thanks!

[peter_hirn]

Holy crap, this API is unbelievable slow 😞

Thanks for the tips. I need to do more testing, but I think just iterating over the triangles (without doing anything) takes multiple hours on larger models. What the heck.

Edit: Yeah, as expected even with empty function bodies in the CallbackGeomListener there is only a ~10% performance improvement. 🤦

@peter_hirn - this post has been edited due to Community Rules & Etiquette violation.

[peter_hirn]

Leaving this here for further investigation (significant performance improvement possible by avoiding dotnet COM interop):

- https://adndevblog.typepad.com/aec/2018/09/get-primitive-from-solid-of-navisworks-by-native-com.html

- https://github.com/xiaodongliang/Navisworks-Geometry-Primitives