attributes.add takes longer and longer to process in the Python API

[chris.paulsonB73L8]

I've noticed some odd behavior when working with attributes in Python. I have a loop over all elements in root.allOccurances. In my test case, I have around 100 occurrences. Inside the loop, I add a fixed JSON string to every element. The first attribute takes just two milliseconds to add, while the final attribute takes almost 300 milliseconds to add. There's a linear trend in the processing time ( with some noise )  as attributes are added to components, which becomes slower and slower. 

Any thoughts on what might cause this type of performance decay? This makes the plugin virtually worthless, as it can take literally hours to work through a large model. 

[BrianEkins]

I have to ask why you want to add an attribute to every element. I don't know the reason it's taking longer with each attribute but I do know they weren't intended for this kind of use.

[chris.paulsonB73L8]

The purpose is to tie a few bits of data to CAD objects. These data provide tags that link specific CAD objects to downstream processing steps. There isn't much indication of intent in the documentation. If attributes aren't designed for this type of activity, what is the best practice for including data at a component occurrence level? Adding user-specific tags to model data seems like it might be a very common use case. The existence of Properties++ and Patrick Rainsberry's FusionTagger seem to be evidence that I'm not the only one interested in this capability. 

Also, just for fun, I posted a quick plot showing the trend of adding attributes. 

[BrianEkins]

I may have read more into what you're doing than you're actually doing.  In your original statement, you said, "I add a fixed JSON string to every element".  The "every element" part is open to a lot of interpretation and I assumed it meant you were adding it to most of the things in a design, which can mean every face, edge, vertex, sketch geometry, constraints, joints, etc.  What exactly is it that you're doing?

[chris.paulsonB73L8]

Sorry for the confusion. This is what I'm trying to accomplish: 

for entry in list (rootcomp.allOccurrences):

            entry.attributes.add("test_Group","test_name", "some test data")

I thought this might be an issue with the Python API, but a similar behavior is present in a c++ implementation.  

[chris.paulsonB73L8]

Here's a complete code example where I see this behavior: 

#include <Core/Application/Application.h>
#include <Core/Application/Documents.h>
#include <Core/Application/Document.h>
#include <Core/Application/Product.h>
#include <Core/UserInterface/UserInterface.h>
#include <Core/UserInterface/Palettes.h>
#include <Core/UserInterface/TextCommandPalette.h>
#include <Core/Application/Attributes.h>
#include <Core/Application/Attribute.h>
#include <Fusion/Fusion/Design.h>
#include <Fusion/Components/Occurrence.h>
#include <Fusion/Components/Occurrences.h>
#include <Fusion/Components/Component.h>

#include <time.h>


using namespace adsk::core;
using namespace adsk::fusion;

Ptr<UserInterface> ui;
clock_t t;

extern "C" XI_EXPORT bool run(const char* context)
{
    Ptr<Application> app = Application::get();
    if (!app)
        return false;

    ui = app->userInterface();
    if (!ui)
        return false;

    Ptr<Design> design = app->activeProduct();
    if (!design)
        return false;

    // Get the root component of the active design
    Ptr<Component> rootComp = design->rootComponent();
    if (!rootComp)
        return false;

    // Get the occurrences from the root component.
    Ptr<Occurrences> occurrences = rootComp->occurrences();
    if (!occurrences)
        return false;
    
    Ptr<TextCommandPalette> textPalette = ui->palettes()->itemById("TextCommands");
    if (!textPalette->isVisible())
        textPalette->isVisible(true);

    std::vector<Ptr<Occurrence>> occs;
    for (int i = 0; i < occurrences->count(); ++i)
    {
        if (Ptr<Occurrence> occ = occurrences->item(i))
            occs.push_back(occ);
    }

    // Iterate through the occurrences and time adding attribute.
    for (Ptr<Occurrence> occ: occs)
    {
        t = clock();
        auto atts = occ->attributes();
        atts->add("Test_Group", "Test_Name", "Test Data");
        t = clock() - t;
        textPalette->writeText( std::to_string( ((float)t)/CLOCKS_PER_SEC ));
    }

    return true;
}


#ifdef XI_WIN

#include <windows.h>

BOOL APIENTRY DllMain(HMODULE hmodule, DWORD reason, LPVOID reserved)
{
    switch (reason)
    {
    case DLL_PROCESS_ATTACH:
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}

#endif 

[KrisKaplan]

Yes. It is understood that the performance of Attribute creation is correlated to the number of attributes in the same owning context (for Occurrences, that would be the source component for each).

I would suggest avoiding any architecture that calls for 'add an attribute to every ...'. You could possibly hold a table of data (blob) in an attribute on a common source component. Or you could hold this data externally indexed by entity tokens (but an implementation that is robust across design mutations would likely encounter the same performance complications).

Kris

[chris.paulsonB73L8]

@KrisKaplaneach occurrence will only have a single attribute, not many. In my use case, I have files with a few thousand unique component occurrences, and I'd like to add an attribute to around 10% of the component occurrences. Doing this today sometimes takes hours, and based on computer fan speeds, the computer is doing a bunch of work. That seems odd when I'm simply trying to add a string that's a few bytes long to a file. Can you share a bit around what's going on behind the scenes when I'm making changes to that data structure?

It would be great if these data could persist in the f3d file itself, just because you've done all the hard work around file synchronization across users and version control already. I'm not sure where to go from here. Is your recommendation to simply give up on the Fusion API and build an entire external tool for tracking occurrence specific metadata?

[KrisKaplan · Accepted solution]

The issue isn't related to the number of attributes on a single entity. It is related to the number of certain kinds of entities that will have attributes in a common context.

The details of why can get a bit involved, but I'll try... For the majority of simple entities (like Features, Sketches, etc...), attributes are attached directly to that owning entity, and performance should be in constant time and should not be an issue. But some more complex types there isn't a simple entity to hang these attributes on. These are called 'tracked' entities. Tracked entities require a more complicated referencing mechanism. The most common examples are references across a path of entities (such as an Occurrence or any assembly 'proxy' entity) , or a sub-entity (like BRep entities). An occurrence 'path' can be significantly altered in assembly restructure operations but still retain its 'identity'. Sub entities (particularly BRep entities) can be split, merged, etc... across updates but still have an identity to the user.

An example of the issue that has to be dealt with would be to consider entity tokens for these 'tracked entity' types. If you generate an entity token for a BRepFace (token1) and then add features that modify the face (split it, drill a hole in it, etc...) and generate a second token (token2): You will notice that token1 and token2 will not be identical. They have a different 'recipe' to resolve to the face to account for the changes at the different points in their change history. So you cannot just compare token1 to token2 to see if they would 'match their recipe' to the same face. You actually have to bind the token (run the recipe and generate the matches in the current state) and compare the matched results.

So in the case of Attributes on these tracked entity types (e.g. Occurrence), in order to be robust across model changes, resolving to the attributes set will involve an O(n) search for a matching tracked entity recipe of the same type and on the same 'source' entity. This isn't an issue issue as long as their is a sparse number of attribute sets created for the same tracked entity type on the same context (on the same source component). The worst case is adding attributes on a large number of tracked entities of the same type (e.g. Occurrence) in the same context (e.g. the same Occurrence.sourceComponent). And this sounds like what you are running into.

It's hard to recommend alternatives without a lot more understanding about what you need to achieve. But one option I mentioned would be to add a single Attribute on the source component with the application state you are trying to save. In some cases it might also be an option to invert things and add the Attributes to all of the target components instead.

Kris

[chris.paulsonB73L8]

Thank you, @KrisKaplan for the detailed description of the inner workings of Fusion. You are correct about the problem I was encountering. As a quick test, I changed this code: 

auto atts = occ->attributes();
atts->add("Test_Group", "Test_Name", "Test Data");

to this:

auto atts = occ->component()->attributes();
atts->add("Test_Group", "Test_Name", "Test Data");

 
With this code modification, attribute add times were consistent and very fast ( well under a millisecond). I'll need to tweak my approach a bit, but I can make it work. Thanks again for providing that context.