Understanding the use of the UniqueId

[Anonymous]

Hi all,

in the early days I thought that ElementId was a stable way of finding elements in a document, then I learned that UniqueId was the proper way.
But I havent been able to find enough coherent information about it to really understand it and I've been reading lately that we should use the uniqueId as a identifier in multifile projects (multiple centralfiles), because its only guaranteed to be unique in one centralfile.

This is what I've learned so far (From Jeremy Tammiks Blog among others):
The UniqueId 40 characters long and is made up from 2 parts, the first is internally called the EpisodeId and the second is the hexadecimal representation of the elementId. I've read that people complain about the fact that two revit files from the same template will have the same UniqueIds for the elements that are created in the template. So I guess that the EpisodeId cannot be used as a "file guid".

However, its stated by Arnost Lobel that the UniqueIds purpose (one of many?) is to have a stable way of identifying elements in a workshared environment, because the elementId of an element can change.

OK, then, but if the last part of the uniqueid is the elementid in hexadecimal form, wont that change the uniqueid also?

I've also found in a forum post that first part is connected to the WorksetId.

With all this in mind, is the best way of identifying elemtents in a multifile  to create a guid for each element and store it to them using a locked and hidden parameter (or extensible storage)?

Thanks for any help figuring this out!

/Erik

[arnostlobel]

Sigh.. 😕 I truly believe we have made a disservice  to our Revit users when we named UniqueId the way we did, for it seems to keep on being misunderstood in spite of many of us trying to shed more light into the problems with identifying model elements. I'll try again to straighten up some of the confusion, and my  hope is that I'll make it better, not worse.

Erik, I think you may actually be on the point in your very first sentence:

ElementId was a stable way of finding elements in a document, then I learned that UniqueId was the proper way.

The most important part of the sentence is the  "a" in "a document", since the purpose of both kinds of IDs is to identify elements in a single model, not in a bunch of models. In other words: none of the IDs can uniquely identify elements across unrelated documents in spite of the name it was given to the UniqueID class. Universal uniqueness of identifiers is simply something that cannot be done. If one takes an existing file and makes a copy of it, no one would probably expect elements change their Ids in that copied file.

So, with that "universal uniqueness" hopefully out of our way, we have only two aspect to discuss:

Regular Ids (integers) can identify elements in a model uniquely as long as the model is not work-shared. It is because Revit does not change the Ids after they were issued, and it also always increments the Ids before assigning them to new elements. That means Ids are never reused. If an element is deleted, its Id will not be used for anything else.

In a work-shared environment, regular Ids cannot be stable, understandably. I'll demonstrate it in an example:

Consider two users, UA, and UB. One creates an empty central file. Then both users create their respective local files from it. Now, say UA creates a wall. It will be assigned Id = 1005 (for example). If UB also creates one wall it will also be given Id = 1005, since both users started off with the same file and Revit increments element Ids linearly. Let's continue: UA saves to central, thus her wall gets there first with its original Id (1005). If UB then syncs, first the wall from the other user will be brought in with it's original Id. Revit will notice that the same Id already exist locally. Since all elements must have unique Ids within a file, something's got to give; Revit takes the UB's wall and assigns it a new Id, say 1006 and uploads that wall to the central file. After UA gets the latest too, both users and the central file as well will have two walls with Ids 1005 and 1006, respectively.

I hope that makes sense. Let's go on with the second one.

Unique Ids (strings) can identify elements uniquely even in a work-shared file. What this basically means is that, unlike regular Ids which may collide when two local versions are merged together and thus one (or more) of the Ids must be re-issued (as I illustrated in the aforementioned example), Unique Ids do not collide, therefore they do not have to be reissued; they merge with no problem.

What that means in practice I'll again try to demonstrate on an example:

An external application creates a new Beam element in a local file. It takes and stores its unique Id - say is "ABCD-1234" (a string!!). Then the user goes on making more changes while his co-workers make even more changes in their local files. At some point they all sync with the central file. If, after that point in time the external application operating on the first user's computer asks for an element to be fetched by providing its UniqueId =  "ABCD-1234", the application will obtain the very same Beam element, providing it still exists.

I think this should be the end of the story. Please note that I used the sample "ABCD-1234" value of an unique Ids on purpose, because I believe it is for the best if users do not try to understand what the parts of an unique Id are and how can one try to decipher it. Please keep in mind that it was not meant to be deciphered. No matter what Revit does internally with a unique Id, it has one and only purpose to Revit programmers (both internal and external, in fact) - it identifies an element in a work-shared model. Once you have an element's unique Id, you can use it to locate the element within the model.

One more note before I stop typing, mainly to address Erik's last question. Neither regular element Ids nor Unique Ids can safely identify elements across multiple unrelated documents. If there is someone who requires such a universal identification system for organizing elements in whatever models, that person/company would have to create some kind of a controlled database. I presume it would not be particularly an easy task.

Post Scriptum: A post too long? That's what holidays are for! 🙂

[Anonymous]

Excellent info, thanks.

I think I'll go and re-code my GetElementIdFromUniqueId() method now so that it actually performs the lookup instead of extracting it from the string to save time. I wasn't aware of the work-sharing issue and this will save me some headaches in the future I'm sure.

[Anonymous]

Arnost,

Thank you for your reply! Actually, I tested it myself, I was too curious to want to wait for a reply. 

So for anyone curious about this: the first part is related to the workset that the element was created in and the second is the hexadecimal representation of the elementid. However the worksharing part is not the workset uniqueId (even though its a guid and not a string).

However, I've not yet tried to force the elementid to change on synchronization, like you talk about. So I dont understand how they cannot collide on merge.

I understand that you have your reasons for not disclosing how the uniqueid is created, Im still curious about it 😃

And with the level of "experimenting" going on with it I Think you should disclose as much as possible to help developers from doing stuff that you know cannot be done.

Regarding the copying of a file, I fully understand that the uniqueids would be the same, and thats the why we want them to be 😃

But because I dont have the information that you do about the inner workings of the uniqueid I'll just have to take your word for it, when you say that ira not safe to use in multifile environments.

However multifile Projects are something that we "in-the-trenches" have to face all the time. Id say more than half of our projects have more than one file and because of this there are alot of companies that develop these of-site databases and those who I've talked to use the uniqueid to identify elements. I hope that there are not 100s of projects that are going to have unexplainable behaviour because of this limitation.

/Erik

[arnostlobel · Accepted solution]

Erik wrote:

"And with the level of "experimenting" going on with it I Think you should disclose as much as possible to help developers from doing stuff that you know cannot be done."



I can both concur and oppose the statement. If we can agree that what I have already disclosed so far about the nature of unique Ids is in fact as much as is actually possible, than I second your notion. I must, however, decline to go even further below the surface. I beg your forgiveness and rely on your understanding, for I must insist that in some cases less is more. By my revealing more internal details about the structure of unique Ids I could put a serious road block on our future development and compromise our freedom to innovate (re-factor). I often dig deep bellow the lid of the Revit API (more often than others do, I believe) , but in this particular case I truly believe that what I have stated is what users need and should know.



That being said, perhaps I can make one more step forward by posting the following statement:



"If and only if your environment guarantees that all files in that environments are genuinely unique, then and only then you can also expect that all elements in all your Revit models will have their unique Ids truly unique (in that domain). This is correct for versions up to 2015, and 2016 will be the same." I do not expect us changing the policy in the foreseeable future, but we should be free to do so, should we find a very good reason for it.



I believe, however, that such an environment is very hard to create and maintain. I would even expect that making sure that all model documents are unique (in a controlled domain) may hard but possible to accomplish (please do not forget about templates - you always need to start from scratch), but unless you also create all families by yourself, it may be as hard as practically impossible to guarantee that all files that ever get included into the domain have been created uniquely (i.e. never by copying).



Erik wrote:

"But because I don't have the information that you do about the inner workings of the uniqueid I'll just have to take your word for it, when you say that [it is] not safe to use in multi-file environments."



I am not sure what the concern here is; I can only repeat what I've said: If you can guarantee that none of your files is created as a copy or from the same template, then (and only then) all your elements across all your documents will have their UniqueIds unique. It is because UniqueIds do (still as of today) in fact contain GUIDs , which makes them genuinely unique. However, since Revit is not an application that itself requires uniqueness of documents (neither models, nor templates, families, etc.), we cannot make that generic statement claiming that UniqueIds will universally identify a model element no matter where and how the element originated from. I believe that is a fair statement.

[Anonymous]

Arnost,

I appriciate your frankness about the API and I do understand that you need freedom to change the the way uniqueId works.

I now have one of the answers I was looking for: I can use the uniqueId in multifile environments if I make sure that there are no duplicate revit documents.

Thanks Arnost!

/Erik

[arnostlobel]

I am glad your curiosity has been satisfied, Erik.

However, reading your confirmation I wonder if I expressed my statements as clearly as I intended to. In order to clear up any possible misunderstanding I think I better restate one important detail again. 

Erik, you do realize that two documents that start off the same template files are in fact duplicates, to a degree, which means that such documents are very likely to contain elements with identical unique Ids? That applies to both project and family documents as well. And since our content team most likely created Revit standard library families with just a few templates, it would in effect mean that not only you could not start any of your document with a template, you also should not use any of the standard families installed with Revit (and most likely you should avoid families from any third part as well.) That is, if you want to avoid duplications of documents in order to have the guarantee of truly unique elements in the scope your company domain. 

Thank you

[Anonymous]

My curiosity can never be satisfied, thankfully 😃

Yes, if you copy a Project document or family document the uniqueids inside the 2 copies will be the same.
This is what I expected and thats perfectly fine.
Im concerned with finding instances and types in mature models and In that case this will do.

Thanks again.

[mastjaso]

I know this thread is a few years old but I wanted to chime and somewhat echo what Erik has said. 

I fully understand the inherent difficulties with keeping UniqueIds when you're faced with the prospect of users just copying .rvt or .rfa files. But as Erik said, virtually every single project done in Revit is a multi-model project. In my time using Revit at an architecture / engineering firm I have never once seen a revit model that was just a single model unless it never made it past early prototype stage. As soon as engineering is involved there will be multiple models and on any reasonably sized project there will be multiple models. 

Having no way of uniquely identifying objects across models is a nightmare and frankly goes against the whole purpose of Revit which is to implement BIM. 

With BIM modelling we are always striving to create a building as accurately as possibly to create linked and parameterized information that represents how a building actually works. As such it seems fundamentally necessary to be able to uniquely identify every element that is going into a single building or project. 

Therefore if Autodesk has chosen a model / paradigm of linked models that does not allow you to uniquely identify objects within a single building or project that is an objective failing of Autodesk in their design of BIM software since it fails at a fundamental component. 

It seems to me that if Autodesk really can't uniquely identify just normal linked files then they need to add another layer of wrapping so that you can specify that this file, this file, and this file are part of a single project and it will refactor those databases to ensure unique identifiers. In Visual Studio terms they need an equivalent of a solution file to their csproj / rvt files. Or like an ethernet switch they need to add another layer of addressing. 

Again, I'm sympathetic to the difficulties involved, but this is BIM software and uniquely identifying what objects go into a building is an extremely fundamental component of BIM. 

[jeremytammik]

Dear Mastjaso,

I don't see any huge problem.

One way to handle this effectively is to implement a model checking algorithm that ensures that all elements of interest to you do indeed have unique ids.

Run this automatically and frequently.

If a situation occurs in which the requirement is violated, notify the user (or automate the correction) to create a new element to replace the duplicated one.

You need model checking anyway, because users can do any number of crazy things.

You need assertions and unit tests anyway, because otherwise your software is not to be trusted, and you need to add to the tests all the time as new issues are discovered and resolved.

I see no big deal here for add-in developers.

Still, if you do, please raise a wish list item for this requirement in the Revit Idea Station and ensure that it gets a lot of votes.

Thank you!

Cheers,

Jeremy

[mastjaso]

Hi Jeremy, thank you for the response, I have implemented this, and you are correct that it is not a huge deal, my response was a little more dramatic than intended or deserved.

However, I do still somewhat stand by my assertion, though I think it actually speaks to a bigger BIM capability gap that goes beyond Revit.

The more I see the more it seems to me like there's a fundamental piece of software missing that would allow robust implementation of BIM concepts and imho it would make the most sense if it were developed by Autodesk given how ingrained they are already. Fundamentally a Building Information Model should store all the information related to a single building or a site. Revit accomplishes a huge chunk of this by containing much of the 3D information related to the building. However it falls short. It doesn't contain things like Specifications and as we've discussed in this thread it doesn't really uniquely identify all the objects within it. So a Revit model is a large subset of BIM information without really being a complete BIM model.

Having to check for and ensure unique identifiers in linked models isn't the end of the world but does seem like something that shouldn't be necessary in a holistic and complete BIM model. It's essentially asking each add-in developer to partially create a true BIM wrapper for every project that has linked models (which is all of them). Again, I know this isn't that huge of a deal from a practical standpoint, but it does seem like a symptom of the BIM software capability gap that is a much bigger deal.

And again, thank you for your active involvement on these boards and I apologize if I came off a little testy 🙂

Cheers,

Jason

[jeremytammik]

Dear Jason, 

Thank you for your input and appreciation.

I passed it on to the development team and look forward to seeing what they respond.

Cheers,

Jeremy

[mastjaso]

Hi Jeremy, I thought I'd just follow up on this as I was reminded this thread existed and am back dealing with linked models again.

I've been developing BIM software that integrates with Revit and other platforms for a little over a year now and it has only reinforced my previous view of uniqueIds and linked models. 

I genuinely think that "linked models" is a fundamentally wrong way for Autodesk to attack the performance / worksharing issues of large projects without some kind of project wrapper that unifies and manages them. The whole point of Revit and BIM is to store all of the information about a building in a structured and queryable database, so that you have a single source of truth that can be referred to. It's how Revit was originally designed, and how a single .rvt file operates. But the whole idea of linked models without any kind of project wrapper kind of completely ruins this. In addition to the problem of not being able to confidently uniquely identify things, it also introduces the necessity of having multiple copies of something fundamental like Levels, and grid line instances, of which there should only ever be one copy that comes from architectural or structural and everyone builds around, but instead exist as duplicated copies in every file and may or may not be synchronized. It means that there is no way of connecting your electrical system to the mechanical equipment that it powers. It means that you have multiple copies of families that are supposed to be representing the exact same piece of equipment because you need one copy in your west wing model and one in your east wing model and there's no guarantee that they'll be the same or in sync.

In short, linked models bring us back closer to the nightmare days of autocad where instead of an organized and queryable database, you had a bunch of disparate, barely connected files that were maybe linked together sometimes.

So I did want to press this because I am still curious, does Autodesk have any plan to address this? Is creating a single project database for a building on the roadmap? Or is Autodesk's focus elsewhere and linked models are here for the foreseeable future? Because I genuinely think that if Revit can't be the single source of truth for a project, some other platform will be.

[jeremytammik]

Dear Jason,

Thank you for your update and important critical observations.

Unfortunately, I am not the right person to discuss this with at all.

I have neither experience working in this area nor any in-depth knowledge of the design and intent.

The requirements you list are obvious and inevitable.

However, I am neither aware of them nor any plans to handle them differently in future than today.

In my imagination, I firmly assume that Revit and BIM experts have developed standard best practices to address them.

I would also claim that the issues you mention have next to nothing to do with the API, and are all about high level upfront project setup and product usage.

You should therefore research the standard best practices in these areas and discuss these issues with application engineers, product usage experts, and product support. Solutions are probably known and available.

Once you have those aspects clearly sorted out, it is time to step up into the programming and examine how to convert the usage best practice into an optimal programming solution.

I hope this clarifies.

Cheers,

Jeremy

[richard.schaffranek]

I am sorry I can't be so polite, I find the "solution" suggested by @jeremytammik out of the question. It sounds a bit like: "Oh I forgot the door into the room, so why don't you use a ladder and clime throuhg the window to go to bed." 

As @mastjaso an ID is often used to link things within BIM processes accross multible softwares and use cases.  I think it is a general design flow to use an id to encode things and id should be an id and it should be possible to generate a real unique ID with ease.

Please fix this, even if it is by adding a filed "realUniquID"

[jeremy_tammik]

Dear Richard,

Thank you for your update and no worry about being polite or not.

As said, I am not an expert in product usage and best practices and am happy to stand corrected and having your needs spelled out to me in full explicit detail.

Since the discussion above is quite long and touches various issues, can you please clarify your exact requirements?

As you put it above, it sounds to me as if the Revit UniqueId should fit your needs.

If not, can you please explain why?

Thank you!

Best regards,

Jeremy

[richard.schaffranek]

No not my specific reasons of uniqueness but its name.

As explained before the uniqueID is unique in workshared files. so if I am working on four building parts in linked files the uniqueness of the unique id can't be guranteed. I had a case where in a follow up process I had 190  collisions of objects sharing the same "uniqueID" sometimes 4 objects haveing the same "uniqueID" this makes identifying elements in later process a bit complicated. 

So all I am aksing is that the unique id is unique. This is complicated if you are using it as an identifier that also encodes information but would be possible if using e.g. GUID.newGUID() this would generate a real uniqueID. Or if you want to encode something in the id than construct your stirng and hash it (e.g. SHA256).

[jeremy_tammik]

Yes, this is indeed an issue.

I would suggest raising a wish list item for an improved unique id in the Revit Idea Station.

Whenever you require new or enhanced functionality, the Revit Idea Station is the place to go.

Please search there for a corresponding wish list entry for the suggested functionality and add your comments to it, or create a new one, if none already exists:

https://forums.autodesk.com/t5/revit-ideas/idb-p/302

Tag it as an API wish:

https://forums.autodesk.com/t5/revit-ideas/idb-p/302/tab/most-recent/label-name/api

Ensure it gets as many votes as possible to underline its importance to you and the rest of the developer community.

The Revit Idea Station is currently one of the main driving input forces for Revit API enhancements.

The Revit development team look there. Your comment here in the discussion forum might be overlooked.

Thank you!

[RPTHOMAS108]

I don't think I would be using UniqueId for external storage.

In the distant past I saw it used in COBie spreadsheets. Always thought that was amusing watching humans sort through those. Especially considering most of the characters in the UID had a worksets relationship origin and so were exactly the same prefix.

It would be ideal if perhaps when an element is copied the copy is assigned a new GUID. Problem is when we accidently delete such an element and can never again use that UID for that thing we know as humans is the same. The history of it is lost.

To address the uniqueness across models is really trivial:

ModelUID + ElementUID = TotalUID.

How can Revit know a model is the same model however? It could be the same model at a different file path but still the same. If the user was allowed to assign the model id prefix used in extracted ElementUID then that would likely solve that.

Shouldn't be using GUIDs though; if we use all the characters of the alphabet along with the numbers 0-9 then we can radically reduce the UID length to something a human can distinguish. I think IFC now does similar in that respect.

In reality most organised organisations probably have their own systems to apply UIDs via the API and manage the 'what if' questions that come about:

'What if we want to reinstate something we consider to be the same element'

'What if we move an element across the site is it really still the same element'...

[richard.schaffranek]

How does unique id and element id relate...

based on this post it looks like that the only difference between the two elements is the element id
https://thebuildingcoder.typepad.com/blog/2009/02/uniqueid-dwf-and-ifc-guid.html

and how does that go together with bim360 synchronisation
https://forum.dynamobim.com/t/why-element-id-changed-after-synced/65806/4

does that mean that it can be that also the unique id of an element changes when syncronising with bim360?

So how can we than identify an object over multible versions of a file synchronised in BIM360?