How many of you developers multihread your revit api requests?

[erikeriksson5686]

 Hi,

I've been using the revit API for some years now, but I have never tried multithreaded requests. Basically because Autodesk told me that it will never work.

I've now grown a little more confident and started testing a bit. So I figured that as long as I keep the calls to the api in a single thread, I can multithread over the results (from the filtered element collector for instance).

I've tested to cache all the rooms with parameter values using the Parallel.Foreach method and it really speed things up. From 1000 ms to 400 ms.

I figured that not all the objects that you can retrieve from the element can be obtained multithreaded, but parameters do apparentely work (in my tests at least).

get_Geometry does not work (but that was a real long shot, imho).

Has anybody else got any experience of do's and dont's here?

Im not integrating this in my production addins, Im just testing.

And also I figure that writing to the Revit document from multiple threads is also too problematic to even try.
So a the moment I just want to speed up my reads from the documents.

/Erik

[jeremytammik]

Dear Erik,

Thank you for your very valid query.

In fact, the Revit development team has never said that you cannot use multiple thread within your add-in.

The only important thing is to ensure that all calls to the Revit API are made from a valid Revit API context, and that context will always reside within the main thread, cf. The Building Coder topic group on Idling Samples for Modeless Access and Driving Revit from Outside:

http://thebuildingcoder.typepad.com/blog/about-the-author.html#5.28

Therefore, by the way, I can absolutely guarantee that the correct final answer to your question will be zero.

There were some loopholes in the past, but they were illegal and the Revit API team is working diligently to close them all.

I hope this clarifies.

Best regards,

Jeremy

[ollikat]

---

@Anonymous wrote:

In fact, the Revit development team has never said that you cannot use multiple thread within your add-in.

---

@eric wrote:

"...but I have never tried multithreaded requests"

I think Eric indeed was referring only to API request trough multiple threads. It's should be pretty obvious that otherwise you can have billions of threads in your add-in if you wish.

I understood that the question was: How many of us have been successfullu used Revit API via (worker)thread and how.

For my behalf, I think I haven't done that...at least intentionally :-). And as Jeremy stated, that would be very risky as even it might work now, it might be incompatible in the future. But in any case...I agree that it's super shame that we aren't able to do anything concurrently with Revit API.

[erikeriksson5686]

Thank you for your replies.

Jeremy, I do extensively use multiple threads to make my addins more responsive and to get Revits attention I use the tools available. Like the idling event and External event framework.

However, my meaning was more that I wanted to use multiple threads when communicating with Revit.

My initial theory was that all properties should be safe to read from any thread and the methods needs to be handled with more care.

Like I said in my first post Im able to get parameter values, despite using methods to get them. However the get_Geometry method does not seem to work.

But just getting parameter values multithreaded will speed up my addins a lot. Since most of what we do is juggle parameter values around.

[ollikat]

Hi again.

Just a quick clarification about properties & methods in .NET. Actually under the hood, there's no such a thing as calling the property. All properties are eventually also methods...either get_... or set_...

In other words...it's just a matter of designer of the API whether he decides which one (method or property) reflects better the purpose of the public member of the class. From technical point of view there's no difference what so ever which one is used and called.

So in this case the only thing that matter is that what's the code behind the property/method.

[arnostlobel]

The official statement is and always has been that accessing the Revit API from other than the main thread is not allowed and not recommended. The primary objective behind such a strong statement is the safety of the model, but also the health and robustness of applications connected to it.

I believe it is generally understood that modifying a model from multiple threads is not a good idea. However, not a lot of programmers realize that also just reading from it, even if “only” parameter values, can lead into very confusing results. The thing is that no one application can be sure that during its reading the model there isn’t some other application modifying it at the same time. Therefore, reading parameters from worker threads could yield values that are out of sync with each other even when taken from the same element (but  at two different times, even if ever so slightly apart of each other.)

I assume we could partially overcome the danger if we post an event about each transaction starting. That way programmers would know when it has stopped being safe to continue reading. However, this idea may look good on paper, but is not so great overall, because it increases the vulnerability of the model. Besides, even with this event the danger cannot be completely eliminated. There are many places in Revit that are not thread-safe and it is almost impossible to single them out. That is why some reads may always be safe (assuming a read-only state of the model), while some will never be safe, and some may be safe only occasionally.

We understand it could be a big deal for many users, but the API is simply not there. We have a lot of multithreading internally In Revit. It is how we get faster with every release even as models get larger. However, the internal multithreading happens in very controlled environment, therefore it is very safe (with respect to the model and Revit process) and robust.

There have been some experiments done with reading Revit models on separated processes (rather than threads). With the ever increasing computing power and available memory this could be actually a better approach for getting results faster than trying to bend the single-threaded API backwards to make it adequately safe.

[erikeriksson5686]

Ollikat, Im aware that one can basically wrap a method inside a property and the otherway around. However my hypothesis was that these were the best ones to start with since you should use properties for more simple "get-a-value"-operations and methods for more "calculate-something-and-return"-operations the properties would be the best bet of getting anything to work.

Arnost, yes I know, thats why I havent tried it in 5-6 years I've been working with the Revit API.

I did actually try what you are suggesting in the end last summer. I created an small API that used WCF named pipes to communicate between addins in different revit instances that opened the same model to be able to multithread both read and writes. This works (not as fast as my current tests, but hey you can write multithreaded.

However this approach has many drawbacks, debugging is tedious for instance. This might have to do with my limited experience with WCF. I guess maybe REST could be easier.

I totally agree on the writing multihreaded part. I wont even try that from the same process.

However, I think you missunderstood my initial intentions. Currently, Im testing how I can iterate over a collection multithreaded, using the Parallel.Foreach method.

To get around the problem of having a background thread reading things, one could just check if the doc is modifiable, if not, then no transactions are open and its safe to read.

However currently, Im not trying to read things in the background while writing in the main thread. I just want to speed up my reads from collections.

/Erik

[arnostlobel]

Hello Erik,

I appreciate you being cautious when experimenting with multithreading in the Revit API. It can only pay off. However, practically nothing you do (short of running separate processes) can guarantee you always a successful execution no matter what you do and how careful you are. I did understand your intention very well. You try to read properties of elements you have collected in a collection. You do it with parallel execution. It sounds like it ought to be safe, and it might, as long as you read thread-safe data, and the end user is doing nothing, and there is no other external application installed with Revit, including the ones we ship Revit with. And that is rather hard to guarantee, I am afraid.

 Your idea about using Document.IsModifiable may sound like one way to mitigate problems, but it is not. For one, the method itself is not thread-safe, but even if it were (and it would not be very difficult to make it if we wanted to), it would still not be an atomic operation, thus the result you get from calling it may not reflect the state of the property at the time the result is delivered to you. In other words: You cannot trust the value of IsModifiable if you are looking at it from other then the main thread. This is in fact similar to reading any non-thread-local value from multiple treads. Unless you put locks around it, you cannot guarantee that the read operation returns a current value.

 Again, I understand well that you are not trying to read values while modifying the model at the same time. You are just reading. However, even just reading is not safe unless you are also completely in control of writing. And this only applies generally –  it does not always apply to every reading in Revit. Let’s assume that you are in control of writing. Let’s say you are executing your external command in a read-only transaction mode, or in manual mode but without a transaction open. Then you are virtually guaranteed the document is in read-only state. However, it does not mean there will be no changes to the memory we need for the model. When a model is opened in Revit, we do not load it in all at once. We only load the parts we need to know about, which include certain information about each element, but not all data associated with every element. If you happen to read only the data from the top-level tier, you would be safe. However, if you happen to read data that require the element to be fully loaded in (we call it “expanding an element”), you may get into trouble. This operation will require memory to get allocated and I am afraid that element expansion may not be completely safe when it is triggered from multiple threads at the same time.

 I might have gone a bit too technical, but my main point was to elaborate on my previous statement, which was that One cannot be ever thread-safe in the Revit API even if only reading from the model.

[jeremytammik]

Hi guys,

Thank you very much for the interesting discussion and important words of caution.

I summarised them here for future reference:

http://thebuildingcoder.typepad.com/blog/2014/11/the-revit-api-is-never-ever-thread-safe.html

Cheers,

Jeremy

[richard.schaffranek]

Just to clerify, I have read through this thread and kind of understood the problems with revit api beeing thread save, however:

I am developing a plugin that havily uses some functions from the revit api which are all static:

CreateExtrusionGeometry
ExecuteBooleanOperation
CreateLoftGeometry

So I thought, hey static function what could go wrong?

And indeed I get a preformance boost of factor 2-4, which given the fact that I am using 6 instead of 1 thread is not the world but given computation times is worth the efford. However sometimes I had an odd behaviour that in certain cases (which actually should be better for parallel processing) the parallel process actually took longer then the single thread.

So I went on debugging and for some reason, which I hope I get an answer too ( @jeremytammik @arnostlobel @ollikat @erikeriksson5686 ), the methode GeometryCreationUtilities.CreateExtrusionGeometry runs a multitude longer when running within a parallel process. difference is 1 milisec  (single thread) 400 milisec (multi thread) when it successfuly creates an extrution and 100 milisec vs. 9000 milisec when it throughts an exception.

this leaves me behind guessing why.....

[jeremytammik]

Afaik, nobody has been able to achieve any significant useful results calling Revit API functions from several threads simultaneously.

Very interesting and very impressive to hear that you can achieve anything at all successfully that way.

Would you like to share a sample showing what you are doing and how?

I am sure that would be interesting for others as well, and maybe someone can say more then.

Sooner or later you may very well corrupt your model.

I recommend not trying this in any model into which people have invest a significant amount of manual effort.

[richard.schaffranek]

Dear ( @jeremytammik @arnostlobel @ollikat @erikeriksson5686 )

I have created a test Project in visual studio 2017 which should run with Revit 2019 including a sample file. Download here: weTransfer

Sorry smallest example I could think of.

As the test project is rather small I used  Ticks to measure the preformance but if you have a larger project miliseconds will also show the issue.

So the multithread was in total 0.03 times faster than the single thread.

However the Operation GeometryCreationUtilities.CreateExtrusionGeometry took almost 44% longer when running in multithread then when running in a single thread.... this is what makes it so strange too me..

And this is even worse when an exception is throughn.

[Kennan.Chen]

Some of my personal understanding about this thread and multi threading in Revit API.

Thread safety problems are always uncertain. It depends on how much memory is shared by how many threads. If there is no error encountered when using multi thread to read data, it’s half percent of luck and another half percent of task splitting. If each thread is handling a relatively independent task and the task is as simple as getting a parameter, it is a great chance to lead to an impression of reading data is thread safe.

Since Revit API is mostly a wrapper of native code of Revit using C++ CLI, thread safety is much more depending on the design of Revit native code, specifically whether native code manages application states in a thread safe way. An managed operation appears to be reading data, but internally the native code actually write some states like logs and time stamps or other business states.This is quite similar to REST API development. A GET operation is reading data on client side but on server side it can result in some database writing.

Since the native code is a black box for us third party developers, it’s quite risky to try any kind of multi thread operations until Revit development team stands out to announce it’s safe.

[richard.schaffranek]

Jap this is basically the sum up of this thread, thx. However question was: 

why is a static method slower when executed within a parallel.foreach „loop“ ? 

and I am Talking something like 400 times slower on big projects especially when it fails.

[jeremytammik]

I can just repeat what Arnost says above, and the Revit development team have always consistently repeated:

One cannot ever be thread-safe in the Revit API, even if only reading from the model.

I am pretty sure that all the API calls are actually executed in one single thread, and that the effects you observe have other causes.

[Kennan.Chen]

I think this article helps.

https://docs.microsoft.com/en-us/dotnet/standard/parallel-programming/potential-pitfalls-in-data-and...

[Kennan.Chen]

It was so amazing that I finally came to realize the truth I was replying to a thread posted 5 year ago.😂

Last word. Feel free to multi-thread your Revit API calls locally but be really cautious when applying it to production because we just appears to be controlling Revit by Revit API calls.

[richard.schaffranek]

@Kennan.Chen 
Have read through the article, not the first time but obviously I always missed: 

"Most static methods in the .NET Framework are thread-safe and can be called from multiple threads concurrently. However, even in these cases, the synchronization involved can lead to significant slowdown in the query."

So I have tested the behaviour against writeline and yes there seams to be an overhead on the writeline command as it executes a bit slower when multithreadining (5 ticks single 8 ticks multi). This seams to be similar with the static call of GeometryCreationUtilities.CreateExtrusionGeometry.
When throwing an exception the execution times with multithread are a multidue worse then with single thread (436 millisec vs. 18856 millisec.; 321 millisec vs. 36007 millisec.) 

However WriteLine actually writes something to somewhere, so there would be an explaintion but given that GeometryCreationUtilities.CreateExtrusionGeometry returns a Solid one could guess that this is done without any interaction unlike WriteLine...

Why I am so insiting on this:

I think that this is a realy important topic when it comes to the usability of automation tools. If one has to wait 1 minute for a result or 15 seconds befor beeing able to controll the result, it makes a big difference in the work process and increases the acceptants of such tools. This still holds if the none automated process would take more than 10 minutes.

Solong, I give up unless Autodesk is willing to provide some inside on the specific functions of:

GeometryCreationUtilities
BooleanOperationsUtils

@jeremytammik  @arnostlobel ?

[ollikat]

Hi

Unfortunately I don't have any new thoughts for the topic. But I just wanted to remind you that AFAIK Arnost is not working for the Autodesk anymore. Of course, it would be delightful to know someone in similar position, who is capable of giving us such deep insights as Arnost did.

[richard.schaffranek]

So we have investigated a bit further and we have two guesses:

a) it seams that the function (CreateExtrusionGeometry ) does a lot of memory allocation copying a lot of information. 

and

b) maybe there is a problem between multithreading of the C# and the multithreading / memory locks within the dll.

We belive that it is a combination of both. However any further insite on this topic would be of intrest....