Performing long running background computations using Idling event

[cdiggins]

We have started experimenting with performing long running computations in the background of Revit by using the Idling event, and breaking the work up into a queue of many small tasks. We have even built a small open-source library around it: https://github.com/ara3d/revit-background-processor. 

Early experiments seem promising, but the real test will be when it is used by actual users in various tools and plug-ins. I can't tell yet how reliable the technique is when Revit has a lot of work to do, and whether it could cause a bad user experience. 

We would like to hear from other people who have used this technique, or similar techniques, in the past to extract data from Revit, without blocking the UI. Specifically, do you know if this technique works well in practice? What other techniques could be used? What are the pros and cons? 
  

[ricaun]

If your code is running inside the Idling event or IExternalEventHandler that code is running in the Revit API Context and gonna block the Revit UI.

If the code is fast, the user will never notice, and if the code takes a long time to finish, it will make Revit laggy.

[jeremy_tammik]

Hi Christopher,

congratulations, and thank you for sharing this, your GitHub Revit Background Processor library, the post to the MEP discussion forum and the heads-up to me. My first impulse is to point out that I did quite a lot of work implementing similar functionality for my RoomEditorApp:

• https://thebuildingcoder.typepad.com/blog/about-the-author.html#5.28

Then, in 2013, I hit certain limitations of the Idling event, especially controlling the timing and frequency of interaction with Revit, affecting how strongly the external code affected and blocked the Revit UI. Therefore, I switched my code from Idling to an external event:

• https://thebuildingcoder.typepad.com/blog/2013/12/replacing-an-idling-event-handler-by-an-external-e...

Maybe things have changed nowadays. I would suggest that you explore that options yourself as well. Unless everything s running so impeccably and smoothly that no improvement is required, of course. Never change a running system!

[jeremy_tammik]

Shared on the blog as well:

• https://thebuildingcoder.typepad.com/blog/2024/08/api-context-background-process-postcommand.html#3

I hope lots of feedback is coming in.

[cdiggins]

As a follow-up to this post, we improved the code to fix the problem where Idle events would stop being triggered when the mouse wasn't moving: https://www.linkedin.com/posts/cdiggins_bowerbird-is-a-free-and-open-tool-from-activity-724978130938.... We used the solution proposed by Bogdan Zavu here: https://forums.autodesk.com/t5/revit-api-forum/how-to-trigger-onidle-event-or-execution-of-an-extern.... 

[ricaun]

---

@cdiggins  escreveu:

we improved the code to fix the problem where Idle events would stop being triggered when the mouse wasn't moving:

---

Never heard about this issue, are you sure that happen in modern Revit? (Revit 2021+)

I have been using ExternalEvent for a long time and never need to poke Revit. Even with Revit minimize the ExternalEvent runs normally in the background.

[cdiggins]

The problem here is not related to ExternalEvent. This fires fine.

What we are working on is the problem of executing long-running work in Revit without blocking the UI. We do this by breaking work up into small chunks, putting it on a queue, and processing a small amount of work every time an OnIdle task fires. 

The problem we solved is that the OnIdle event would not fire on regular intervals when Revit doesn't have the focus or the user stops moving the mouse. So what we do is send is regularly send Revit a "null" message on a background thread, that causes it to wake up and send out an OnIdle event at regular intervals.      

[ricaun]

What is the OnIdle event, you mean the Revit Idling event?

I know that Idling trigger slower when Revit is out of focus, but still trigger. And you are using the SetRaiseWithoutDelay that should remove the delay and make trigger really fast.

Pretty sure you can remove PokeRevit and the application gonna work the same.

[jeremy_tammik]

Please note that for constant processing tasks such as you describe, the Idling event has no advantage whatsoever over an external event, and a couple of disadvantages, such as less control. So, as I believe I already suggested above, I recommend scrapping the Idling event, switching to an external event with a Windows timer driving it, and thus avoiding the problem you describe completely:

• Replacing an Idling Event Handler by an External Event

[cdiggins]

Yes I meant `Idling`.

No the app doesn't behave the same.

>> I know that Idling trigger slower when Revit is out of focus, but still trigger

And then it stops. 

>> And you are using the SetRaiseWithoutDelay that should remove the delay and make trigger really fast.

The problem with this approach is that the application becomes unresponsive, as it gets too many messages. The goal was to strike a balance between doing work and keeping the UI responsive. 

>> Pretty sure you can remove PokeRevit and the application gonna work the same.

No it doesn't, you can try it for yourself using the Bowerbird scripting solution. What happens is that you get a few messages and then it stops sending Idle events. So work inside of the queue, remains unprocessed. 

[cdiggins]

>> So, as I believe I already suggested above, I recommend scrapping the Idling event, switching to an external event with a Windows timer driving it, and thus avoiding the problem you describe completely:

I carefully read your article Jeremy, and the problem you described at the end as follows is the one that I tackled:

>>> ... when the external event has been raised, Revit does not actually call the external event Execute method until a cursor movement or some other system event wakes it up.

It was similar for me when using Idling event. In hindsight it is true that I could replace using the "Idling" event from the code probably, but it just had the same problem that you described. The "PokeRevit" algorithm triggers an OnIdle, perhaps it will also trigger an "ExternalEvent"?  

Anyway, it works now, and I have limited time to experiment further.  

[jeremy_tammik]

>> Anyway, it works now

Brilliant! Congratulations. So, what is your final solution? Idling event without SetRaiseWithoutDelay + Poke? Is the full code provided somewhere, e.g., GitHub? Thank you!

[cdiggins]

I'm going to do some more testing first, and I'll share it shortly. Thanks!

[ricaun]

---

@cdiggins  escreveu:

No it doesn't, you can try it for yourself using the Bowerbird scripting solution. What happens is that you get a few messages and then it stops sending Idle events. So work inside of the queue, remains unprocessed.

---

I tested an standalone plugin and works fine without PokeRevit, I use the BackgroundProcessor in the code: https://github.com/ara3d/revit-background-processor. The Idling event never stops.

Don't know what process you are running the in Idling, I only tested with Thread.Sleep and still triggers normally. I tested with a 1 second delay, which makes Revit really slow/laggy, and it works fine with Revit minimized/unfocused.