Best unit test Framework 2022 ?

[TomB.ADV]

Hi,

I started developing a new plug-in for Revit recently, it's been a while since the last time I worked

on a new coding project, I was wondering which framework is best for the specific tast of making

a Revit Add-on ? I'm developing with VS22 for windows only.

[ricaun · Accepted solution]

Here are some unit tests:

• Dynamo's Revit Tester Framework - https://github.com/DynamoDS/RevitTestFramework
• Geberit's Revit Test Runner - https://github.com/geberit/Revit.TestRunner
• Speckle's Revit Test Runner - https://github.com/specklesystems/xUnitRevit

I have been using the Geberit version and was trying to create my own to run directly in Visual Studio 2022.

Is kinda working: https://youtube.com/shorts/ZwtmHP72_Zc

[TomB.ADV]

So if I understand correctly, Speckle's Revit Test Runner, will allow me to test my code against any .rvt model without manually activating each custom function/command in the add-on ? Is that right ? 

[danielJY6FB]

TomB.AD,

I believe so? I don't 100% understand what you are asking, but I will second that speckle's Revit unit test is the best available test framework for Revit that we've found. You can run test functions using the speckle UI, and you don't have to test a full external command - you can test individual methods.

Note that you still have to open Revit to use it, as I haven't seen a Revit Test Framework that can get around that - as far as I know Revit has to be open to have access to it's API, which is required to test anything that uses and Revit API calls. We've used Speckle a bit at our company, but eventually moved away from it in support of developing our own Revit Unit Test framework that better suited our needs and process.

[danielJY6FB]

ricaun,

Cool vid! I think it's awesome that you are developing your own! I think more Revit devs should be more focused on testing to ensure better and more stable applications! Is that test class calling the Revit API? I would be amazed if you found a way to get that working with the Revit API but outside of Revit - if so lemme know how!

Our company actually ended up making our own Revit Test Framework to better suit our needs, and we'd like to make it open source one day! Here's a screenshot:

[TomB.ADV]

@danielJY6FB @ricaun 

I was reffering to the below example where on line 4 there is a method calling a Revit model from the solution folder.

I could be wrong but looks like there is a method to load rvt model and execute API methods directly (line 7), meaning I don't have to open a new revit session to run tests.

https://github.com/specklesystems/xUnitRevit

[Fact]
public void WallsHaveVolume()
{
  var testModel = GetTestModel("walls.rvt");
  var doc = xru.OpenDoc(testModel);

  var walls = new FilteredElementCollector(doc).WhereElementIsNotElementType().OfCategory(BuiltInCategory.OST_Walls).ToElements();

  foreach(var wall in walls)
  {
    var volumeParam = wall.get_Parameter(BuiltInParameter.HOST_VOLUME_COMPUTED);
    Assert.NotNull(volumeParam);
    Assert.True(volumeParam.AsDouble() > 0);
  }
  doc.Close(false);
}

[danielJY6FB]

TomB.ADV,

Yes I believe Speckle provides methods for you to load and switch between revit files, among other utilities. However, I also though speckle tests needed to be run via the speckle test ui, which is a revit addin - so you have to open that to run the tests. I could be mistaken, as I haven't used speckle in a little while!

Edit:

Here's a link to a speckle unit test blog post - I think this is the framework we are talking about? It mentioned three parts, one of which is a revit addin that actually runs the tests:

https://speckle.systems/blog/xunitrevit/ 

[TomB.ADV]

I see, well, this is pritty much still better IMO compared to the alternatives but I wish there was a framework to execute tests without launching a new revit session, otherwise it is very restricting in terms of the machine specs I have to run the tests on 😞

[ricaun]

Without Revit open is not possible I guess, but you could use the Design Automation to run some tests: The demo that I created for the Forge: https://youtu.be/EL5uJm_Nj8g

The Test part is the missing key in my workflow, I was thinking to add a UI for testing in the AppLoader plugin but makes more sense to use the Test Explorer in Visual Studio 2022, just need to create a IPC between Revit and Visual Studio.

[TomB.ADV]

That looks very interesting, looks like your are building a useful solution, looking forward to see this developed.

Unfortenutlly, I didn't have the time to expirement with forge, I will look into it at a later stage, I guess
for the time being I will take whatever is available...

[danielJY6FB]

ricaun,

This is REALLY cool, thanks for sharing! I like the idea of using forge, and that does seem to be where Autodesk wants to head in the future.

Edit: Does running that via the Design Automation require the use of any Autodesk Cloud Credits?

[jeremy_tammik]

Very many thanks to all, especially ricaun, for the fruitful discussion. Summarised it here for posterity:

https://thebuildingcoder.typepad.com/blog/2022/11/unit-test-your-revit-add-in.html#6

[ricaun]

---

@danielJY6FB wrote:

Edit: Does running that via the Design Automation require the use of any Autodesk Cloud Credits?

---

Yes requires Autodesk Cloud Credits to use Design Automation, and could become too expensive to run each PR.

You are gonna pay for the time the DA is running, around 0.0334 Credits per minute or around $0.10 / Minute. (https://forge.autodesk.com/pricing)

Make sense to use before merging or releasing, especially for testing in each Revit version available.

One thing that is slow is opening a document, and if the document needs to be updated gonna wastes time converting every time the test runs.

[daniel]

In our soulution we have "decoupled" the Revit Api and our addin. 

So what we do is that we have created a layer on top of Revit, which we call "Model".  (and one API) 

The Model layer has no refrence to RevitAPI, just the API refrence then the API holds the Revit refrence. 

This layer basicly creates an copy of the Revit element or method. 

Implementing this as an class based on an interface, makes me able to mock the api 

behavior when unit testing. 

This means that when creating addin tools, we use our "MElement" class, instead of Element.. 

When calling an "MElement" we route the traffic down to the Element throug the implementation of this API func.

When testing, we asume api to give "correct" results.. and then we can Mock the results wich we 

excpect from the api, and test our logics. 

This means that we have no 3000++ unit tests running when building i azure devops. 

This is a bit of work to get going on, but do give som good decoupling from Revit when developing.. 

[jonathan.taVCBM2]

Hi ,

I have been looking into testing the revit plugins I am working on and decided to try the RevitTestFrameWork , mainly because I wanted to create a powershell script to copy dlls, models and run the test , simillar to what is shown in this AU session  . I have managed to git clone , replace the Revit API references with the Revit 2023 Dlls and compile successfully.

When I try to run a test from the GUI it just hangs , no response . 
However, when I run the console version I get the following error  :   

Unable to cast object of type 'System.Xml.XmlComment' to type 'System.Xml.XmlElement'.

I am guessing this may have something to do with either the .net version , or the nUnit part that produces the output ? 
Did anyone else get that error ? 

Thanks , Jonathan 

[ricaun]

It's finished, I created a unit test framework that supports multiple Revit versions and works inside Visual Studio or dotnet test.

Here is some information about ricaun.RevitTest framework.

• Sample: https://github.com/ricaun-io/RevitTest
• Discussions: https://github.com/ricaun-io/RevitTest/discussions
• Nuget: https://www.nuget.org/packages/ricaun.RevitTest.TestAdapter

And a video on how to start and some specific configurations.

If you have any questions just join the Discussion on GitHub.

Should work with the Revit 2025 as well. 😆

[jeremy_tammik]

Brilliant!  Thank you for that! 

Please note a small URL typo, an extraneous space at the end of the (second) link to the discussion:

https://github.com/ricaun-io/RevitTest/discussions%20

That doesn't work.

Happy Easter, and thank you for the nice Easter gift!

[ricaun]

Thanks, I fix the link. 😀

Now just wait for the Revit release!

Happy Easter everyone!

[eN27]

Congratulations, you finally made it public :D. It is about time to bring a better unit testing experience to the Revit community.

For those who prefer MSTest2 over NUnit, I am experimenting with a similar approach to remote unit test execution with my https://github.com/NeVeSpl/RevitTestLibrary