topic Re: Plugin languages comparison in Revit API Forum

Plugin languages comparison

jeroen.van.halderen — Mon, 13 Sep 2021 14:38:44 GMT

Dear fellow developers,

While I was learning some programming techniques I heard somewhere that C++ is the fastest language out there, so I decided to do a test to see which of the three languages you can use to write a Revit plugin is the fastest. During my test it occurred to me that, because the Revit API is managed, the C++ implementation you can use to write a Revit plugin is considerably slower than native C++.

My test was to create 1000 modelcurves in Revit, the result are as follows:

With visual C++ it took 4 seconds, 733 milliseconds and 9619 ticks,

With C# it took 7 seconds, 989 milliseconds and 4320 ticks,

With VB.NET it took 5 seconds, 16 milliseconds and 3316 ticks.

The code for each of the three plugins is as follows:

Visual C++:

-Header:

#pragma once using namespace System; namespace HelloRevitCpp { [Autodesk::Revit::Attributes::TransactionAttribute( Autodesk::Revit::Attributes::TransactionMode::Manual)] [Autodesk::Revit::Attributes::RegenerationAttribute( Autodesk::Revit::Attributes::RegenerationOption::Manual)] public ref class Command : Autodesk::Revit::UI::IExternalCommand { public: virtual Autodesk::Revit::UI::Result Execute( Autodesk::Revit::UI::ExternalCommandData^ commandData, System::String^% message, Autodesk::Revit::DB::ElementSet^ elements); }; }

-Main code:

#include "HelloRevitCpp.h" using namespace Autodesk::Revit::DB; using namespace Autodesk::Revit::UI; using namespace HelloRevitCpp; Result Command::Execute(ExternalCommandData^ commandData, String^% message, ElementSet^ elements) { Diagnostics::Stopwatch^ watch = gcnew Diagnostics::Stopwatch(); watch->Start(); Document^ doc = commandData->Application->ActiveUIDocument->Document; Plane^ plane = Plane::CreateByNormalAndOrigin(gcnew XYZ(0, 0, 1), gcnew XYZ()); Transaction trans(doc); trans.Start("Test"); SketchPlane^ sketchPlane = SketchPlane::Create(doc, plane); for (int i{ 0 }; i < 1000; ++i) { doc->Create->NewModelCurve(Line::CreateBound(gcnew XYZ(), gcnew XYZ(i + 1, 0, 0)), sketchPlane); } trans.Commit(); watch->Stop(); TaskDialog testDialog("Test"); testDialog.Show("Test", "It took " + watch->Elapsed.Seconds + " seconds, " + watch->Elapsed.Milliseconds + " milliseconds and " + watch->Elapsed.Ticks + " ticks to create 1000 modelcurve"); return Result::Succeeded; }

C#:

using System.Diagnostics; using Autodesk.Revit.Attributes; using Autodesk.Revit.DB; using Autodesk.Revit.UI; namespace HelloRevitCS { [Transaction(TransactionMode.Manual)] public class Command : IExternalCommand { public Result Execute(ExternalCommandData commandData, ref string message, ElementSet elements) { Stopwatch watch = new Stopwatch(); watch.Start(); Document doc = commandData.Application.ActiveUIDocument.Document; Plane plane = Plane.CreateByNormalAndOrigin(XYZ.BasisZ, new XYZ()); Transaction trans = new Transaction(doc); trans.Start("Test"); SketchPlane sketchPlane = SketchPlane.Create(doc, plane); for (int i = 0; i < 1000; ++i) { doc.Create.NewModelCurve(Line.CreateBound(new XYZ(), new XYZ(i + 1, 0, 0)), sketchPlane); } trans.Commit(); watch.Stop(); TaskDialog.Show("Test", "It took " + watch.Elapsed.Seconds + " seconds, " + watch.Elapsed.Milliseconds + " milliseconds and " + watch.Elapsed.Ticks + " ticks to create 1000 modelcurves"); return Result.Succeeded; } } }

VB.NET:

Imports Autodesk.Revit.Attributes Imports Autodesk.Revit.UI Imports Autodesk.Revit.DB <Transaction(TransactionMode.Manual)> Public Class Command Implements IExternalCommand Public Function Execute(commandData As ExternalCommandData, ByRef message As String, elements As ElementSet) As Result Implements IExternalCommand.Execute Dim watch As Stopwatch = New Stopwatch watch.Start() Dim doc As Document = commandData.Application.ActiveUIDocument.Document Dim plane As Plane = plane.CreateByNormalAndOrigin(XYZ.BasisZ, New XYZ()) Dim trans As Transaction = New Transaction(doc) trans.Start("Test") Dim sketchPlane As SketchPlane = SketchPlane.Create(doc, plane) For i = 1 To 1000 doc.Create.NewModelCurve(Line.CreateBound(New XYZ(), New XYZ() + XYZ.BasisY.Multiply(i)), sketchPlane) Next trans.Commit() watch.Stop() TaskDialog.Show("Runtimetest", "It took " & watch.Elapsed.Seconds & " seconds, " & watch.Elapsed.Milliseconds & " milliseconds and " & watch.Elapsed.Ticks & " ticks to create 1000 modelcurves") Return Result.Succeeded End Function End Class

Re: Plugin languages comparison

sahin.ikbal — Tue, 14 Sep 2021 04:26:52 GMT

Interesting comparison 🙂

Re: Plugin languages comparison

RPTHOMAS108 — Tue, 14 Sep 2021 22:23:08 GMT

I always knew VB.Net was 2 seconds quicker than C#!

I've never been convinced that such comparisons are truly accurate i.e. was the machine restarted between tests and in the exact same state. There is an amount or graphics caching that goes on I've sometimes noticed the same test conducted has differing results, best to take an average of multiple tests. Especially considering the state of other OS processes and services are constantly changing.

Note that your VB.Net code is not exactly the same as the other two. Your line is heading in the Y direction and you are using XYZ.Multiply rather than the constructor.

Code Revit API in C++?...Never in a million years. Using a managed language is the way to go. You can use managed C++ CLR compliant, so that statement in itself is a bit ambiguous. I guess these choices are based on personal experience. Noticed some RevitAPI code in F# once.

Re: Plugin languages comparison

jeroen.van.halderen — Wed, 15 Sep 2021 14:31:21 GMT

I know it would have been better to take an average of multiple tests because the state of my machine and maybe even the state of Revit changed between tests, but I didn't have time to do multiple tests when I wrote this post. The results of my little comparison shouldn't be taken as the truth, if you want to know in which programming language your tool runs fastest I would advise to just test it for yourself.

Thanks for pointing out that my VB.NET code isn't quite the same as in the other two languages, I hadn't noticed that yet!

Re: Plugin languages comparison

jeroen.van.halderen — Tue, 02 Nov 2021 14:13:22 GMT

Dear fellow developers,

It took me quite long, but I have found some time to correct the mistake in my VB.NET code pointed out by @RPTHOMAS108 and run multiple tests to get a better comparison between the three programming languages.

First of all, here's the corrected VB.NET code:

Imports Autodesk.Revit.Attributes Imports Autodesk.Revit.UI Imports Autodesk.Revit.DB <Transaction(TransactionMode.Manual)> Public Class Command Implements IExternalCommand Public Function Execute(commandData As ExternalCommandData, ByRef message As String, elements As ElementSet) As Result Implements IExternalCommand.Execute Dim watch As Stopwatch = New Stopwatch watch.Start() Dim doc As Document = commandData.Application.ActiveUIDocument.Document Dim plane As Plane = plane.CreateByNormalAndOrigin(XYZ.BasisZ, New XYZ()) Dim trans As Transaction = New Transaction(doc) trans.Start("Test") Dim sketchPlane As SketchPlane = SketchPlane.Create(doc, plane) For i = 1 To 1000 doc.Create.NewModelCurve(Line.CreateBound(New XYZ(), New XYZ(0, i, 0)), sketchPlane) Next trans.Commit() watch.Stop() TaskDialog.Show("Runtimetest", "It took " & watch.Elapsed.Seconds & " seconds, " & watch.Elapsed.Milliseconds & " milliseconds and " & watch.Elapsed.Ticks & " ticks to create 1000 modelcurves") Return Result.Succeeded End Function End Class

Secondly, to get an acceptable average testresult I ran each of the three plugins that I made ten times, the results are as follows:

During these tests I had a few programs running in the background, but not more than what I would consider to be normal, so the result should be fairly comparable to what the average Revit user would get.

I hope this helps you in your journey that is learning how to program.

Re: Plugin languages comparison

RPTHOMAS108 — Tue, 02 Nov 2021 14:24:59 GMT

Thanks for the update, just two questions really:

What order did you conduct the tests in i.e. did you intermingle tests for each programming language or run each batch one after the other e.g. did tests for VB.Net come at end after other languages?

What was the state of Revit between each test i.e. did you restart Revit at all or restart the system in general?

Given the path of travel of .Net I'd probably be choosing C# if starting out today.

Re: Plugin languages comparison

jeroen.van.halderen — Tue, 02 Nov 2021 18:32:40 GMT

I ran each sequence of 10 runs for each plugin one after the other, without restarting Revit or my computer in between. I only restarted Revit once after the first run of the VB.NET plugin because I closed Revit by mistake, however it didn't make a really big difference in execution time, so I think it's okay.

Re: Plugin languages comparison

Sean_Page — Tue, 02 Nov 2021 21:10:34 GMT

This appear to be more what I'd expect to see especially between VB and C# as they should basically run identical in IL.

Re: Plugin languages comparison

RPTHOMAS108 — Wed, 03 Nov 2021 01:38:10 GMT

Yes in the end it all ends up as MSIL. Perhaps in a slightly different form due to language but not to a significant extent regarding performance.

Below are the results I gathered. Using your code I compiled into three assemblies and invoked via an F# assembly. Each assembly was invoked in the order VB/CS/CPP 11 times. One set of results was discarded (debug hitting break mode due to ContextSwitchDeadlock).

namespace RevitAddin open System open Autodesk.Revit.UI open Autodesk.Revit.DB open System.Reflection open System.Collections.Generic open System.Diagnostics open System.Linq open VB_Test open RevitTestCS open RevitTestCPP [<Autodesk.Revit.Attributes.RegenerationAttribute(Autodesk.Revit.Attributes.RegenerationOption.Manual)>] [<Autodesk.Revit.Attributes.Transaction(Autodesk.Revit.Attributes.TransactionMode.Manual)>] type EntryPoint1() = interface IExternalCommand with member x.Execute(commandData:Autodesk.Revit.UI.ExternalCommandData, message:byref<String>, elements:Autodesk.Revit.DB.ElementSet) = let IntUIDoc = commandData.Application.ActiveUIDocument let IntDoc = commandData.Application.ActiveUIDocument.Document let vb = new VB_Test.Command() let cs = new RevitTestCS.Command() let cpp = new RevitTestCPP.Command() for i = 0 to 10 do vb.Execute(commandData, &message, elements) |> ignore cs.Execute(commandData, &message, elements) |> ignore cpp.Execute(commandData, &message, elements) |> ignore Result.Succeeded

Test_VB			Test_CS			Test_CPP
Seconds	Milliseconds	Ticks	Seconds	Milliseconds	Ticks	Seconds	Milliseconds	Ticks
3	590	35909869	3	539	35398977	3	591	35916002
3	570	35708886	3	563	35632770	3	550	35503094
3	792	37927712	3	620	36206010	3	515	35157249
3	495	34956705	3	544	35446136	3	524	35249609
3	509	35098848	3	538	35384087	3	489	34898128
3	548	35482160	3	567	35675425	3	529	35294007
3	511	35115181	3	778	37784373	3	558	35589702
3	606	36065778	3	600	36006141	3	580	35802883
3	537	35378800	3	525	35251008	3	542	35423529
3	530	35308956	3	626	36269563	3	551	35512552
Average
3	568.8	35695289.5	3	590	35905449	3	542.9	35434675.5

Re: Plugin languages comparison

RPTHOMAS108 — Wed, 03 Nov 2021 12:39:33 GMT

Out of curiosity also tested F# and is slightly slower than the others. Tried with and without transaction using structure but slower result was the same.

Also found on this occasion C++ to be slower, bucking the trend (don't know why that was).

I suppose in the end people will choose a language based on the features it offers.

FS			VB			CS
Seconds	Milliseconds	Ticks	Seconds	Milliseconds	Ticks	Seconds	Milliseconds	Ticks
4	610	46104674	3	567	35675102	3	541	35412883
4	242	42423981	3	588	35889337	3	557	35570359
4	305	43057801	3	552	35525057	3	774	37741724
4	337	43375458	3	584	35847504	3	556	35563868
4	316	43165970	3	622	36229785	3	572	35727086
4	289	42898363	3	574	35746935	4	182	41821380
4	281	42814815	3	582	35825139	3	574	35744402
4	258	42583895	3	572	35727252	3	624	36248141
4	336	43365424	3	572	35727866	3	578	35784266
4	289	42898260	3	588	35886823	3	586	35869409
Average
4	326.3	43268864.1	3	580.1	35808080	3.1	554.4	36548351.8

CPP
Seconds	Milliseconds	Ticks
3	568	35687214
3	570	35703568
3	550	35507877
3	611	36118407
3	571	35716139
3	562	35622363
3	804	38046229
3	593	35937931
3	745	37451302
3	585	35852185

3	615.9	36164321.5

namespace FS_Test open System open Autodesk.Revit.UI open Autodesk.Revit.DB open System.Reflection open System.Collections.Generic open System.Diagnostics open System.Linq //[<Autodesk.Revit.Attributes.RegenerationAttribute(Autodesk.Revit.Attributes.RegenerationOption.Manual)>] //[<Autodesk.Revit.Attributes.Transaction(Autodesk.Revit.Attributes.TransactionMode.Manual)>] type EntryPoint1() = //interface IExternalCommand with member public x.Execute(commandData:Autodesk.Revit.UI.ExternalCommandData, message:byref<String>, elements:Autodesk.Revit.DB.ElementSet) = let watch:Stopwatch = new Stopwatch() watch.Start() let IntUIDoc = commandData.Application.ActiveUIDocument let IntDoc = commandData.Application.ActiveUIDocument.Document let plane:Plane = Plane.CreateByNormalAndOrigin(XYZ.BasisZ, XYZ.Zero) let Lst = new List<ElementId>() let TS:TransactionStatus = use Tx = new Transaction(IntDoc,"Transaction Name") if(Tx.Start() = TransactionStatus.Started) then for i = 1 to 1000 do let sketchPlane:SketchPlane = SketchPlane.Create(IntDoc, plane) let MC:ModelCurve = IntDoc.Create.NewModelCurve (Line.CreateBound (XYZ.Zero,XYZ.Zero + XYZ.BasisY.Multiply((float i))),sketchPlane) Lst.Add(MC.Id) Tx.Commit() else TransactionStatus.Uninitialized let TS:TransactionStatus = use Tx = new Transaction(IntDoc,"Transaction Name") if(Tx.Start() = TransactionStatus.Started) then IntDoc.Delete(Lst) |> ignore Tx.Commit() else TransactionStatus.Uninitialized watch.Stop() Debug.WriteLine("Test_FS",watch.Elapsed.Seconds.ToString() + ":" + watch.Elapsed.Milliseconds.ToString() + ":" + watch.Elapsed.Ticks.ToString()) Result.Succeeded