Why Using LINQ to Filter Shared Parameter is Slower Than a While-Loop?

[JC_BL]

I am trying to get a list of ElementId of FabricationParts filtered by their shared parameter.  I thought using LINQ statement should be faster than using a while-loop.  But the result is the opposite.  Both are slow; but my LINQ statement is slower than my while-loop.  I try to find a way to speed up my LINQ.

In the examples below, I try to use LINQ to retrieve a list of ElementId of FabricationPart whose shared parameter "OurMechNum" is "123".  This LINQ takes 13 seconds to finish:

List<ElementId> lisFoundPcElmIds =
   new FilteredElementCollector( doc )
      .OfCategory( BuiltInCategory.OST_FabricationDuctwork )
      .WhereElementIsNotElementType()
      .Where( x => x.LookupParameter( "OurMechNum" ) != null  &&
                   x.LookupParameter( "OurMechNum" ).HasValue &&
                   x.LookupParameter( "OurMechNum" ).AsString().Equals( "123" ) )
      .Select( x => x.Id )
      .ToList();

On the other hand, using a while-loop to do the same thing (like the one shown below) takes 7 seconds:

FilteredElementCollector collDuct =
   new FilteredElementCollector( doc )
      .collDuct.OfCategory( BuiltInCategory.OST_FabricationDuctwork )
      .WhereElementIsNotElementType();

List<ElementId> lisFoundPcElmIds = new List<ElementId>();
FilteredElementIterator eit = collDuct.GetElementIterator();
eit.Reset();
while( eit.MoveNext() )
   {
   FabricationPart curPc = eit.Current as FabricationPart;
   if ( curPc != null )
      {
      // Filter out the current piece if it doesn't have mechanical number 123.
      Parameter oParamMchNum = curPc.LookupParameter( "OurMechNum" );
      if ( oParamMchNum == null )
         continue;
      if ( ! oParamMchNum.HasValue )
         continue;
      if ( ! oParamMchNum.AsString().Equals( "123" ) )
         continue;

      // Here, we have found a piece that has mechanical number 123.
      lisFoundPcElmIds.Add( curPc.Id );
      }
   }

Both versions are slow.  But somehow my LINQ is slower than my while-loop.  This is odd.

Seem like the reason why the LINQ is slow has to do with the filter of the shared parameter.  If I remove the filter of the shared parameter "OurMechNum" like the LINQ statement shown below, the LINQ statement runs very fast (like 1 second):

List<ElementId> lisFoundPcElmIds =
   new FilteredElementCollector( doc )
      .OfCategory( BuiltInCategory.OST_FabricationDuctwork )
      .WhereElementIsNotElementType()
      .Select( x => x.Id )
      .ToList();

Unfortunately, I cannot remove the filter for my application.  What should I do to improve the query?

Please help.  Thanks,

JC_BL

[nice3point · Accepted solution]

Because you call LookupParameter 3 times in the lambda function.

Use statement lambda for it

var lisFoundPcElmIds =
    new FilteredElementCollector( doc )
        .OfCategory( BuiltInCategory.OST_FabricationDuctwork )
        .WhereElementIsNotElementType()
        .Where( x =>
        {
            var parameter = x.LookupParameter("OurMechNum");

            if (parameter == null) return false;
            if (!parameter.HasValue) return false;
            if (parameter.AsString().Equals("123")) return true;
            return false;
        })
        .Select( x => x.Id )
        .ToList();

I have not changed the code style, however the variable names should have been improved

[JC_BL]

Very Good!  Getting rid of the extra calls to LookupParameter() has significantly reduced the time that my LINQ statement takes to finish.  Now my LINQ statement only takes half the time as the previous version with the 2 extra LookupParameter().  This means now my LINQ statement takes as long as my while-loop to finish.  This solves the original question in my post.  And I have marked your post as the solution.  Thanks for your help!

One extra question:

Why my LINQ statement takes as long as my while-loop?  I thought using LINQ statement will do most of the filtering work inside Revit program module (instead of inside my program), and will significantly improve the performance.  But that doesn't seem to be the case.

JC_BL

[TripleM-Dev.net · Accepted solution]

Maybe look into: ElementParameterFilter Class ?

[ctm_mka]

as a side note, try removing the "null" and "hasvalue" checks completely, you shouldn't need them since you are looking for a specific value. unless you were getting incorrect returns before? Also, try a foreach loop, the while seems and odd choice. as for the why, not a programmer, but i have read something about the size of the data set affecting the performance of the LINQ vs loop.

[jeremy_tammik]

As @TripleM-Dev.net  points out, the Revit API provides dedicated filter functionality for parameter values. Both LINQ and the .NET while loop can only process data that has been marshalled out of the Revit memory space into the .NET context. The marshalling takes a lot more time than the simple Revit parameter read value and compare. The time is spent transferring data from one context to another. Read here about Quick, Slow and LINQ filters:

• https://thebuildingcoder.typepad.com/blog/2015/12/quick-slow-and-linq-element-filtering.html

[JC_BL]

Thanks for the suggestion!  I have changed the LINQ to ask it to use ElementParameterFilter.  Originally, I was a bit skeptical about this because Revit API describes ElementParameterFilter as a "slow" filter.  The result is more positive than negative.  The LINQ using ElementParameterFilter always runs slow in the very first run.  But it runs VERY FAST in subsequent runs.  Seem like Revit has cached the result to speed up the filter.  This is good.

The only complication is that ElementParameterFilter needs to use ParameterValueProvider, and ParameterValueProvider needs the ElementId of the shared parameter.  Luckily I find a sample program in this forum to retrieve the ElementId of the shared parameter.  The sample program was in VB.  I changed it to C#:

private Parameter paramGetInstanceOfSharedParameter( String sParamName )
   // Get one instance of element that has the specified shared parameter.
   //
   // Reference:
   //    Title: Element ID of shared parameter
   //    URL:   https://forums.autodesk.com/t5/revit-api-forum/element-id-of-shared-parameter/td-p/7473547
   {
   SharedParameterApplicableRule oFilterRule = new SharedParameterApplicableRule( sParamName   );
   ElementParameterFilter        oFilter     = new ElementParameterFilter       ( oFilterRule  );

   FilteredElementCollector collector = new FilteredElementCollector( g.doc );
   collector.WhereElementIsNotElementType();
   IList<Element> lisElements = collector.WherePasses( oFilter ).ToElements();

   foreach( Element eCur in lisElements )
      {
      IList<Parameter> lisParams = eCur.GetParameters( sParamName );
      foreach( Parameter paramCur in lisParams )
         {
         if ( paramCur != null )
            return paramCur;
         }
      }

   return null;
   }

The latest version of my LINQ that is incorporated with the use of ElementParameterFilter:

// Get one instance of element that has the shared parameter for mechanical number.
Parameter paramMechNum = this.paramGetInstanceOfSharedParameter( "OurMechNum" );
if ( paramMechNum == null )
   return Result.Failed;

// Get a list of ElementId of duct that have the mechanical number 123.
//
// Reference:
//    Title: ElementParameterFilter with a Shared Parameter:
//    URL:   https://thebuildingcoder.typepad.com/blog/2010/08/elementparameterfilter-with-a-shared-parameter.html

ParameterValueProvider    pvp         = new ParameterValueProvider( paramMechNum.Id );
FilterStringRuleEvaluator oStrEqual   = new FilterStringEquals();
FilterValueRule           oFilterRule = new FilterStringRule( pvp, oStrEqual, "123" );
ElementParameterFilter    oFilter     = new ElementParameterFilter( oFilterRule );

List<ElementId> lisFoundPcElmIds =
   new FilteredElementCollector( g.doc )
      .OfCategory( BuiltInCategory.OST_FabricationDuctwork )
      .WhereElementIsNotElementType()
      .WherePasses( oFilter )
      .Select( x => x.Id )
      .ToList();

Thanks for the suggestion to speed up the query.  This fixes my extra question of this post.

JC_BL

[JC_BL]

I assume I need to check if the parameter object is null or not -- at the minimum.  Otherwise, the clause below may crash the program:

   oParamMchNum.AsString().Equals( "123" )

Anyway, I copied the code from a sample program posted in this forum.  Therefore, I tend not to change too much of the code from the sample program.

I also copy the while-loop from another sample program.  In any case, I doubt changing the loop to a foreach-loop will make a difference that I can measure.

Thanks for your comments.

JC_BL

[admaecc211151]

If you insure that no same name pararmeters in your file, you can find the Definition first.

And then using element.get_parameter(Definition) to get parameter.

//Find the Definition first
//
List<ElementId> lisFoundPcElmIds =
   new FilteredElementCollector( doc )
      .OfCategory( BuiltInCategory.OST_FabricationDuctwork )
      .WhereElementIsNotElementType()
      .Where( x => x.get_parameter( theParaDef )
         ?.AsString()
         ?.Equals( "123" ) 
      )
      .Select( x => x.Id )
      .ToList();

It might be a little faster since no need to lookup a string for all collect elements?

[jeremy_tammik]

Don't say "might be"... benchmark it 🙂

[JC_BL]

Hello, I try to use your suggestion.  But the program cannot be compiled.  The following is the sample program that I try:

Guid guidMchParam = new Guid( "XXXXXXXX-XXX-XXXX-XXXX-XXXXXXXXXXXX" );
   // The GUID above is not real.

List<ElementId> lisFoundPcElmIds =
   new FilteredElementCollector( doc )
      .OfCategory( BuiltInCategory.OST_FabricationDuctwork )
      .WhereElementIsNotElementType()
      .Where( x => x.get_Parameter( guidMchParam )
         ?.AsString()
         ?.Equals( "123" )
         )
      .Select( x => x.Id )
      .ToList();

I have to change "x.get_parameter" in your sample with "x.get_Parameter" because yours triggers a compiler error saying that get_parameter cannot be found in the definition of Element (or something like that).

Unfortunately, the above code triggers the following 2 compiler errors:

• Cannot convert lambda expression to intended delegate type because some of the return types in the block are not implicitly convertible to the delegate return type
• Cannot implicitly convert type 'bool?' to 'bool'. An explicit conversion exists (are you missing a cast?)

Would you please help me with this?  Thanks.

JC_BL

[admaecc211151]

Definition is a type. Not guid.

//lookup for one element
Definition paraDef = anElement.LookupParameter("theParameterName").Definition;

//get the parameter by def
element.get_parameter(paraDef);

[admaecc211151]

Sorry. I've check after that by Stopwatch.
element.get_parameter(Definition) is faster than element.LookupParameter("Name").
Since get_parameter returns immediately when found target parameter, but LookupParameter always search for all parameters.

[TripleM-Dev.net]

Yes, "ElementParameterFilter" is slow because of the Parameter search part. It needs to "unpack" the element in memory to retrieve the parameters.

That's also the reason on a second pass it's quicker, the object info is already in memory (unless it's changed)

The same as "LookupParameter" or "get_Parameter" does, only ElementParameterFilter is made for this kind of situation.

Only if you need to edit the parameter I would use "get_Parameter", or if a selection of elements need parameters checked. But for selecting elements based on a parameter, I would use ElementParameterFilter.

Ps. For this reason also, first reduce the number of elements in the FilteredElementCollector by using QuickFilters like the OfCategory and then the slow filters, so they are applied to less elements.

[JC_BL]

Thanks for the explanation why the "slow" filter can run fast after the first run.

As for applying a "quick" filter before applying the "slow" filter, I believe I can do this by having something like this:

List<ElementId> lisFoundPcElmIds =
   new FilteredElementCollector( doc )
      .OfCategory( BuiltInCategory.OST_FabricationDuctwork ) <-- Quick filter
      .WhereElementIsNotElementType() <-- Quick filter
      .WherePasses( oFilter ) <-- Slow filter
      .Select( x => x.Id )
      .ToList();

JC_BL

[JC_BL]

Thanks for showing how you retrieve the definition of a parameter.

Now I realize that your code is in Python, not in C#.  This explains the reason why I cannot use get_parameter() -- that is a Python function.  OK, I got it.

I have attempted to try your suggestion of using the definition of the parameter instead of using LookupParameter() as shown below:

// Get an element that has the mechanical-number shared parameter.

SharedParameterApplicableRule oFilterRule = new SharedParameterApplicableRule( "OurMechNum" );
ElementParameterFilter        oFilter     = new ElementParameterFilter( oFilterRule );

FilteredElementCollector collector = new FilteredElementCollector( doc );
collector.WhereElementIsNotElementType();
IList<Element> lisElements = collector.WherePasses( oFilter ).ToElements();   // <-- This takes time.
if ( lisElements.Count == 0 )
   return Result.Failed;
Element oAnyElement = lisElements[ 0 ];

// Get the ForgeTypeId of the definition of the mechanical-number shared parameter.
Definition paramMechDef = oAnyElement.LookupParameter( "OurMechNum" ).Definition;
ForgeTypeId idMechParamDefType = paramMechDef.GetGroupTypeId();

// Find a list of elements that have the mechanical-number 123.
List<ElementId> lisFoundPcElmIds =
   new FilteredElementCollector( doc )
      .OfCategory( BuiltInCategory.OST_FabricationDuctwork )
      .WhereElementIsNotElementType()
      .Where
         ( x =>
            {
            Parameter oParamMchNum = x.GetParameter( idMechParamDefType ); <-- Error
            // Run Time Error "The argument does not identify a built-in parameter"
            if (   oParamMchNum == null                    ) return false;
            if ( ! oParamMchNum.HasValue                   ) return false;
            if ( ! oParamMchNum.AsString().Equals( "123" ) ) return false;
            return true;
            }
         )
      .Select( x => x.Id )
      .ToList();

But there are two problems: (1) I need to get an instance of the element that has the shared parameter, and then retrieve the definition of the shared parameter from the element; this step actually takes several seconds to finish in the first run (this step should take no time in subsequent runs); but that is less a problem.  (2) The real problem is that seem like Element.GetParameter(idParamDefType) expects an ID of a built-in parameter, not for a shared parameter; this clause triggers a run-time-error.

Seem like using ElementParameterFilter is more appropriate for filtering by shared parameters.

JC_BL

[admaecc211151]

Well... I'm using C#, and maybe is a old version? (I can't find it on Revit API doc website now)

There is get_parameter in my dll reference, but no GetParameter.

Maybe I should prepare for the time method changing in my reference.