Why do some shape-related nodes have their scheduling type set to serial?

[jmreinhart]

To give a bit of context, I am working on a custom node that takes an array of matrices as an input and exports a curve where the cvs are driven by those matrices (I'm aware you can do this with vanilla nodes). Most of the time the node works fine but it will occasionally randomly cause as crash. Using a debugger shows this error 

Exception thrown at 0x00007FF8AFC6792F (NurbsEngine.dll) in maya.exe: 0xC0000005: Access violation reading location 0x0000000000000000.

The error does not occur during the compute function of my custom node.

I was able to prevent the crash by changing the scheduling type to serial, but I don't fully understand why that fixes the issue. Below is all the information I was able to find in trying to understand the issue.

What is serial scheduling?

To clarify, I'm talking about serial not globallySerial.

Depending on where you look in the documentation, the details can be a bit unclear. Here is the information provided from several Autodesk sources:

https://damassets.autodesk.net/content/dam/autodesk/www/html/using-parallel-maya/2026/UsingParallelM...

All nodes of this type should be chained and executed sequentially

Maya Help | Evaluation Toolkit | Autodesk 

Asserts it is safe to run this node along with instances of other nodes. However, all nodes of this scheduling type are executed sequentially within the same evaluation chain.

Maya Help: MPxNode Class Reference

Groups are formed for nodes having this ShedulingType when they are directly connected to each other.

Within a same group nodes are guaranteed to not to be concurrently evaluated. However nodes in distinct groups can still be concurrently be evaluated.

You are able to see these groups by using:

Evaluation Tookit > Debugging > Evaluation Graph Inspection > Graphical Output > Visual Scheduling Graph

Here you see the definition from the MPxNode documentation in action. When the rebuildCurve node and pointOnCurveInfo nodes are directly connected, they get flagged as a serial group for the purpose of scheduling. I was not able to locate a list of serial nodes, and there doesn't seem to be a command we can use the query that information, so I did some testing with various nodes and the scheduling graph to find out. 

Which nodes are serial?

I mainly limited my testing to nurbs-curve related nodes since that was most related to my issue.

This is not a comprehensive list, just the ones I was able to find during my testing:

• Curve-Related
  • rebuildCurve
  • pointOnCurveInfo
  • curveInfo
  • motionPath
  • fitBSpline
  • reverseCurve
  • nearestPointOnCurve
  • The nurbsCurve shape node itself is NOT serial
• Others
  • polySmoothFace
  • transformGeometry

It might be possible that these nodes are not always serial, since at least on MPxNode the schedulingType for a node is returned by a function (see here) which could contain some logic, but I wasn't able to find an example of this from Autodesk.

Why are they serial?

I understand vaguely that nodes are serial if them accessing the same data would cause problems. In the past I've noticed that pointOnCurveInfo nodes (and some others) sometimes appear suspiciously long in the profiler. Below you can see the scheduling graph and the profiler graph for the same scene, and you can see that the pointOnCurveInfo nodes that are not in the serial group take a long time to evaluate, while the ones in the serial group take less time individually. 

My guess is that this has something to do with the fact that Maya does not cache shape data in the same way it caches basic numerical data. You can observe this by creating a mesh, applying a skinCluster, then a lattice, and then profiling the scene while moving the lattice.

You'll be able to see the skinCluster does get recomputed even though none of its inputs were dirty.

But with a simple scene like this if I update locator4, the blendColors node does not get re-update because the output is clean and we can use the cached output from the last time it evaluated. 

I think that Maya does this because caching the mesh output after each deformer would cost too much memory. 

So my best explanation for the difference in evaluation time is that even though the pointOnCurveInfo nodes are in parallel, they all need to access the same curve data, which requires them to either copy it, or take turns accessing it (there's probably a technical term for this), either of which would add overhead. That same overhead does not exist if they are scheduled in serial.

But when you connect a nurbs curve shape node to several pointOnCurveInfo nodes it can schedule them in parallel, so there is clearly some safe guards that allow for parallel scheduling even if they are bottlenecked by being unable to access memory in parallel. Then why if you replace the nurbs curve shape node with a rebuildCurve node does Maya require the nodes to be scheduled serially?

[brentmc]

Hi,

When the Evaluation Manager (EM) was added to Maya it had to deal with existing nodes which were not written with parallel execution in mind and scheduling modes are a way to allow these nodes to run safely in EM. This model also allows nodes to be upgraded over time to support better parallelism.

Here is a summary of the different scheduling types with diagrams. Note: "instances of other nodes "refers to nodes that have a different underlying node type.

Parallel

• node and all third-party libraries used by the node are thread-safe

• instances of this node may be evaluated at the same time as instances of other nodes without restriction

• maximum performance
  

Serial

• safe to run this node with instances of other nodes

• all nodes with this scheduling type should be executed sequentially within the same evaluation chain

• Trade off between performance and safety – default for OpenMaya plug-in nodes

• In the diagram below E and B are executed at the same time (because they are in different sub-graphs) but E & F are evaluated serially (since they are in the same sub-graph)
  

Globally Serial

• safe to run this node with instances of other node types

• only a single instance of this node type should be run at a time

• use this type if the node relies on static state, which could lead to unpredictable results if multiple node instances are simultaneously evaluated

• Note: the same restriction may apply if third-party libraries store state. e.g. Python GIL (Global Interpreter Lock)
  

Untrusted

• node is not thread-safe and that no other nodes should be evaluated while an instance of this node is evaluated

• untrusted nodes are deferred as much as possible (that is, until there is nothing left to evaluate that does not depend on them), which can introduce costly synchronization

• e.g. Expression Nodes
  

I don't know the specific reason the curve related nodes need to be run serially but it is likely done because of limitations in the underlying NURBS library used by these nodes. (which was also developed a long time ago when computers were mostly single core)

In this case there is probably some cached data in the curve to speed up computation so nodes that operate on the same curve (e.g. in the same sub-graph) need to run serially.

[jonah_reinhart]

In this example, E and F would only be scheduled in serial if the scheduling type of node D is also serial, right?

That is one thing I am still confused about. My understanding was that the scheduling and evaluation of a node would only depend on the data that it receives as inputs, but in the example below, only the pointOnCurveInfo nodes that are downstream from the rebuildCurve node need to be scheduled serially, which seems to indicate that where the data comes from also needs to be considered? 

I also did a bit more digging, and I was able to find some mesh-related nodes that are marked as serial as well:

• polyRemesh
• closedPointOnMesh
• curveFromMeshEdge
• polyColorDel

So it doesn't seem to be a limitation caused by the exclusively by the nurbs library.

Is it maybe that DAG nodes (like the nurbsCurve shape nodes) can have their shape data accessed with less restrictions?

[brentmc]

Hi,

In this example, E and F would only be scheduled in serial if the scheduling type of node D is also serial, right?

---

No, E & F are scheduled serially because they have a scheduling type of Serial and they are in the same evaluation chain.
If they were in a different chain (like B) they could be run in parallel.

That is one thing I am still confused about. My understanding was that the scheduling and evaluation of a node would only depend on the data that it receives as inputs, but in the example below, only the pointOnCurveInfo nodes that are downstream from the rebuildCurve node need to be scheduled serially, which seems to indicate that where the data comes from also needs to be considered?

---

In an ideal world this would be correct but scheduling modes were added because there can be hidden dependencies that a node may be relying upon. e.g. cached or static data

I was using NURBS as an example but many nodes use Serial since that is the default scheduling mode. There are lots of parallel nodes but they are mostly math, shading, and conversion nodes along with a handful of deformer nodes. There are only a few untrusted and globally serial nodes.

I presume you already know about the Evaluation Toolkit scheduling tools that let you inspect the scheduling type of selected nodes in your scene.
Note: I can't speak to why a specific not is say serial and not parallel since I wasn't part of the Animation team when those decisions were made.

[jonah_reinhart]

No, E & F are scheduled serially because they have a scheduling type of Serial and they are in the same evaluation chain.
If they were in a different chain (like B) they could be run in parallel.

---

That doesn't align with my observations. For example, in this scene, the lower pointOnCurveInfo nodes that are arranged like in your example, but they are not displayed in a serial group when I used the evaluation toolkit debug tools, and according to the profiler they are evaluating in parallel. It seems as though for a node to be scheduled serially, it needs to have serial as its scheduling type, and it needs to be directly connected to at least one other node that has serial as its scheduling type.  Or am I missing something?

 

[brentmc]

Honestly, I haven't looked deeply into this but the profiler is the source of truth and it will tell you which nodes are being serialized.

It is quite possible that the rules are more complex than what is presented in the whitepapers/docs.
i.e. the code changed but no one updated the docs!

I loaded up some test scenes I have on hand but I didn't have anything that is comparable to your scene. (and not so complex that it becomes too unwieldy to debug the evaluation graph)