accessing rotateOrder in MPxTransformMatrx::asMatrix

[jmreinhart]

I have a custom transform node. The calculation of the matrix requires access to the transform nodes rotateOrder attribute. I'm trying to store it using validateAndSet like the documentation suggests for custom attributes. But when I do that it prevents the rotateOrder attribute from changing .It's stuck as "XYZ".

MStatus normalTransformNode::validateAndSetValue(
	const MPlug& plug,
    const MDataHandle& handle)
{
    MStatus status = MS::kSuccess;
    // Make sure that there is something interesting to process.
    if (plug.isNull())
        return MS::kFailure;

    MDataBlock block = forceCache();//???
    MDataHandle blockHandle = block.outputValue(plug, &status);

	//========================
	//	Rotate
	//========================
	if (plug == rotateOrder)
	{
		MGlobal::displayInfo("validate rotateOrder");
		int rotateOrderIn = plug.asInt();//???
		MGlobal::displayInfo(MString("set to:") + rotateOrderIn);

		normalTransformMatrix *ltm = getNormalTransformMatrix();

		ltm->setRotateOrder(rotateOrderIn);

		//blockHandle.setInt(rotateOrderIn);
		//blockHandle.setClean();
		//dirtyMatrix();

		//return MS::kSuccess;
	}

This part of the code is being evaluated but it always prints a value of 0.

[zewt]

rotateOrder is an enum (16-bit), not an int, so try asShort instead of asInt.

[jmreinhart]

Thanks for the tip. Unfortunately that didn't solve the problem.

[jmreinhart]

if (plug == rotateOrder)
	{
		short rotateOrderIn = handle.asShort();
		normalTransformMatrix *ltm = getNormalTransformMatrix();
		MEulerRotation::RotationOrder rotateOrder_val;

		MGlobal::displayInfo(MString("rotateOrderIn:") + rotateOrderIn);
		blockHandle.set(rotateOrderIn);


		switch (rotateOrderIn)
		{
		case 0:
			rotateOrder_val = MEulerRotation::kXYZ; break;
		case 1:
			rotateOrder_val = MEulerRotation::kYZX; break;
		case 2:
			rotateOrder_val = MEulerRotation::kZXY; break;
		case 3:
			rotateOrder_val = MEulerRotation::kXZY; break;
		case 4:
			rotateOrder_val = MEulerRotation::kYXZ; break;
		case 5:
			rotateOrder_val = MEulerRotation::kZYX; break;
		default:
			rotateOrder_val = MEulerRotation::kXYZ; break;
		}

		ltm->setRotateOrder_val(rotateOrder_val);
		blockHandle.setClean();

		// Mark the matrix as dirty so that DG information
		// will update.
		dirtyMatrix();
		return MS::kSuccess;
	}

this is my current code in the validateAndSetValue method. I gets the right value and stores it on the MTransformationMatrix but does not actually update the attribute.

[jmreinhart]

I've come back to this problem a few times this week but I don't understand what I'm doing wrong in the validateAndSet step thats keeping the attribute from updating. Is there something I need to do differently since it's a standard attribute but usually doesn't call validateAndSetValue?  I am using mustCallValidateAndSetValue() for rotateOrder.

[jmreinhart]

So I've found another issue which may point to the cause.

Loading the plugin for my MPxTransform that has the problem, actually causes all transform nodes in the scene (including constraints) to be unable to set their rotateOrder. I've attached a video of this strange problem. This persists after unloading the plugin and only closing Maya fixes it.

[zewt]

Just a guess from what I know about Maya attributes: MPxTransform nodes inherit from transform, and the rotateOrder attribute is actually inherited from the transform node.  It's not specific to your node.  So, when you mark it with mustCallValidateAndSet, you're actually doing that for all transform node types, not just your own.

[jmreinhart]

That makes sense. I'll try doing something like mustCallValidateAndSet(customTransform::rotateOrder).

Any guess as to why it's not able to change though?

I tried that and I still get the same result. 

[zewt]

I don't know, but it's modifying an attribute internal to the transform node, and internal nodes don't necessarily work the same as API nodes, so it's probably just breaking what the internal code expects.  The builtin nodes are a black box though, so it's hard to say.

[jmreinhart]

Validating and setting
Maya uses a standard validate and set mechanism for updating the attribute values of custom transforms. You are required to use this mechanism for any attributes that affect the matrix. The MPxTransform::mustCallValidateAndSet() must be called in the initialize() method for all attributes that affect the matrix of the transform. It ensures that the matrix is properly calculated when the attribute is changed and that the MPxTransform::validateAndSet() method is called for this attribute.

https://help.autodesk.com/view/MAYAUL/2016/ENU/?guid=__files_Writing_a_Custom_Transform_Node_Impleme...

This part of the documentation says that all the attributes that affect the matrix need to call "validateAndSet" but rotateOrder does not call it by default, but rotate, translate, and scale do. So it seems more like it needs to be used for attributes with constraint/limit stuff, and in the default transform node the attributes with constraint/locking stuff happen to affect the matrix. 

I'm going to do some test by flagging other default attributes with "mustCallValidateAndSetValue" and I may try using the setDependentsDirty method to get the data to the MPxTransformMatrix

[jmreinhart]

so, the setDependentsDirty method will not work because the rotatOrder plug will not be up-to-date when that method is evaluated.

[zewt]

Here's one workaround you could try.  Create another attribute, like "internalRotateOrder".  Mark it affected by rotateOrder, and compute it to just output the value of rotateOrder, and set it to hidden and not writable.  Then, just use that for your transform instead of rotateOrder.  That way the transform is using your own attribute and you're not fighting with trying to do things with the builtin rotateOrder attribute directly.

[jmreinhart]

So that workaround does work, but only if you also have an outgoing connection from the added "internalRotateOrder" attribute. Otherwise the compute method won't be called. I could fix that with a dummy connection but that may not work in parallel evaluation (unless the dummy output affects a visible object). I appreciate the suggestion but in this case I definitely need a clean solution. It's unfortunate that there is really only one MPxTransform example in the devKit.

[zewt]

Well, good luck, but you might just have to settle for a workaround.  MPxTransform has a lot of quirks: off the top of my head it doesn't get along well with constraints (spams bogus cycle warnings) and it doesn't update the matrix properly when duplicated, this is probably just another to add to the list.  Nothing using this API is going to be very clean.

[jmreinhart]

So I've considered a few workarounds and I think I've found one that is not too problematic. If I create a attributeChanged callback in the postConstructor for the node I can use that to update the value on the MPxTransformMatrix when the rotateOrder changes. That should be robust, and shouldn't affect performance. I'm going to test that out as soon as I can.

[zewt]

I don't think that'll be called if it changes due to a connection, only if its value is set directly.

[jmreinhart]

I think kAttributeEval should cover connections, but I'm not sure. But it definitely doesn't cover changes during playback. I'd need another timeChanged callback for that which would cause a major slowdown since we have to update every frame :(.

I definitely think there's info regarding how the mustCallValidateAndSet method works that I'm missing, because similar methods don't affect the attributes on other node types. 

The duplicate issue I think I've gotten sorted by updating the attributes when the node is created in the postConstructor but I need to run it through some more rigorous tests before I can confirm that.

[zewt]

If you want it to work 100% it starts to get hairy: you also need to monitor animCurves for changes (which is a pain since they can be attached far off in the graph, such as through character sets, animation layers and time editor nodes), muting animation layers doesn't trigger change callbacks, probably other things.  Also, monitoring changes won't work with evaluation contexts, which means it won't work correctly with cache if the value can change over time.  It's a pretty deep rabbit hole.

I also tried working around the duplicate problem with postConstructor and copyInternalData, but none of that worked for me.

[jmreinhart]

private:
static void initialSetup();
static MEulerRotation getEulerRotationFromAttrs(MDataBlock& block);
static MEulerRotation::RotationOrder getEulerRotationOrderFromAttrs(MDataBlock& block);
};

So I looked at the MPxTransform.h to see if there was extra info there. The only thing I found were these private methods. I don't see how this would help but it's related to the rotateOrder so I thought I'd bring it up.

[jmreinhart · Accepted solution]

	MTransformationMatrix::RotationOrder rotateOrderMatrix_val = rotationOrder();
	MEulerRotation::RotationOrder rotateOrderEuler_val;
	switch (rotateOrderMatrix_val - 1)
	{
	case 0:
		rotateOrderEuler_val = MEulerRotation::kXYZ; break;
	case 1:
		rotateOrderEuler_val = MEulerRotation::kYZX; break;
	case 2:
		rotateOrderEuler_val = MEulerRotation::kZXY; break;
	case 3:
		rotateOrderEuler_val = MEulerRotation::kXZY; break;
	case 4:
		rotateOrderEuler_val = MEulerRotation::kYXZ; break;
	case 5:
		rotateOrderEuler_val = MEulerRotation::kZYX; break;
	default:
		rotateOrderEuler_val = MEulerRotation::kXYZ; break;
	}

This method seems to work reliably. 

I am still very curious about why validateAndSet has such a strange affect when working with rotateOrder and other default  attributes. 

I'm also curious how the rotateOrder gets passed to the MPxTransformationMatrix if not through the validateAndSetValue. 

Even though those secondary questions haven't been answered, I'm gonna mark this as a solution and I may create a new post for those other questions.

And just a note for anyone who's reading this forum in the future, MPxTransformationMatrix::getRotation() does not always return the same values that are in the channel box, because converting to a matrix and back again constrains you to that 180,180,180 range.