World space rotation of MFnTransformation::rotateBy does not behave as expected

[olarn]

I can't get MFnTransformation::rotateBy with MSpace::kWorld flag to rotate object in world space.

Was I doing something wrong? Quite certain MFnTransform was created using dagpath constructor as MSpace documentation suggests.

sample:

1. Create a mesh under a transform group
2. Apply random translation and rotation to both group and mesh object
3. Use the following script with the mesh object

import pymel.core as pm
import pymel.core.datatypes as pmdt
import maya.OpenMaya as om
import math
sel = pm.ls(sl=1)[0]

# Works as expected
sel.rotateBy(pmdt.EulerRotation(0,10,0), "preTransform")
# Works as expected
sel.rotateBy(pmdt.EulerRotation(0,10,0), "object")
# Same behavior with object flag
sel.rotateBy(pmdt.EulerRotation(0,10,0), "world")
# Also same
msell = om.MSelectionList()
msell.add(sel.name())
dagpath = om.MDagPath()
msell.getDagPath(0, dagpath)

mfn = om.MFnTransform(dagpath)
mfn.rotateBy(om.MEulerRotation(0,math.radians(10),0), om.MSpace.kWorld)
# Desired result
cmds.rotate(sel, 0, 10 , 0, relative=True, worldSpace=True)

[zewt · Accepted solution]

I see the same thing.  It looks like the MEulerRotation version of rotateBy is broken with kWorld.  Try the MQuaternion version, that one works.  I'd have thought the MEulerRotation overload would just convert to an MQuaternion, it's weird that they have different behavior.

[olarn]

The quaternion version does works, thank you.

It's unfortunate that the euler version does not work properly because it can be used right out of a pynode.