Render crash on Linux

[crazybridge]

When we start a render with Maya 2016 on Linux using Muster, the render immediately crashes. We are able to run the render successfully from the command line in sh, however when the same command is run through the Muster render client service, we get the following crash on startup:

Starting "/usr/autodesk/maya2016/bin/maya"

Qt: Session management error: None of the authentication protocols specified are supported

 

terminate called after throwing an instance of 'std::logic_error'

what(): basic_string::_S_construct null not valid

Stack trace:

/lib64/libc.so.6(+0x35650) [0x7fe635b90650]

gsignal

abort

__gnu_cxx::__verbose_terminate_handler()

/lib64/libstdc++.so.6(+0x5e926) [0x7fe636607926]

/lib64/libstdc++.so.6(+0x5e953) [0x7fe636607953]

/lib64/libstdc++.so.6(+0x5eb73) [0x7fe636607b73]

std::__throw_logic_error(char const*)

char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag)

std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)

/usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7330) [0x7fe634a06330]

/usr/autodesk/maya2016/lib/libMC3.so.8(+0x139c7f) [0x7fe634a88c7f]

/usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7f69) [0x7fe634a06f69]

CMLFacade::Initialize(CMLWaypoint*, wchar_t const*, wchar_t const*, wchar_t const*, int, unsigned int, long long, long long, MC3_MODE, wchar_t const*)

TCIPClient::initialize()

TCIPClient::TCIPClient()

TCIPClient::theOne()

TbaseApp::cipReportStartup()

TbaseApp::initGeneral()

/usr/autodesk/maya2016/bin/maya.bin() [0x415a52]

Tapplication::start()

/usr/autodesk/maya2016/bin/maya.bin() [0x40e6ff]

main

__libc_start_main

/usr/autodesk/maya2016/bin/maya.bin() [0x40debd]

 

Segmentation fault (core dumped)

// Maya exited with status 139

 

We've tried this both as a user and as root. Maya 2015 and earlier all work fine. We are running CentOS 7.

[warnold1]

Can you please file a bug report on this issue

Is the actual command line that muster uses available

A workaround until we can reproduce and know what is going on would be to disable the CIP module
export MAYA_DISABLE_CIP=1
Before running Maya

There is a NULL pointer string being used from somewhere. Whether Maya cannot find some resource
Once Maya can be run under muster having the listing of environment variables as seen by Maya would be useful to see how the setup is configured

[MikeWile]

Have you contacted Muster support to see if they have any ideas / fixes?

[crazybridge]

The "export MAYA_DISABLE_CIP=1" workaround works. So we were able to get the render to start up.

We placed the command in the init.d script for the Muster render client (as suggested by Muster support).

[MikeWile]

Wayne was able somewhat reproduce the behaviour by making the $HOME/.autodesk directory read-only:

Is the directory "$HOME/.autodesk" writable? When running the renderer CIP requires the directory to be r/w.

From the list of environment variables I don't see HOME being defined

If I make the ~/.autodesk directory read-only I get a similar error, but not 100% the same.

--------------------------

Can you try setting that variable, creating that directory, if it doesn't exist, and making it r/w? Then take the CIP line out of your render script and try a render? Be interesting to see if that was the problem.

Mike

[murphy]

I have experienced exactly the same problem. I started to test m2016 in our environment and renders on linux are failing. We are using squidnet as our render manager.

I'm including my output to help you investigate the problem. We are running the render using a batch script to set all environments correctly for each maya version we want to use for render. I altered the script to output as much info as possible: Maya2016_arnold.sh

#!/bin/bash
export HOME=/root
export MAYA_LOCATION=/usr/autodesk/maya2016
export MAYA_APP_DIR=/root/maya
export RLM_LICENSE=xxxxx
export ARNOLD_LICENSE_HOST=xxxxxxxxxx
export ARNOLD_LICENSE_PORT=xxxxxxxx
export MAYA_SCRIPT_PATH=/mnt/projects/.sys/maya/customScripts:/mnt/projects/.sys/maya/customScripts2016
export MAYA_MODULE_PATH=/mnt/projects/.sys/maya/modules2016_linux
export MAYA_PLUG_IN_PATH=/usr/autodesk/maya2016/bin/plug-ins:/mnt/projects/.sys/maya/plug-ins2016_linux
export PATH=$PATH:/usr/autodesk/maya2016/bin
export MAYA_RENDER_DESC_PATH=/mnt/projects/.sys/maya/modules2016_linux/_MtoA
export MTOA_EXTENSIONS_PATH=/mnt/projects/.sys/maya/modules2016_linux/_MtoA/extensions:/mnt/projects/.sys/maya/modules2016_linux/_Yeti/plug-ins
export ARNOLD_PROCEDURAL_PATH=/mnt/projects/.sys/maya/modules2016_linux/_Yeti/bin
#export SHAVE_LOCATION=/mnt/projects/.sys/maya/modules2016_linux/_Shave/_maya_folder
export LD_LIBRARY_PATH=/mnt/projects/.sys/maya/modules2016_linux/_Shave/plug-ins:/mnt/projects/.sys/maya/modules2016_linux/_Yeti/bin
export PYTHONPATH=/mnt/projects/.sys/maya/modules2016_linux/_MtoA/scripts:/usr/autodesk/maya2016/scripts:/mnt/projects/.sys/maya/customScripts:/mnt/projects/.sys/
maya/customScripts2016
env
echo -e "Running command:\n/usr/autodesk/maya2016/bin/Render" $@
/usr/autodesk/maya2016/bin/Render $@ 

Render farm will assemble the command like this /opt/squidnet/159977512-4-1878534136.sh

#!/bin/bash
/xxxxxx/Maya2016_arnold.sh -s 4 -e 4 -b  1 -im "test__<RenderPass>" -rd /mnt/projects/3D_TESTY_0000/Maya2016_test/images -ai:lve 1 -a
i:as 8 -r arnold -proj /mnt/projects/3D_TESTY_0000/Maya2016_test  /mnt/projects/3D_TESTY_0000/Maya2016_test/scenes/test_anim.ma

When I run this command in shell, everything works as expected. But running the render through render manager fails:

Application Path        : /xxxxxx/Maya2016_arnold.sh
Application Args        : -s 19 -e 19 -b  1 -im "'test__<RenderPass>'" -rd /mnt/projects/3D_TESTY_0000/Maya2016_test/images -ai:lve 1 -ai:as 8 -r arnold -proj /mn
t/projects/3D_TESTY_0000/Maya2016_test  /mnt/projects/3D_TESTY_0000/Maya2016_test/scenes/test_anim.ma
Application Workdir     : NULL
Frames                  : 19 - 19
Output Directory        : /mnt/projects/3D_TESTY_0000/Maya2016_test/images
Processing Node name    : xxxxxxxxx
Version                 : 2.66P135
Execution Shell script  : /opt/squidnet/tmp//3675261096-19-3298186051.sh
Time                    : Wed Sep  9 17:02:14.551

Application Starting...:
-------------------------------------------------------
ARNOLD_PROCEDURAL_PATH=/mnt/projects/.sys/maya/modules2016_linux/_Yeti/bin
SHELL=/bin/bash
MTOA_EXTENSIONS_PATH=/mnt/projects/.sys/maya/modules2016_linux/_MtoA/extensions:/mnt/projects/.sys/maya/modules2016_linux/_Yeti/plug-ins
MAYA_RENDER_DESC_PATH=/mnt/projects/.sys/maya/modules2016_linux/_MtoA
RLM_LICENSE=xxxxxxxxx
MAYA_SCRIPT_PATH=/mnt/projects/.sys/maya/customScripts:/mnt/projects/.sys/maya/customScripts2016
MAYA_APP_DIR=/root/maya
LD_LIBRARY_PATH=/mnt/projects/.sys/maya/modules2016_linux/_Shave/plug-ins:/mnt/projects/.sys/maya/modules2016_linux/_Yeti/bin
MAYA_PLUG_IN_PATH=/usr/autodesk/maya2016/bin/plug-ins:/mnt/projects/.sys/maya/plug-ins2016_linux
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/autodesk/maya2016/bin
PWD=/opt/squidnet
LANG=en_US.UTF-8
ARNOLD_LICENSE_PORT=xxxxxx
ARNOLD_LICENSE_HOST=xxxxxxxxxx
MAYA_LOCATION=/usr/autodesk/maya2016
SHLVL=5
HOME=/root
PYTHONPATH=/mnt/projects/.sys/maya/modules2016_linux/_MtoA/scripts:/usr/autodesk/maya2016/scripts:/mnt/projects/.sys/maya/customScripts:/mnt/projects/.sys/maya/cu
stomScripts2016
MAYA_MODULE_PATH=/mnt/projects/.sys/maya/modules2016_linux
AURORA=/mnt/projects/
_=/bin/env
Running command:
/usr/autodesk/maya2016/bin/Render -s 19 -e 19 -b 1 -im 'test__<RenderPass>' -rd /mnt/projects/3D_TESTY_0000/Maya2016_test/images -ai:lve 1 -ai:as 8 -r arnold -pro
j /mnt/projects/3D_TESTY_0000/Maya2016_test /mnt/projects/3D_TESTY_0000/Maya2016_test/scenes/test_anim.ma

Starting "/usr/autodesk/maya2016/bin/maya"

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_S_construct null not valid
Stack trace:
  /lib64/libc.so.6(+0x35650) [0x7f54302e0650]
  gsignal
  abort
  __gnu_cxx::__verbose_terminate_handler()
  /lib64/libstdc++.so.6(+0x5e926) [0x7f5430d57926]
  /lib64/libstdc++.so.6(+0x5e953) [0x7f5430d57953]
  /lib64/libstdc++.so.6(+0x5eb73) [0x7f5430d57b73]
  std::__throw_logic_error(char const*)
  char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag)
  std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)
  /usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7330) [0x7f542f145330]
  /usr/autodesk/maya2016/lib/libMC3.so.8(+0x139c7f) [0x7f542f1c7c7f]
  /usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7f69) [0x7f542f145f69]
  CMLFacade::Initialize(CMLWaypoint*, wchar_t const*, wchar_t const*, wchar_t const*, int, unsigned int, long long, long long, MC3_MODE, wchar_t const*)
  TCIPClient::initialize()
  TCIPClient::TCIPClient()
  TCIPClient::theOne()
  TbaseApp::cipReportStartup()
  TbaseApp::initGeneral()
  /usr/autodesk/maya2016/bin/maya.bin() [0x415c62]
  Tapplication::start()
  /usr/autodesk/maya2016/bin/maya.bin() [0x40e76f]
  main
  __libc_start_main
  /usr/autodesk/maya2016/bin/maya.bin() [0x40df2d]

Segmentation fault (core dumped)
// Maya exited with status 139
APPLICATION log ending:
---------------------------------------------------


Jobslice processing statistics:
---------------------------------------------------
Application exit Status:                 139 (0x8B)
Application expected Status:             0 (0x0)
Processing Start time:                   Wed Sep  9 17:02:14 2015
Processing end time:                     Wed Sep  9 17:02:19.165
Application log file limit:              0 bytes
GHZ Hours used:                          0.009458 
CORE Hours used:                         0.009 
Render credits used:                     0.030 
Max CPU Frequency:                       1064 MHz
Application log file size (allocated):   10000 bytes
Total Processing Time:                   04s

setting the CIP env will help just wanted to share my output to help as in my case it is not a problem of r/w home . . . 

thx murphy

[dt11aal]

Just a quick message to say that we are having the same issue with a Tractor based farm running maya 2016 sp2 Centos 7.

I have tried the CIP env variable fix but to no avail, any further ideas?  or is this something that was later fixed in sp3?  I couldn't see it on the release notes.

Many thanks 

Error:

Starting "/usr/autodesk/maya2016/bin/maya"
terminate called after throwing an instance of 'std::logic_error'
what(): basic_string::_S_construct null not valid
Stack trace:
/lib64/libc.so.6(+0x35650) [0x7f70fef9e650]
gsignal
abort
__gnu_cxx::__verbose_terminate_handler()
/lib64/libstdc++.so.6(+0x5e926) [0x7f70ffa15926]
/lib64/libstdc++.so.6(+0x5e953) [0x7f70ffa15953]
/lib64/libstdc++.so.6(+0x5eb73) [0x7f70ffa15b73]
std::__throw_logic_error(char const*)
char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag)
std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)
/usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7330) [0x7f70fde03330]
/usr/autodesk/maya2016/lib/libMC3.so.8(+0x139c7f) [0x7f70fde85c7f]
/usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7f69) [0x7f70fde03f69]
CMLFacade::Initialize(CMLWaypoint*, wchar_t const*, wchar_t const*, wchar_t const*, int, unsigned int, long long, long long, MC3_MODE, wchar_t const*)
TCIPClient::initialize()
TCIPClient::TCIPClient()
TCIPClient::theOne()
TbaseApp::cipReportStartup()
TbaseApp::initGeneral()
/usr/autodesk/maya2016/bin/maya.bin() [0x415c62]
Tapplication::start()
/usr/autodesk/maya2016/bin/maya.bin() [0x40e76f]
main
__libc_start_main
/usr/autodesk/maya2016/bin/maya.bin() [0x40df2d]
// Maya exited with status 139
Segmentation fault (core dumped)
[2015/10/13 16:30:37 exit with non-zero status code rc=139, ["Render", "-r", "default", "-s", "1", "-e", "1", "-b", "1", "-im", "UD_RM_main_101_RF", "-proj", "/mnt/RenderStore/mw10aah/farmTest/urbanDecay/", "/mnt/RenderStore/mw10aah/farmTest/urbanDecay///_UD_RM_main_101_RF_9102.mb"]]

Environment Vars:

export solidangle_LICENSE=xxx@xxx
export ARNOLD_PLUGIN_PATH=/usr/autodesk/solidangle/mtoa/2016/procedurals:/usr/autodesk/solidangle/mtoa/2016/alshaders/bin
export MTOA_TEMPLATES_PATH=/usr/autodesk/solidangle/mtoa/2016/alshaders/ae
export RMSTREE=/opt/pixar/RenderManStudio-20.3-maya2016
export MAYA_DISABLE_CIP=1

[d.tree]

Hi All,

So the fix after all:

in your maya.env file append

MAYA_DISABLE_CIP=1

where I went wrong was to place this in the system Environment Variables which didn't work. very strange indeed.

Thanks for the help.

David

[murphy]

SP4 out and this issue is not mentioned in fixes

Maya_2016_SP4_Readme_enu

Did anyone tried it already on linux?

[vendor]

I've tried, it's still not fixed in SP4, same crash as before. Render manager over here is Qube, OS is CentOS 6.5 (Entirely fresh install of both just to verify that previous installs weren't a factor):

terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_S_construct NULL not valid
Stack trace:
  /lib64/libc.so.6() [0x3bb70326a0]
  gsignal
  abort
  __gnu_cxx::__verbose_terminate_handler()
  /usr/lib64/libstdc++.so.6() [0x3bc20bcbd6]
  /usr/lib64/libstdc++.so.6() [0x3bc20bcc03]
  /usr/lib64/libstdc++.so.6() [0x3bc20bcd22]
  std::__throw_logic_error(char const*)
  /usr/lib64/libstdc++.so.6() [0x3bc209ce59]
  std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)
  /usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7330) [0x7f6c0a667330]
  /usr/autodesk/maya2016/lib/libMC3.so.8(+0x139c7f) [0x7f6c0a6e9c7f]
  /usr/autodesk/maya2016/lib/libMC3.so.8(+0xb7f69) [0x7f6c0a667f69]
  CMLFacade::Initialize(CMLWaypoint*, wchar_t const*, wchar_t const*, wchar_t const*, int, unsigned int, long long, long long, MC3_MODE, wchar_t const*)
  TCIPClient::initialize()
  TCIPClient::TCIPClient()
  TCIPClient::theOne()
  TbaseApp::cipReportStartup()
  TbaseApp::initGeneral()
  /usr/autodesk/maya2016/bin/maya.bin() [0x415d82]
  Tapplication::start()
  /usr/autodesk/maya2016/bin/maya.bin() [0x40e8af]
  main
  __libc_start_main
  /usr/autodesk/maya2016/bin/maya.bin() [0x40e06d]

I unfortunately do not know the commands sent by Qube to open Maya or initiate the render since it builds those with a script upon job submission and doesn't seem to put them in the logs, but I cannot replicate this issue by any other means other than submitting a render through Qube. On the Linux desktop I can launch command line and GUI sessions of Maya without issue. SOMETHING about the way that render managers (Squid, Qube, Muster, Tractor, etc) launch Maya has been broken by Maya 2016 on Linux and Autodesk still has not addressed this show-stopper. The thing is, I've read that Qube just launches Maya via "/usr/local/bin/maya -prompt" and just throws render commands, no different than I can do manually. I really don't know what's different but based on the input from others in this thread it seems to be universal across all render farms.

[warnold1]

the process that is running Maya needs to set  MAYA_DISABLE_CIP=1  before launching maya itself

if Qube does not inherit the user's environment to launch Maya then it needs some other means of setting environment variables.

I don't know Qube so can't help there.

CIP is expecting to have some  access to stdin/stdout

the issue can be reproduce from launching Maya from a desktop icon when there Terminal setting is turned off

[vendor]

Thank you warnold, this is the "duuh" realization that I needed. Qube launches renders under a user called qubeproxy so I just had to modify that user's Maya.env to get Qube to launch Maya. Qube also has the ability to set environmental variables pre-render, so we can do that until Autodesk fixes the CIP issue.

Funny that a stupid "customer involvement" process keeps the actual application from working. Can't say it didn't manage to get the customers involved

[leifpedersen3d]

This is still going on in Maya extension 2 BTW. 

doing:

setenv MAYA_DISABLE_CIP 1

worked for me.

Is a fix planned for 2017?

[jerome.samson]

Hi,

Just bumping the thread: we have the same pb with maya2017 on cent-7.2 using gridengine.

Disabling CIP worked fine for us though.

Cheers,

Jerome

www.freefolk.com

[n.burdan]

Still a problem in maya2018

[JamesDeschenes]

Yeah I can confirm the same issue with Maya 2018 update 4 and using Qube as the render farm manager.