MtoA gpu problem with rendering on my 2 gtx cards

Anonymous — Mon, 20 Apr 2020 21:58:41 GMT

Hi,

I have problem with rendering on my 2 gtx cards gtx 1070 8GB and 1070 Ti 8GB
I create simple scene nothing fancy sphere and plane plus dome light , I switch arnold to gpu render .When I press render a strange thing is happening , my computer freezes for a few minutes and then starts rendering very slowly only on one card .
I will add that if I manually select one of the cards, everything works well and arnold renders the full power of the selected card. Please help me .

Hardware info :
linux Fedora 30 kernel 5.3.15-200.fc30.x86_64
Nvidia driver 440.82
Maya 2019.2
Mtoa 4.0.2.1

and here is log on this render :

00:00:00 1515MB | Arnold 6.0.2.1 [088e32fd] linux clang-5.0.0 oiio-2.1.4 osl-1.11.0 vdb-4.0.0 clm-1.1.1.118 rlm-12.4.2 optix-6.5.0 2020/03/06 10:17:06
00:00:00 1515MB | host application: MtoA 4.0.2.1 49490427 (master) Maya 2019
00:00:00 1515MB | 1 x AMD Ryzen Threadripper 1950X 16-Core Processor (16 cores, 32 logical) with 64245MB
00:00:00 1515MB | NVIDIA driver version 440.82 (Optix 60600)
00:00:00 1515MB | GPU 0: GeForce GTX 1070 Ti @ 1683MHz (compute 6.1) with 8119MB (8107MB available) (NVLink:0)
00:00:00 1515MB | GPU 1: GeForce GTX 1070 @ 1784MHz (compute 6.1) with 8119MB (7271MB available) (NVLink:0)
00:00:00 1515MB | Fedora 30 (Thirty), Linux kernel 5.3.15-200.fc30.x86_64
00:00:00 1515MB | soft limit for open files raised from 524286 to 524286
00:00:00 1515MB |
00:00:00 1515MB | loading plugins from /opt/fcg/ext/mtoa-4.x-2019-linux-testing/shaders ...
00:00:00 1515MB | xgenSpline_shaders.so: XGenHairMapping uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkFloatConstant uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkFloatComposite uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkFloatCondition uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkFloatCorrect uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkFloatLogic uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkFloatMask uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkFloatMath uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkColorComposite uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkColorCondition uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkColorConstant uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkColorCorrect uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkColorLogic uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkColorMask uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkColorMath uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkChannels uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkPremultiply uses Arnold 6.0.2.1
00:00:00 1515MB | lookdevkit_shaders.so: ldkSimplexNoise uses Arnold 6.0.2.1
00:00:00 1515MB | hairPhysicalShader_shaders.so: HairPhysicalShaderDiffuseColorAdapter uses Arnold 6.0.2.1
00:00:00 1515MB | hairPhysicalShader_shaders.so: HairPhysicalShaderSpecularWeightAdapter uses Arnold 6.0.2.1
00:00:00 1515MB | hairPhysicalShader_shaders.so: HairPhysicalShaderSpecularGlossAdapter uses Arnold 6.0.2.1
00:00:00 1515MB | hairPhysicalShader_shaders.so: HairPhysicalShaderTransmissionWeightAdapter uses Arnold 6.0.2.1
00:00:00 1515MB | hairPhysicalShader_shaders.so: HairPhysicalShaderTransmissionSpreadAdapter uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaMultiplyDivide uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaClamp uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaGammaCorrect uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaCondition uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaReverse uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaBlendColors uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSamplerInfo1D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSamplerInfo2D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaPlusMinusAverage1D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaPlusMinusAverage2D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaPlusMinusAverage3D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaRemapValueToValue uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaRemapValueToColor uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaRemapColor uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaFile uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaPlace2DTexture uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaRamp uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaProjection uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaChecker uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaBulge uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaCloth uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaGrid uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaFractal uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaNoise uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaPlace3DTexture uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaRgbToHsv uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaHsvToRgb uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaLuminance uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaCloud uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaCloudAlpha uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSnow uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaContrast uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaLayeredTexture uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSolidFractal uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaVolumeNoise uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaBrownian uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaStucco uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaRemapHsv uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSetRange uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaImagePlane uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSurfaceShader uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaColorToFloat uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaVectorToFloat uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaPointToFloat uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaPoint2ToFloat uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaWriteColor uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAovWriteColor uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaWriteFloat uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAovWriteFloat uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaEnvSphere uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAnimFloat uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAnimPoint uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAnimVector uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAnimColor uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAnimMatrix uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaUserDataVector uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaUserDataColor uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaShadingEngine uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSamplerInfo3D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAovWriteVector uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaAovWriteVector2 uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaNormalDisplacement uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaVectorDisplacement uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaUserDataBool uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaHair uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaMeshLightMaterial uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaUserDataVec2 uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaBump2D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaFluid uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaMarble uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaMountain uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaWater uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaCrater uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaLeather uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaGranite uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaRock uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSingleShadingSwitch uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaDoubleShadingSwitch uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaTripleShadingSwitch uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaQuadShadingSwitch uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaFluidData uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaFluidTexture2D uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaSurfaceLuminance uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MtoaCameraUvMapper uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_shaders.so: MayaFlatClosure uses Arnold 6.0.2.1
00:00:00 1515MB | [metadata] loading metadata file: /opt/fcg/ext/mtoa-4.x-2019-linux-testing/shaders/mtoa_shaders.mtd
00:00:00 1515MB | renderSetup_shaders.so: applyAbsFloatOverride uses Arnold 6.0.2.1
00:00:00 1515MB | renderSetup_shaders.so: applyAbs2FloatsOverride uses Arnold 6.0.2.1
00:00:00 1515MB | renderSetup_shaders.so: applyAbs3FloatsOverride uses Arnold 6.0.2.1
00:00:00 1515MB | renderSetup_shaders.so: applyAbsIntOverride uses Arnold 6.0.2.1
00:00:00 1515MB | renderSetup_shaders.so: applyRelFloatOverride uses Arnold 6.0.2.1
00:00:00 1515MB | renderSetup_shaders.so: applyRel2FloatsOverride uses Arnold 6.0.2.1
00:00:00 1515MB | renderSetup_shaders.so: applyRel3FloatsOverride uses Arnold 6.0.2.1
00:00:00 1515MB | renderSetup_shaders.so: applyRelIntOverride uses Arnold 6.0.2.1
00:00:00 1515MB | loading plugins from /opt/fcg/ext/mtoa-4.x-2019-linux-testing/procedurals ...
00:00:00 1515MB | mtoa_ParticleInstancer_proc.so: particleInstancer uses Arnold 6.0.2.1
00:00:00 1515MB | xgenSpline_procedural.so: xgenProcedural uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_ParticleVolume_proc.so: volume_particle uses Arnold 6.0.2.1
00:00:00 1515MB | mtoa_ParticleVolume_proc.so: implicit_particle uses Arnold 6.0.2.1
00:00:00 1515MB | xgen_procedural.so: xgen_procedural uses Arnold 6.0.2.1
00:00:00 1515MB | loaded 121 plugins from 9 lib(s) in 0:00.00
00:00:00 1515MB | loading plugins from /opt/fcg/ext/mtoa-4.x-2019-linux-testing/plug-ins/../bin/../plugins ...
00:00:00 1515MB | synColor_shaders.so: color_manager_syncolor uses Arnold 6.0.2.1
00:00:00 1515MB | cryptomatte.so: cryptomatte uses Arnold 6.0.2.1
00:00:00 1515MB | cryptomatte.so: cryptomatte_filter uses Arnold 6.0.2.1
00:00:00 1515MB | cryptomatte.so: cryptomatte_manifest_driver uses Arnold 6.0.2.1
00:00:00 1515MB | usd_proc.so: usd uses Arnold 6.0.2.1
00:00:00 1515MB | alembic_proc.so: alembic uses Arnold 6.0.2.1
00:00:00 1515MB | loaded 6 plugins from 4 lib(s) in 0:00.00
00:00:00 1515MB | [metadata] loading metadata file: /opt/fcg/ext/mtoa-4.x-2019-linux-testing/plug-ins/mtoa.mtd
00:00:00 1515MB |
00:00:00 1527MB | [gpu] setting OptiX disk cache location to: /var/tmp/OptixCache/arnold-6.0.2.1_driver-440.82
00:00:00 1777MB | GPU 0: GeForce GTX 1070 Ti enabled
00:00:00 1777MB | GPU 1: GeForce GTX 1070 enabled
00:00:00 1777MB | [color_manager] using color manager of type "color_manager_syncolor"
00:00:00 1777MB | [color_manager_syncolor] synColor version 2018.0.80
00:00:00 1777MB | [color_manager_syncolor] native catolog directory: /usr/autodesk/maya2018/synColor
00:00:00 1777MB | [color_manager_syncolor] optional custom catalog directory: /home/s/maya/synColor/Shared/
00:00:00 1777MB | [color_manager] rendering color space is "scene-linear Rec 709/sRGB" with declared chromaticities:
00:00:00 1777MB | r(0.6400, 0.3300) g(0.3000, 0.6000) b(0.1500, 0.0600) and w(0.3127, 0.3290)
00:00:00 1799MB |
00:00:00 1799MB | there are 1 light and 3 objects:
00:00:00 1799MB | 1 persp_camera
00:00:00 1799MB | 1 skydome_light
00:00:00 1799MB | 1 utility
00:00:00 1799MB | 1 lambert
00:00:00 1799MB | 1 driver_exr
00:00:00 1799MB | 1 gaussian_filter
00:00:00 1799MB | 2 polymesh
00:00:00 1799MB | 1 list_aggregate
00:00:00 1799MB | 1 color_manager_syncolor
00:00:00 1799MB | 1 renderview_display
00:00:00 1799MB |
00:00:00 1799MB | rendering image at 960 x 540, 3 AA samples
00:00:00 1799MB | AA samples max <disabled>
00:00:00 1799MB | AA sample clamp <disabled>
00:00:00 1799MB | diffuse samples 2 / depth 1
00:00:00 1799MB | specular samples 2 / depth 1
00:00:00 1799MB | transmission samples 2 / depth 8
00:00:00 1799MB | volume indirect <disabled by depth>
00:00:00 1799MB | total depth 10
00:00:00 1799MB | bssrdf samples 2
00:00:00 1799MB | transparency depth 10
00:00:00 1799MB | initializing 12 nodes ...
00:00:00 1799MB | creating root object list ...
00:00:00 1799MB | node initialization done in 0:00.00 (multithreaded)
00:00:00 1799MB | updating 13 nodes ...
00:00:00 1799MB | /aiSkyDomeLight1/aiSkyDomeLightShape1: skydome_light using 1 sample, 2 volume samples
00:00:00 1815MB | scene bounds: (-5.69700384 -0.22149843 -5.69700384) -> (5.69700384 1.77850151 5.69700384)
00:00:00 1815MB | node update done in 0:00.02 (multithreaded)
00:00:00 1815MB | [gpu] preparing 13 nodes ...
00:00:00 1815MB | [gpu] preparing shapes for intersection ...
00:00:00 1815MB | [gpu] preparing scene for gpu ...
00:00:01 1877MB | [gpu] updating 13 nodes ...
00:00:01 1888MB | [gpu] GPU update done in 0:00.361
00:00:01 1888MB | [aov] parsing 2 output statements ...
00:00:01 1888MB | [aov] registered driver: "defaultArnoldDisplayDriver@renderview_display" (renderview_display)
00:00:01 1888MB | [aov] * "RGBA" of type RGBA filtered by "defaultArnoldFilter@gaussian_filter" (gaussian_filter)
00:00:01 1888MB | [aov] registered driver: "defaultArnoldDriver@driver_exr.RGBA" (driver_exr)
00:00:01 1888MB | [aov] * "RGBA" of type RGBA filtered by "defaultArnoldFilter@gaussian_filter" (gaussian_filter)
00:00:01 1888MB | [aov] done preparing 2 AOVs for 2 outputs to 2 drivers (0 deep AOVs)
00:00:01 1915MB | starting render ...
00:00:01 1918MB | [gpu] building scene and compiling shaders ...
00:00:02 2139MB | Computed stack size : direct 2736 continuation 11200
00:00:02 2139MB | [gpu] building scene and compiling shaders done in 0:00.410
00:00:02 2139MB | 0% done (0/9 samples per pixel)

Re: MtoA gpu problem with rendering on my 2 gtx cards

Stephen.Blair — Tue, 21 Apr 2020 12:54:28 GMT

It looks like you have two different cards: a 1070 and a 1070 Ti

That might explain the problem

How much slower is it with both cards? Can you post the final render times for both scenarios?

thanks

Re: MtoA gpu problem with rendering on my 2 gtx cards

Anonymous — Tue, 21 Apr 2020 22:46:25 GMT

I didn't know they had to be the same two cards to work.
Both cards are Gigabyte and even the same series:
GeForce® GTX 1070 Ti Gaming 8G
GeForce® GTX 1070 G1 Gaming 8G

Here is test on MtoA 4.0.3
Now the latest version MtoA is different when I turn on render both cards, rendering begins and nothing happens except that 1070Ti works on 100% but in renderview nothing happens 😞
I attach logs on render gtx1070 , gtx1070Ti and on both cards gtx1070_gtx1070Ti ( but the render on both cards stopped after 5 minutes because nothing happened )
arnoldGpu_gtx1070.txt
arnoldGpu_gtx1070Ti.txt
arnoldGpu_gtx1070Ti_gtx1070.txt

I will add that I tested these cards on Redshift and Blender Cycles and everything works well (both cards render together)

Re: MtoA gpu problem with rendering on my 2 gtx cards

thiago.ize — Wed, 22 Apr 2020 15:58:07 GMT

Mixing rtx with non-rtx cards is not allowed, but otherwise I think mixing GPUs should work.

Definitely something wrong is happening here, but I'm not sure yet what it could be. I don't recall hearing any other reports from users experiencing this problem in recent versions of Arnold, which makes this all the more perplexing.

Re: MtoA gpu problem with rendering on my 2 gtx cards

thiago.ize — Wed, 22 Apr 2020 16:42:10 GMT

For the combined case, if you had let it render for an hour or so, would anything have been shown in the log or in the image? We're curious if this is a hang or if it's just really slow.

What happens if you make the rendered image smaller? What if you take it all the way down to 1x1 pixel? Will that complete within 5 minutes?

Finally, can you try enabling more debug messages? Try launching maya with the following environment variable and report back the log output:

ARNOLD_GPU_DEBUG_USAGE_REPORT=3  path_to_/maya

Re: MtoA gpu problem with rendering on my 2 gtx cards

Anonymous — Fri, 24 Apr 2020 16:47:29 GMT

Everything started to work when i did update kernel .
Sorry for the confusion 🙂
Everything works well on kernel 5.5.16-100.fc30.x86_64 and Nvidia driver 440.82

thanks for help 🙂

Re: MtoA gpu problem with rendering on my 2 gtx cards

thiago.ize — Fri, 24 Apr 2020 17:50:03 GMT

That's great!

Would you mind leaving this as an official "answer" and marking your answer as the accepted answer so that this can help other people that come across this same issue?

topic Re: MtoA gpu problem with rendering on my 2 gtx cards in Arnold GPU Forum

MtoA gpu problem with rendering on my 2 gtx cards

Re: MtoA gpu problem with rendering on my 2 gtx cards

Re: MtoA gpu problem with rendering on my 2 gtx cards

Re: MtoA gpu problem with rendering on my 2 gtx cards

Re: MtoA gpu problem with rendering on my 2 gtx cards

Re: MtoA gpu problem with rendering on my 2 gtx cards

Re: MtoA gpu problem with rendering on my 2 gtx cards