MR Satellite Hanging at 100%

[steve_patterson]

Hi!

I'm trying to use MR Satellite to use 4 mac minis (recent i7 models) tied to my recent model iMac to render network jobs, but they keep failing on 100% of the first frame! It's killing me! I've had flaky success (it might get 4 frames completed if entered via the command line as opposed to Batch Render via the GUI...although the number of successful frame seems random), but it still ends up hanging indefinitely. I've tired limiting the memory via the command line, but no such luck. Using BSP 2 as the accelerator. Tried using Large BSP, but it didn't seem to make a difference. Here's an except from the log from whence the hang is occuring:

DB 0.2 debug: job execution for tag 0x1000e54 type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e55 type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e56 type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e57 type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e58 type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e5a type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e5b type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e5d type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e5e type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e5f type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e60 type 68 yields nothing, trying network.
DB 0.2 debug: job execution for tag 0x1000e61 type 68 yields nothing, trying network.
RC 0.2 info : rendering statistics
RC 0.2 info : type number per eye ray
RC 0.2 info : eye rays 4685310 1.00
RC 0.2 info : reflection rays 7642985 1.63
RC 0.2 info : environment rays 5717599 1.22
RC 0.2 info : probe rays 27751448 5.92
RC 0.2 info : fg points interpolated 4357764 0.93
RC 0.2 info : on average 88.31 finalgather points used per interpolation
JOB 0.2 debug: memory increased by 2136 to 1573009574
DB 0.2 debug: entering delete delayed tag 0x1000139
DB 0.2 debug: entering delete delayed tag 0x100013a
DB 0.2 debug: entering delete delayed tag 0x100013b
DB 0.2 debug: entering delete delayed tag 0x100013c
DB 0.2 debug: entering delete delayed tag 0x100013d
DB 0.2 debug: entering delete delayed tag 0x100013e


It looks like this "yields nothing, trying network", or,"delayed tag" is somehow related, but I'm not sure of how to fix this! Please tell me there's a way to make this all work!

The minis are networked via a gigabit ethernet switch and all machines have read/write access to the drive (a thunderbolt drive connect to the master machine...the iMac). Since I'm getting successful frame renders, this makes me think that I need to somehow wrangle this all together by limiters.

Running most recent OSX on all machines (10.8.4), and running Maya 2012 on the master, and MR Satellite 3.9.1 on all slaves.

Please help! Google has yielded no worthy responses.

THANK YOU!

[Anonymous]

What if you render another scene, will it work better or always fail ?
Is always the same MR satellite failing or do they all fail ?
And if you render on just 1 MR satellite, will it improve ?
Sorry, no solution for you, but just troubleshooting steps that may help putting the finger on the real issue.

[steve_patterson]

Hi EarthHobbit!

Thanks for the suggestions. My mileage varies on different scenes. Some seem to luck up fast, and others make it 2000 frames before the hang occurs.

Do you think it may be because I'm using 2012? Is it possible that 2014 and the upgrades to Satellite will help resolve the issue?

It seems to me like the probelm is either packet I/O releated, or memory related. The minis all have 4GB of ram, but the master machine (iMac) has 24GB of RAM (should be plenty...). 

Thanks!

[steve_patterson]

Also, the minis (slaves) are not reporting any errors on their end. It seem to be upon wititng the rendered frame to disk that the problem occurs (100%). 

[steve_patterson]

Hey Gang,

SO, I believe I've narrowed down the culprit to being related to the BSP2 accelerator. If I set the scene to render with Regular BSP, and lower the depth size to 20 (from default of 40), the scene network-rendered fine via command line.

Reliance on Regular BSP isn't ideal, as render times will skyrocket. Does anyone know if improvements in the stability of BSP2 have been made in more recent versions of Mental Ray for Maya? I'm using the latest patched version of Maya 2012 and the latest version of MRSatellite for the 2012 version of Maya.

Thanks,
-Steve