Error: [gpu] Arnold stack overflow exception encountered during render

[fureniku]

Hello,

I'm a bit of a rendering/Maya/Arnold noob so please bare with me! I'm trying to render my assignment for university, and I'm getting the error in the title. This only happens when I try to render with my RTX 3080 (10gb) GPU. If I render with my i9 10900k (64gb system RAM) it does slowly but surely render the frame.

I'm guessing the stack overflow means the frame simply requires more RAM than my graphics card can give it, but I find that a little hard to believe... especially as task manager only shows Maya using 2.6gb RAM total.

Am I perhaps doing something wrong? Rendering was working fine until I added a few new objects and changed to 1080p for my output from whatever the default was, but I've tried most settings and the GPU won't work on any. I imagine my GPU would be quite a bit faster for this than my CPU so any help to get it running would be greatly appreciated 🙂

[thiago.ize]

Assuming you're using the latest Arnold (arnold core 6.1.0.0 as of now), this is probably a known issue due to the GPU using too small an amount of memory per thread (per-thread stack size). This isn't related to scene size but to the amount of temporary data Arnold needs while computing the color of a pixel. We think we might have a solution for this, but we won't know for sure until it's released (sorry, can't give an ETA or guarantee this is what is affecting you).

In the meantime, you might be able to work around this by manually specifying the stack size using the ARNOLD_GPU_DEBUG_STACK_SIZE environment variable. I would start at setting it at 4000 and keep increasing it in increments of 1000 until it works. The log, especially in debug verbosity might also give some hints on what stack size to start at. How to use environment variables will depend on the OS and possibly the plugin you're using. If you were using kick on linux it'd be a simple: ARNOLD_GPU_DEBUG_STACK_SIZE=5000 kick your_scene.ass. You'll know it worked if you look at the debug output and see a message like "[gpu] changing gpu stack size $ARNOLD_GPU_DEBUG_STACK_SIZE: 5000".

[fureniku]

Thanks for the insight - I'm running Windows with MtoA, but I've had experience with environment values in other programs so I'll have a look.

Failing that, do you know if an older version might not have this bug on? I'm not sure where I'd see the version but it's a relatively safe guess I'd be on latest as I only installed it last week. I wouldn't want to rely on waiting for an update regardless of ETAs as I have a deadline for the assignment.

[thiago.ize]

There's various ways for this incorrect stack size to come about, so I don't know if an older version would have this fixed. Likewise, it could be that a newer released version has this already fixed. Unless you downloaded mtoa from the arnold website, the one that comes with maya is usually outdated.

Instructions to find version: https://knowledge.autodesk.com/support/arnold/troubleshooting/caas/sfdcarticles/sfdcarticles/How-to-...

[fureniku]

Thanks, I checked and I am indeed on 6.1.0.0

After changing the environment value, the initial error did indeed stop, but was replaced with a very similar error about OptiX stack overflow:
"Error: [gpu] OptiX stack overflow exception encountered during render (32400 threads threw an exception) //"

As I said, I'm not very experienced at all with these things but I believe I read somewhere that OptiX isn't really supposed to be used for final frame renders? I honestly can't tell if this is another related issue, or just user error.

[thiago.ize]

Did you keep increasing this to see if it stopped that error? Maybe to make it easier, instead of adding 1000 each time you can try doubling the size. I believe arnold defaults to 13,664 so for sure you want larger than that. If you hit a million, you can give up (with doubling, that shouldn't take too many tries).

GPU Arnold can be used for final renders. What you heard has more to do with not all CPU features being available on GPU (we're working on it), so as long as you don't depend on that, you'd be fine.

[fureniku]

Doubling the size eventually changes the error again from about 16,000:
Error: [gpu] Arnold stack overflow [...]

I can continue up to about 128,000 where I get a different error entirely (which is too big for the limited comment box), suggesting somewhere between 64,000 and 128,000 is my maximum limit? I'm not entirely sure what these numbers represent but does that sound right for a very modern high end GPU?

[thiago.ize]

Thanks for trying that out. Since you used a big enough stack and it still failed, it looks like it's not this particular stack that is being exceeded. There are other stacks in gpu arnold, so I suspect it is one of those. I'll reply in another answer with another possibility.

[thiago.ize]

Since it's not the ARNOLD_GPU_DEBUG_STACK_SIZE that helps, it probably means that it's some other gpu code. Do you happen to be using refractive objects, such as glass? How about volumes? If so, does temporarily removing those fix things? If that helped, does enabling/disabling the dielectric_priorities help (see https://docs.arnoldrenderer.com/display/A5AFMUG/Transmission#Transmission-DielectricPriority)?

Another possibility is that you have complex shading (perhaps lots of layer shaders?) which is exceeding the fixed stack size used for these processes.

For sure we need to make Arnold more robust to these, and it is an active area of work by us, but hopefully this might give you some ideas for how to work around it in the meantime (simplify shaders, reduce amount of dielectrics and volumes, set lower depths, etc.)

[thiago.ize]

We just released Arnold 6.1.0.1 which contains the below bug fix which I hope might solve your problem. Could you give it a try and let us know how it goes?

core#10060 [GPU] Improve handling of stack overflow exceptions

[fureniku]

Apologies for not responding previously, the forum seemed to just not let me press submit at all... This new version seems to have resolved my problem though! Typically it was just as I finished doing a CPU render split across three devices, but I'm hoping the GPU one might have less noise so I'll leave it running overnight anyway and see how it comes out.

Thanks for the fast fix!