Arnold GPU - NaN / Zero value pixels

[am_wilkins]

Good day,

In recent render testing, I've noticed that the Arnold GPU engine is generating zero value pixels / NaN's compared to the CPU engine.

Is this a known issue and anyway to prevent this from happening?

There is just one "Area Light" in the scene, nothing camera side. So there are some dark values, but the Arnold CPU engine doesn't appear to have as much of a problem with this.

Overall render example:

Arnold CPU - Higher exposure (Nuke)

Arnold GPU - Higher exposure (Nuke)

You could be able to see large patches of pure black values.

Additionally, I sampled a pixel in the center of the dragon, which shows the different between CPU above and GPU below.

All the best,

Amwilkins

[thiago.ize]

I see the black pixels, but what makes you suspect there's a nan? Nans should trigger options.error_color_bad_pixel which by default is blue, not black. If you made it black, can you make it blue again just so it's more obvious this is a nan and not some other issue?

On the GPU, what is the camera AA set to? If you increase the AA, do these black pixels go away, or are there more of them?

[am_wilkins]

Hi Thiago,

Apologies—"NaN" was probably the technically incorrect term, but yes I mean pure black zero values which appear all over the Arnold GPU render in patches.

The render settings are:
AA_Min: 3, AA_Max: 26, Threshold: 0.001

I haven't noted additional AA samples changing the result, but will do some more testing and let you know.

All the best,

Amwilkins

[thiago.ize]

What happens if you raise the AA_min? Just for testing purposes, try something large, like AA_min=6 to see if that helps. What can sometimes happen is if all the AA_min samples at that pixel and the surrounding pixels have no noise, then the adaptive sampler will assume it's supposed to be like that and stop sampling.

[am_wilkins]

Sure, here's another test with settings:

GPU Engine

AA_Min: 6, AA_Max: 40, Threshold: 0.001

Before: (previous post settings)

After: (above settings-higher min and max AA)

Looks like your theory is correct...the result is much better, not yet perfect though.

Since I changed the Min and Max AA I'm not sure which improved the result more but will test that tomorrow and may post more findings.

Thank you.

Amwilkins

[am_wilkins]

Hi a gain,

Just confirming that it was indeed the Min AA samples which needed to be higher. (Max AA made no difference)

Raising the Min AA from the initial 3 to around 10 was required to remove majority of the zero pixel value patches—however there were still some pixels remaining even at a Min AA of 10.

Min AA 10

Amwilkins

[thiago.ize]

Thanks for confirming. Since this metal dragon is reflecting a pitch black background, it's not surprising it's going to have regions where most reflections only see the black background and the adaptive sampler assumes that is the expected result. I just want to caution you that this is not a GPU specific result. This finding is scene specific. You might find some CPU scenes require min AA of 8 and other GPU scenes can get by with min AA of 2. I suspect CPU is getting by with a lower min AA because you're using diffuse and specular samples of more than 1 on CPU, while GPU is fixed to 1?

[am_wilkins]

No problem.

Yeah I'd like to do another round of tests with an additional dome light or something to reflect camera side, but I wanted to keep my test conditions very lean and clinical.

I suspect CPU is getting by with a lower min AA because you're using diffuse and specular samples of more than 1 on CPU, while GPU is fixed to 1?
Ok cool, I wasn't sure if GPU sampler was set to 1 or 2 on the sub-samples. Since in Houdini at least, the sub-samples stay visible (greyed out) and set to the default of 2. It's not very obvious they'd be set to 1 by default.

What strikes me as interesting, is that across Arnold, Karma, Vray & Redshift testing—only Arnold appears to have this problem in this scene.

Thanks,

Amwilkins

[niscatmi]

Howdy!
Any recent developments on this front? I have a simple car scene that exhibits this exact issue. The scene is lit by an HDRi skydome. In some reflective parts I see black spots when rendering on GPU, but no issues for CPU. Notice the headlight.
For what is worth, my machine specs are - CPU: Ryzen 5960x; GPU: 2 x RTX3090; 128 Gb RAM

[niscatmi]

Hi!

I have this issue as well, in a simple scene lit by an HDRi skydome light.
Any recent news about this?

[am_wilkins]

Hi niscatmi,

Sorry for the late reply.

Unfortunately I ran out of time to continue testing on this front, however I didn't find a solution at the end of the day. Other than trying to minimize the issue with extra min AA samples or light sources.