Matching sampling between GPU and CPU

[am_wilkins]

Hi there,

Resurrecting some Arnold GPU render testing again, has anyone managed to figure out the exact ratio for sampling required between CPU and GPU so that the noise amount is the same or at least similar? Double, triple?

For example:

CPU: 8_AA_Min and 12 AA_Max (threshold 0.015)

=

GPU: 8_AA_Min and 36_AA_Max (threshold 0.015)

I've been trying to get the above settings to match for example, but the GPU result appears more noisy even at 36_AA_Max + but then seems to cap out and not improve the results further. So I slight difference in threshold settings might be needed too.

All the best,

Andrew

[thiago.ize]

The adaptive sampling should give you the same quality provided the adaptive sampler is not hitting up against the min/max AA limits. If CPU looks less noisy, it might be that it's doing the min 8 AA for the cleaner pixels when really it could have gotten away with say AA=6. I suggest normally setting min AA to 3 or 4. On the other hand GPU might look more noisy because your max AA should be higher (maybe 50 or 70?).

There's no perfect formula for comparing GPU and CPU noise since it depends on the types of shaders used. Glass, for instance, might require many more samples on GPU than CPU to resolve. But a rough guideline is to take your biggest secondary sample setting (diffuse, specular, etc.) that you use on the CPU and multiply that into the GPU's AA sample setting.

[am_wilkins]

Hi Thiago,

Okay thanks very much for the info.

I've actually barely used the Adaptive Sampler over the years mainly due to increased overall render time and RAM usage.

The way I've always thought about it, is that the "Min" AA samples should at least be the minimum required for a scene. Generally speaking—on the CPU mode—we render shots with a lot of hair, fur, grass, sharp spec edges (buzzing), motion blur etc. so can rarely get away with anything less than around 8AA samples frequently needing to boost to 12AA+

This is interesting however that there is no exact ratio and needs to be tweaked.

I'll keeping testing and try even Lower/Higher Min/Max AA values as per your suggestion.

All the best,

Andrew

[thiago.ize]

The adaptive sampler should have min AA set low (3 is normally fine) and max AA high enough that it will clean up the most stubborn noise in your image. Don't try to guess what the average AA should be and set that as your min AA: that's Arnold's job. Instead you use the threshold to control the quality (how many samples are actually used in each pixel). Try it and I bet you'll find you get faster renders.

[am_wilkins]

Thanks for the info.

In some render testing I'm doing atm (a simple Cornell Box scene) I do find that simply using the Adaptive sampler is much more noisy (and higher RAM usage) for CPU rendering.

Quick example/extract, I've done a slight sharpen in Nuke to help show the noise difference.

Arnold CPU

8AA and 3 on the sub-sampling - 35.62 secs

3.29GB

Arnold CPU - Adaptive Sampler

Min AA  3, Max AA 12 and 0.01 Threshold - 31.18 secs

4.15GB

We can see we're pretty close in render time while getting a significantly more noisy result with the adaptive sampler.

It seems like its not great at cleaning up flat surfaces.

If I increase the Max AA or decrease the Threshold to even 0.009 - then the render time is the same or higher while the result is generally more noisy. (or decrease Max AA and decrease Threshold)

Just my observations so far, of course we only have the Adaptive Sampler available for GPU rendering (which is part of my testing too)

All the best,

Andrew

[thiago.ize]

To do a fair comparison with your -as 8 render, you'd want a -as 3 -asmax 8 setting, keep the subsamples the same, and then keep lowering the threshold until the quality matches. I tried 0.009 for the threshold and got what to my eyes looks to be similar noise and the adaptive results in a faster render (25s adaptive vs 35s fixed). I did this on CPU and did not adjust the sub-sample setting, so maybe that's why I got better results? If you find the noise to be worse, you can keep lowering the threshold until you hit 0, at which point adaptive is effectively disabled and you're just doing Max AA everywhere.

The other issue with your test is that if the entire frame is essentially identical, then there's not much for adaptive to do since all pixels need the same number of samples and you might as well just manually type that in. Adaptive will give you a big speedup when some pixels need few samples to be noise free and other pixels need many.

Where adaptive gets even better is once you found a threshold that is a good balance of noise and speed, you can increase the asmax, say to 30, and render time will only increase in the regions of your image that are overly noisy. I've also attached a -as 3 -asmax 30 -set options.AA_adaptive_threshold 0.009 render that takes 41s but looks much more converged since this was a high enough max AA  that it was likely able to handle the noise in all pixels. If I do a render on GPU with the same threshold and a high enough max AA, the two should end up looking identically noisy.

[am_wilkins]

Hi again,

Just to try bring closure to this thread, I tested settings
Min: 3, Max: 12 and Threshold: 0.001

Between CPU (default) and CPU (adaptive), while not entirely the same - the noise was very similar and I was satisfied with that.
The GPU required at least double +2-3 Max AA before it got a similar result to the CPU. (Around 26 AA)

However increasing the complexity of the testing, we discovered that we need at least Min AA samples of 8+ on the GPU engine in order to start removing zero pixel values—as per this thread:
https://forums.autodesk.com/t5/arnold-gpu-forum/arnold-gpu-nan-zero-value-pixels/td-p/11948733

The good news is these extra Min AA samples didn't increase the render time.

When moving to these more complex test (Gold, Glass, Skin, Volume, etc. shaders) unfortunately there was only a small seconds difference in render time between CPU default and CPU adaptive. Sometimes longer on the adaptive sampler and sometimes shorter.
I haven't found the adaptive sampler to be faster in the testing so far.

Example settings:
Default: 12AA and 2 on the sub-samples
Adaptive 3 Min, 12 Max and Threshold 0.001

Examples times
Stanford Dragon / Cornell Box
Gold Shader:
CPU default (127s) VS CPU adaptive (125s)
Glass Shader:
CPU default (199s) VS CPU adaptive (202s)
SSS Shader:
CPU default (242s) VS CPU adaptive (247s)

Volume Shader: (VDB file)
CPU default (1422s) VS CPU adaptive (1491s)

The visual noise results are the same.

Best,

Amwilkins

[thiago.ize]

It sounds like the adaptive sampler was doing almost 12 AA in every pixel, so it's not going to be faster than your 12AA non-adaptive render as they are both doing about the same amount of work. Your threshold of 0.001 is very low so it's no surprise almost all the pixels used 12AA.  For comparison, the default threshold is 0.015.

Maybe what will make this more obvious is to try and render converged images. Set the non-adaptive to AA=60. Set the adaptive to min AA=3 and max AA=60 and adjust the threshold until they look the same to you. Since you very likely won't be able to tell the difference between a AA=30 or AA=60 render (give it a quick look, don't pixel peep) the adaptive sampler will be able to automatically use a lower AA without you seeing the difference and you'll get a speedup.

Once you have the largest threshold possible that looks good to you (maybe 0.01?), you can reuse these adaptive settings with entirely different scenes and probably still get the same quality. Now try and repeat this quality without adaptive sampling in these different scenes and you'll find that each one requires a different AA which is time consuming to find.

So the benefit of adaptive here is that it makes it easier since you don't need to worry about AA settings.

The second benefit of adaptive is that if your scene has some pixels that converge quickly and others that don't, the adaptive sampler can give massive speedups. For instance, imagine you have a 4K image but there's a 256x256 pixel region where you need AA=30 to get a good result but the rest of the image AA=10 is fine...

[am_wilkins]

Thanks for the reply!
All very interesting information, I would love to hone in better sampling solutions and see this theory in practice.

What strikes me from seeing Max AA=60 mentioned (I understand it's just to demonstrate), as that past testing with Adaptive Sampler indicated that RAM usage was much higher. (I think usually cause max AA was set higher) Arnold seemed to instantly spike that overhead.

In most complex production scenes, we can't set AA higher than around 28-30 otherwise the RAM usage will spike too high and we'd run out of memory on our 128GB machines.

I will continue testing, taking into account your suggestions and see if anything works out for me.

EDIT:

I'm actually seeing little difference visually with Threshold=0.01 compared to Threshold=0.001 with a big difference in render time. Not sure why I opting for 0.001 to being with. Might have been me trying to get rid of specular reflection and diffuse (corners of the box) noise assuming Threshold would solve it.
continuing on...

All the best,
Amwilkins

PS.
in these different scenes and you'll find that each one requires a different AA which is time consuming to find.
Haha, I'm extremely used to this.

[thiago.ize]

Yes, the high memory usage with high AA is something we've been improving over the years and there's still more we want to do. However, I think on the GPU there might not be the same overhead, so there you can set it high all the time (although I don't have a GPU on this machine to verify).

If you're lowering the threshold to get rid of noise in some parts of the image, the threshold won't help. The issue is that you need higher AA, either min or max depending on what's going on.

There's an aa_inv_density AOV that might help you to figure these things out. It will tell you if you're bumping into either of these limits (see https://help.autodesk.com/view/ARNOL/ENU/?guid=arnold_user_guide_ac_render_settings_ac_adaptive_samp... for details).