Arnold Inconsistant and unpredictable Render Times

[Florian_Senand]

Hello guys.

I contact you because I use Arnold on C4D since several years now, and I often note very inconsistant and unpredictable render times that make me waste weeks and weeks.

I managed to make several companies (where I often workas a freelancer) to buy some licenses and new machines, and even to orient the all pipeline toward Arnold to share seamlessly project between C4D and 3DSMax. But the matter still the same, whatever the computer, whatever the software, at a certain point of a film production, we had unsolvable increasing render times.

Everything always began correctly. I build my shaders, compare the quality and the render times while I go forward in the project, always trying to keep reasonable computing duration using rayswitch ... And one day, I wake-up, open my project, and render times explode without any reason. This is a bit cartoonish but not so far from the truth.

For exemple, on this project, I made all textures in Mari and rebuild the sader in Arnold. This is one unique shader, only the framing change. One evening, I was at 15 to 30 min for theses frames. The next day morning, I corrected some textures in Mari, exported and reloaded them in Arnold, and relaunch renders. Result : 18min to 3h52 depending on the framing. If you take a look to the 1 and 2, the body fit the same proportion of the picture, only the angle change. However, one angle take 32min, the other 1h48 ! There in no sense ...

An other Exemple. On this project, I have made a first design with minerals. Achieve acceptable render time has been tricky becaused of displace, complex refraction and Ao that allow a metallic shader to propagate over quartz and stones, to blend them evenly. At the middle of the film production, we had to change this design by another, apparently more simple.Only two shaders with a bump, somme SSS, and a bit refractive for one of the them. Result : Same Render Times (if I use Ray switch)

I can continu with another old scene where changing texture size from 4k to 2k divided the render times by 10. But on the same scene, changing 2k by 1k, re-increase render times. Oposite exemple.

On a scene I had 4K substance textures. I opened them in photoshop to make a 2K version, and instead of decrease, render times increased. The same exr output directly in 2k from substance allowed me to reduce times. Last exemple.

I had a scene with rocks, stairs and a tree. All of these are HD geometry with 4k to 8K textures. In solo mode, each one of these elements take above 10 minutes to compute, which that is incredibly fast regarding of there complexity. When they are all acivated in the render, the scene take several hours to render.

Why a so big difference ?

I'm curious to get yours advice on these phoenomes because I really don't know what to do anymore.

[ACCCC]

Generally you can reduce rendertimes by

- Using arnold's build in shader and map nodes rather than the legacy nodes of your 3d application

- Pre-processing your textures with -> Maketx

- Optimizing noise and samples via AOV's -> Remove Noise

- Using the Arnold or Optix Denoiser

[Stephen.Blair]

You need to get Arnold logs to understand where the time is spent. And if you get profile logs too, then we can look at those as well.

Textures are loaded into a cache with a hard maximum size, so you shouldn't have to worry about texture size. You are using tx, right?

[Stephen.Blair]

One object versus a whole scene? There's going to be a lot more rays bouncing around.

[thiago.ize]

As Stephen said, logs, making sure you've enable the info/stats verbosity is crucial to understanding what is going on. Another useful tool is the cpu render time AOV. Looking at that might indicate where the time is going. For instance, maybe you'll see that there's a particular part of the model that is expensive, which explains why moving the camera makes it slower.

[Florian_Senand]

Thanks for your replies.

-On C4D, only Arnold nodes work, so I use only native ones.

-I obviously use .tx, but this scene as only one map. Shaders are only build from one noise, use for the bump and to drive roughness. By the way, I have made lot of render time tests in the past, and .tx are not always faster. Sometime, a good exr or targa make the job faster.

-I always denoise in post process with "Neat Video", so my aim is always to get an acceptable noise that I can remove after.

-Concerning the size of the textures, I assure you that size matter. I observe it on each project that I made these 3 last years. I only work on 64 ou 92Go RAM machines, so this is not a limitation.

-I assume that one object vs three is obviously faster to compute. But my point is the difference of computing time wich is really huge. Jump from 10minutes to several hours is like a big kicking for a 3D artist with time and money constraints !

-About the render time AOV, I don't I don't understand how does it work. It always give me a full white or black image. Now, the only thing I never think to do (except when I get errors) is to check the log in attachment :

arnold.txt

[thiago.ize]

Can you post the log for the fast render and then the slow render so we can see what changed? At the very least, from what you posted it doesn't appear so far to be related to texture speeds, though for future reference, if you ever do have lots of textures being used, it could be that your texture cache needs to be made bigger (default is 2GB).

[thiago.ize]

cputime AOVs needs to be written out as a float since it usually includes times that are either very small or way above 1. In an image editor you can adjust the exposure to see things properly. Another option is to attach a heatmap filter to the cputime AOV. If you can run it from kick, this is done automatically; you can interactively see this using the instructions in https://arnoldsupport.com/2017/11/15/kicking-a-cputime-heat-map/ . If you can reproduce these slowdowns running with fewer camera samples (maybe even AA=1), then the kick approach, in an interactive setting might be easiest.

[Stephen.Blair]

It doesn't look texture related, or are you using textures to drive opacity or transmission?

- top self-times are polymesh:intersect and BVH:intersect
- 25% of rays are shadow and volume_shadow, rest are bssrdf
- lots of transparent_shadow and volume shader calls

If the file size matters, do you have a second Arnold log we can compare with? (a log for different sized textures).

[Florian_Senand]

---

Ok, I follow your advise and recompute an old project whith Log and Heatmap (Cpu time pass).

This is exactly the project that I leave in July, the same Arnold version, the same machine, only C4D running (no background tasks) but this time, render times are completely different !

The only thing that has change, is windows which update several times since this summer.

I attach Logs bellow.carole-aafinal0000.txtcarole-aafinal0001.txtcarole-aafinal0002.txt

[Florian_Senand]

-Yes, I use the same texture to drive the displace and the opacity, but the opacity one pass through a ramp which randomize and clamp the opacity in lines.

-Ok, I see these line in the log.

polymesh:intersect mean that some polygones intesections lead to high render time I suppose ?

What is BVH:intersect

[Florian_Senand]

Concerning the file size of the textures, I don't have any log because I never use log before 🙂

I need to re-open an old project and remake some renders with 2K and 4K to Illustrate my purpose.

[ramon.montoya]

One thing that jumps out at me is that your textures are not in .tx format. They should be tiled and mipped for maximum efficiency. From the 3rd log it looks like 85% of time is spent trying to getting texture handles, that might include time spent mipping your textures.

So next thing I would try is using .tx files, They always help, specially if you have 55 high res unmipped maps. Try it and you'll see!

[Florian_Senand]

Thanks Ramon.

I tried it lot of time, but I always work in exr and tiff, and .tx never give me really better efficiency.

On the contrary, I often had gamma issies with .tx in the past.

But I will try, again, in case of 🙂

[thiago.ize]

Well, the 1. and 2. renders are almost identical times, so that's fine. The 3. render is much slower, yes, but the log shows it's because of your textures being jpgs:

05:28:08  7044MB         |     File I/O time : 4d 22h 27m 46.7s (14m 14.6s average per thread, for 499 threads)
05:28:08  7044MB         |     redundant reads: 530042 tiles, 2218.0 GB
05:28:08  7044MB         |     Peak cache memory : 2.0 GB

Please try again with tx files.

[thiago.ize]

gamma issues are because the proper color space was not used. If you come across gamma issues again, create a new arnold answers question and we'll help you out with that.

[Florian_Senand]

Well.

I converted all my textures in .Tx and has I claimed it lot of time here, it changes ... Nothing ...

Renders are always so long.

4K Textures (Reference Render) :

CAROLE-AAFinal.0000.txt

CAROLE-AAFinal.0002.txt

4K Textures converted in .Tx :

CAROLE-AAFinal-Tx.0000.txt

CAROLE-AAFinal-Tx.0002.txt

[Florian_Senand]

Worste, I decrease progressively the size of the textures and the gain of rendering time is not so big and overall depend of the camera view.

2K .Tx :

CAROLE-AAFinal-Tx-2K.0000.txt

CAROLE-AAFinal-Tx-2K.0002.txt

1K .Tx :

CAROLE-AAFinal-Tx-1K.0000.txt

CAROLE-AAFinal-Tx-1K.0002.txt

[Florian_Senand]

I tried 8 K to complet the test (that will be the size of the final model), but after 10 hours, Arnold computed nothing.

8K .Tx :

CAROLE-AAFinal-Tx-8K.0000.txt

So, I began to test others render engines and decided to use a CPU one.

And Surprise, the same picture with 8K textures compute in only few minutes !

So, again, textures are not the matter, and render time still very random, like I said in the subject of this thread

[thiago.ize]

None of these logs are indicating that .tx files were used or at least that they were fully used. You need to use only .tx files, otherwise the non-tx files will still end up dominating. Here's what I see in carole-aafinal-tx0000.txt:

Top files by bytes read:

1 3984.0 MB (14.5%) 4096x4096x3.u8 F:\CAROLE\TEX\CAROLE-Specular.1001.tif UNTILED UNMIPPED MIP-COUNT[77,4,3,2,1,1,2,52,1,1,1,1,3]

Notice how it's a tif file and worse, it's untiled and unmipped -- that is almost guaranteed to make the renderer slower.

I think we need to figure out why .tx files aren't getting used.