Logs - TextureResolveUseExistingTx

[am_wilkins]

Hello,

I'm seeing some very slow render performance in IPR
machine:

2 x AMD Ryzen Threadripper 3990X 64-Core Processor  (64 cores, 128 logical) with 130944MB

I've noticed from the logs that a large percentage is being used on TextureResolveUseExistingTx
Never seen this behavior in our MtoA days.

00:01:24  1273MB         | top session self-times by category 
00:01:24  1273MB         |  TextureResolveUseExistingTx           0:32.02 (42.45%) 
00:01:24  1273MB         |  thread blocked                        0:31.31 (41.51%) 

Alongside this getting the "bound to single thread" warning:

00:01:03  1173MB         | performance warnings:
00:01:03  1173MB WARNING |  Rendering CPU utilization was only 4%. Your render may be bound by a single threaded process or I/O.

Every texture has an existing TX file. Any ideas why handling the TX files appears to be be very slow?

render_log.png
---
Houdini Core Version 19.0.383
HtoA 6.0.1.0
Arnold 7.0.0.0

Thanks,
Andrew

[Stephen.Blair]

Always hard to tell for sure from snippets...

Arnold Core now handles "Use Existing Tx"
Looks like Use Existing TX Textures is off in HtoA. So HtoA exports paths to the png/tif/jpg/exr files, and Arnold has to resolve those (check if a tx file exists for each png/tif/jpg/exr)

[am_wilkins]

Hey Stephen!

I can confirm "Use Existing TX Textures" is enabled. (it's our bread and butter!)

Is there another part of the log I can get for you?

I'm not sure how Core VS HtoA might relate for us here, but it does feel to me like on each IPR pass or potentially for each thread Arnold is checking for the TX files?

The way the threads behave is strange, while in IPR rendering only a few threads are ever at 100% at a time until near the end of the IPR pass then only all 128 threads max out for a moment before we're back to 2-3 of them struggling away.

io_statistics.png

Andrew

[am_wilkins]

I can "Diagnostic Ignore" almost everything (Subdivisions, displacement, SSS, bump, etc.) and the render time doesn't improve.

Disabling "textures" however and we're back to lightning speed. I know this doesn't prove too much as textures generally are pretty heavy.

But I expect the Arnold cache load (including TX files) and then render each IPR pass at max CPU usage until the frame is completed. Not spent 80%+ of the time using only 3-5 threads of 128.

[Stephen.Blair]

Arnold itself will check for tx files...if the plugin gives it non-tx file paths.

The TextureResolveUseExistingTx suggests that's what is happening here. I've never seen that in a log file before, so this is a first. Can you send the full log to support @ arnoldrender dot com?

[thiago.ize]

I hate ruining surprises, but I suspect we've already fixed this and it should appear in our next major release (look for ARNOLD-11689). Unfortunately I can't say when this will be publicly available and of course it's possible this is an unrelated bug compared to what you're hitting and the bug will still be present.

[am_wilkins]

Sure, thank you!
A first for me too

I was thinking of trying to "hard-path" the image nodes to the TX and see if that makes a difference, is this a worthwhile test?

[am_wilkins]

The behavior is definitely that after destroying the Arnold cache each IPR "iterative pass" up til the last takes significantly longer.

Not just the first IPR pass which I'm used to take longer as more textures / other data enter the cache but each iterative pass after that until it's completed the frame is slow

Only then after changing a lights parameter for example, I get the performance I'm expecting.

Example, from zero cache (roughly timed)
1st IPR pass: 50s
2nd: 1m 15s
3rd: 1m 36s
4th: 1m 54s
5th: 2m 16s - frame completion

Then once frame is completed after rotating a light, from 1st IPR to final frame completion the character renders in 14secs.

(of course a basic empty scene and a few lights for testing)

[am_wilkins]

Hey Thiago,

Ok thank you for the info, would be interested to know if this is the solve—as you say could be unrelated as things go, haha.

However, I'm doubtful we'd be able to update to a later version as productions are underway on our end.

[am_wilkins]

"hard-path" the image nodes to the TX

Ok, I just tried this and it solved the issue!!

[am_wilkins]

Hi again Thiago,

With regards to a solution I found that solves the issue on my end:
"hard-path" the image nodes directly to the TX

This however is not ideal as it of course throws a spanner into our pipeline. Now we have to handle TX files instead of the in-built feature in Arnold.

Does this solution make it clearer and is what you suspect being the cause solved in this new Arnold version "ARNOLD-11689" ?

[thiago.ize]

That's right, it's the OIIO texture searchpath code that was causing this issue. The fix is related to that.