"Cant open ptex file" Maya 2016

[qvisten]

Hello!
We have stepped into a strange issue that only affects one type of our renderpool. We have tried the same cmd-line on different machines. It works on all of our machine except 8 machines. And they are our most powerfull machines. They have (pretty sure) the exact same setup as our other render machines. They have the exact same version of maya installed.
This issue is driving me crazy and I cant find a good solution for it. Thats why Im trying to see if any bright minds can help me

This is the error we get:

13:22:34 XGen Error: Can't find root of XGen installation. Using temp directoryC:/Users/RenderUser/AppData/Local/Temp/ 

XGen Version: NOT VERSIONED Build Date: Nov 30 2015 12:10:16

13:22:34 XGen Stats(1): Preprocessing of P:/Project/asset/char/K//look/v001/K_K.xgen took 0.300 seconds. 
13:22:34 XGen Stats(1): Setup for XG_RENDER_:K_rig:K|XG_RENDER_:K_rig:K_hair took 0.150 seconds. 
13:22:39 XGen Stats(1): Setup for XG_RENDER_:K_rig:K|XG_RENDER_:K_rig:K_eyebrows took 0.000 seconds. 
13:22:40 XGen Stats(1): Preprocessing of P:/Project/asset/char/B//look/v001/B_B.xgen took 0.321 seconds. 
13:22:40 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_mustasje took 0.024 seconds. 
13:22:40 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_eyebrows took 0.014 seconds. 
13:22:40 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_suit_fur took 0.000 seconds. 
13:23:00 XGen Help 
13:23:00 XGen Help 
13:23:00 XGen Help 
13:23:00 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_hair took 0.000 seconds. 
13:23:00 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_hairs took 0.143 seconds. 
13:23:29 XGen Stats(1): Preprocessing of P:/Project/asset/char/E//look/v001/E_E.xgen took 0.295 seconds. 
13:23:29 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_suit_fur took 0.000 seconds. 
13:23:30 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_hat_fur took 0.000 seconds. 
13:23:30 XGen Error: Failed to find E_hat_fur/groom/orient/p_hat.ptx 
13:23:30 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_hat_fur/groom/orient/p_hat.ptx (Part of path doesn't exist)
${HOME}/xgen/E_hat_fur/groom/orient/p_hat.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_hat_fur/groom/orient/p_hat.ptx (Part of path doesn't exist) 
13:23:30 XGen Error: Failed to find E_hat_fur/groom/bend/p_hat.ptx 
13:23:30 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_hat_fur/groom/bend/p_hat.ptx (Part of path doesn't exist)
${HOME}/xgen/E_hat_fur/groom/bend/p_hat.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_hat_fur/groom/bend/p_hat.ptx (Part of path doesn't exist) 
13:23:31 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_pants_fur took 0.000 seconds. 
13:23:31 XGen Error: Failed to find E_pants_fur/groom/orient/p_trousers.ptx 
13:23:31 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_pants_fur/groom/orient/p_trousers.ptx (Part of path doesn't exist)
${HOME}/xgen/E_pants_fur/groom/orient/p_trousers.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_pants_fur/groom/orient/p_trousers.ptx (Part of path doesn't exist) 
13:23:31 XGen Error: Failed to find E_pants_fur/groom/bend/p_trousers.ptx 
13:23:31 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_pants_fur/groom/bend/p_trousers.ptx (Part of path doesn't exist)
${HOME}/xgen/E_pants_fur/groom/bend/p_trousers.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_pants_fur/groom/bend/p_trousers.ptx (Part of path doesn't exist) 
13:23:32 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_hair took 0.097 seconds. 
13:23:35 XGen Error: Failed to open p:/Project/xgen/collections/E/E_eyebrows/paintmaps/clump/p_head.ptx.
Error: Can't open ptex file: p:/Project/xgen/collections/E/E_eyebrows/paintmaps/clump/p_head.ptx
Too many open files 
13:23:35 XGen Error: Failed to open p:/Project/xgen/collections/E/E_eyebrows/Clumping1/Maps/p_head.ptx.
Error: Can't open ptex file: p:/Project/xgen/collections/E/E_eyebrows/Clumping1/Maps/p_head.ptx
Too many open files 
13:23:35 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_eyebrows took 0.038 seconds. 
13:23:35 XGen Error: Failed to open p:/Project/xgen/collections/E/E_eyebrows/Clumping2/Maps/p_head.ptx.
Error: Can't open ptex file: p:/Project/xgen/collections/E/E_eyebrows/Clumping2/Maps/p_head.ptx
Too many open files 
13:23:35 XGen Error: Failed to open P:/Project/asset/char/E/texture/v001/eye_brow_color.ptx.
Error: Can't open ptex file: P:/Project/asset/char/E/texture/v001/eye_brow_color.ptx
Too many open files 
13:23:35 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_face_fur took 0.000 seconds. 

" Error: Can't open ptex file:" and "Too many open files"
What can be the cause of this?

This is the log of a working render machine:

13:23:00 XGen Error: Can't find root of XGen installation. Using temp directoryC:/Users/RenderUser/AppData/Local/Temp/ 

XGen Version: NOT VERSIONED Build Date: Nov 30 2015 12:10:16

13:23:01 XGen Stats(1): Preprocessing of P:/Project/asset/char/K//look/v001/K_K.xgen took 0.738 seconds. 
13:23:01 XGen Stats(1): Setup for XG_RENDER_:K_rig:K|XG_RENDER_:K_rig:K_hair took 0.147 seconds. 
13:23:07 XGen Stats(1): Setup for XG_RENDER_:K_rig:K|XG_RENDER_:K_rig:K_eyebrows took 0.000 seconds. 
13:23:07 XGen Stats(1): Preprocessing of P:/Project/asset/char/B//look/v001/B_B.xgen took 0.322 seconds. 
13:23:07 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_mustasje took 0.025 seconds. 
13:23:08 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_eyebrows took 0.016 seconds. 
13:23:08 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_suit_fur took 0.000 seconds. 
13:23:31 XGen Help 
13:23:31 XGen Help 
13:23:31 XGen Help 
13:23:31 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_hair took 0.000 seconds. 
13:23:32 XGen Stats(1): Setup for XG_RENDER_:B_rig:B|XG_RENDER_:B_rig:B_hairs took 0.140 seconds. 
13:24:02 XGen Stats(1): Preprocessing of P:/Project/asset/char/E//look/v001/E_E.xgen took 0.611 seconds. 
13:24:02 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_suit_fur took 0.000 seconds. 
13:24:03 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_hat_fur took 0.000 seconds. 
13:24:03 XGen Error: Failed to find E_hat_fur/groom/orient/p_hat.ptx 
13:24:03 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_hat_fur/groom/orient/p_hat.ptx (Part of path doesn't exist)
${HOME}/xgen/E_hat_fur/groom/orient/p_hat.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_hat_fur/groom/orient/p_hat.ptx (Part of path doesn't exist) 
13:24:03 XGen Error: Failed to find E_hat_fur/groom/bend/p_hat.ptx 
13:24:03 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_hat_fur/groom/bend/p_hat.ptx (Part of path doesn't exist)
${HOME}/xgen/E_hat_fur/groom/bend/p_hat.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_hat_fur/groom/bend/p_hat.ptx (Part of path doesn't exist) 
13:24:04 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_pants_fur took 0.000 seconds. 
13:24:04 XGen Error: Failed to find E_pants_fur/groom/orient/p_trousers.ptx 
13:24:04 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_pants_fur/groom/orient/p_trousers.ptx (Part of path doesn't exist)
${HOME}/xgen/E_pants_fur/groom/orient/p_trousers.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_pants_fur/groom/orient/p_trousers.ptx (Part of path doesn't exist) 
13:24:04 XGen Error: Failed to find E_pants_fur/groom/bend/p_trousers.ptx 
13:24:04 XGen Error: Searched path includes:
p:/Project/xgen/collections/E/E_pants_fur/groom/bend/p_trousers.ptx (Part of path doesn't exist)
${HOME}/xgen/E_pants_fur/groom/bend/p_trousers.ptx (Part of path doesn't exist)
${XGEN_LOCATION}presets/E_pants_fur/groom/bend/p_trousers.ptx (Part of path doesn't exist) 
13:24:05 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_hair took 0.088 seconds. 
13:24:08 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_eyebrows took 0.043 seconds. 
13:24:09 XGen Stats(1): Setup for XG_RENDER_:E_rig:E|XG_RENDER_:E_rig:E_face_fur took 0.000 seconds. 

Hope to hear from you!

[Michael_Todd]

It seems like the machine are not able to resolve the paths for the files. I'd look to see if your File Data path was set correctly. By default it uses a relative path to the current Maya project, but you can set an absolute path as well.

You can mix absolute and relative paths and use ; separation. XGen will look to the first location in the list and if it can't find the files there, it will look to the next and so on.

The 'Too Many Open Files" issue may be down to an OS issue where ptex file handles are not being released and then windows throws that error. There is a workaround for it but it involves a piece of software, whose name I don't have to hand right now, that can look for open file handles and close them if they are not in use. This may be down to those 8 machines having more cores than the others, I would guess. 

If I can find the name of the software that can look for the handles, I'll post it here.

[qvisten]

Thank you for the respons!

This is when I ran the render:

and when in idle the handles are 291277

So about 7748 handles.

And when I tried local render on another machine it used 6474 handles. Differs less 1k handles. 

Also did a test:

[qdelobel]

Hi,

I'm having the exact same file path issue with our renderfarm. The server machine is a Linux and all the blades and the project managing computer (master) are on windows. Could that be a NFS / UNC access path conflict ?

On the renderman / tractor changelogs they say that there has been a fix for this specific issue but we have a hard time figuring out what to modify....

https://renderman.pixar.com/resources/current/tractor/tractor_notes.html

"Changes in 2.2 1593580

The "rfm--maya-" environment block in the new stock shared.*.envkeys configuration files now includes additions that allow xgen procedurals to load correctly when batch rendering from Tractor. Copy the "shared.*.envkeys" files to your tractor config directory, or integrate similar changes into your "rfm" handler block if you have customized files."

Does anybody know how to set this up ? The server does have a shared.linux.envkeys but the blades and the master only have a localqueue.windows.envkeys.
I'm not an expert in those kinds of problems so I cannot see what I'm missing here.

Thanks in advance for your help !

[Anonymous]

Hi,

We're having the exact same error during Arnold standalone renders with our renderfarm, especially at server nodes with two or four physical processors, if there is only one, the error doesn't come. We are using Maya 2016.5 ands Arnold 4.2.13.6.

The server machine types where it is happening:
2x Xeon E5-2695 v3 2.3GHz (2 physical processors) + Window 8.1 enterprise, 256GB RAM
and
4x 10 core Xeon E7-4860 (4 physical processors) + Windows Standard Evaluation 2016 Server, 512GB RAM

Error message:

"Error: Can't open ptex file: ...
Too many open files"

It seems, that it his error is not exactly connected with the number of open files, because it happens only, when the .ptx files are reachable for the render node, if it can reach it, than it renders without error (but the XGen hair's look is wrong). We are using currently 90 pieces of .ptx files, so it's not so much.

Any quick answer would be appreciated.

Best,

Attila

[qvisten]

We ended up getting a custom build of the xgen procedural from SolidAngle. Write to their support 🙂

I believe this is a Windows issue and wont happen on linux fex.

Sorry that I cant offer a quick fix for this

[Anonymous]

We've found a working solution:

Switching off the Hyper Threading on the machines in bios helped us to render without this error.

[Anonymous]

We had someone here with similar problems (Renderman for Maya 2018, XGen creating a lot of .ptx files, turning off hyperthreading worked); it was  2 x 8-core-Xeon = 16 cores, hyperthreaded to 32 cores. 

On Windows, the C runtime library stdio (which I'm assuming renderman ultimately is using) has a default limit of 512 open files.  This can be bumped up to as high as 2048 if the program makes a call to _setmaxstdio(), and I believe that this total would be shared between all prman threads for a particular job.  Each prman thread would open a certain number of files, and the more threads running on a job, the greater the likelihood that the total maximum of open files would be reached?  Turning off Hyperthreading would simply halve the number of logical cores available, and so the number of threads prman would spawn, but Hyperthreading itself isn't the problem?

I don't suppose anyone knows how renderman is being compiled for Windows, and whether it is setting the maxstdio higher than the default 512?

[josueselva]

Dear Michael Todd: Hi!

Is there any alternative, or additional, way to try fixing this? I've followed on your suggestion and still getting the same error message:

ERROR MESSAGE

"12:28:31 XGen Error: Can't find root of XGen installation. Using temp directoryC:/Users/Josue/AppData/Local/Temp/

XGen Version: NOT VERSIONED Build Date: Jul 2 2019 17:39:53

12:28:32 XGen Warning(2): No patch with name: XG_RENDER_:groom in description: XG_RENDER_:MyCollection|XG_RENDER_:MyDescription"

Thank you! Sincerely, Josh Selva