Arnold USD with Maya

[riltis]

Hi all,

I have successfully pulled down the arnold-usd repo from GitHub and have been able to compile and build the Visual Studio project on Windows, but I am unsure what to do with the files that I have compiled in order to get Maya to recognize and load them. In specific, I am looking to update this file and it's corresponding header file and then load it into Maya 2022:

https://github.com/Autodesk/arnold-usd/blob/master/translator/reader/reader.cpp

I have built the translator portion of the repo and I see a 'translator.lib' file, but I'm not sure how to get Maya to use it. Also, is there a way to package up these files for a deployment across a network?

I'm using:
Maya 2022.3
Arnold v6.2.1.1
USD: v0.20.8

Thanks for the help.

[Stephen.Blair]

Arnold USD is packaged as part of the MtoA plugin.
I'm curious why you have to build it yourself, we'd like to avoid forcing users to do that...you're using some different version of something?

[riltis]

Hi Stephen,

We were just trying to get a fix out for a bug you guys are working regarding the random results with instanced USD objects. We have a project in production right now and just need a fix out ASAP.

Thanks for the help!

[sebastien.blaineau.ortegaHRM3U]

Hi Ryan, are you using the mayaUSD proxy shape or the standin to load the usd files ? (the way it needs to be built and packaged depends on that)

In any case, I need to warn you that apart from the complexity to build everything properly, you'd be jumping several major versions, the current head of the arnold-usd repo is being used and tested against an Arnold 7.x that hasn't even been released yet. If you're in production, such a huge jump could introduce many more issues.

That being said, what you need to do is to build the procedural, following the build instructions in https://github.com/Autodesk/arnold-usd/blob/master/docs/building.md . You can't build just the translator portion of the repo, you need to go at the root and build it as described. Basically you need the build variable BUILD_PROCEDURAL enabled and you can turn off all the other BUILD_x variables that are for the render delegate, etc... If you're building it for mayaUSD then you also need to set a variable USD_PROCEDURAL_NAME='usd_cache' .Then you'll get either a usd_proc.dll or a usd_cache.dll that you need to copy to the right location, depending on whether this is for the standin or mayaUSD proxy shape.

On the other hand, there's a chance we could help you workaround the problem if changing the usd scenes is a viable option. This randomness was caused by a very specific use case, where an instance points "directly" to a file reference. If it was pointing to e.g. a Xform node in the scene, which has a reference just below its hierarchy, then this randomness should disappear. So if you're ok to share example scenes with us, feel free to contact us at support@arnoldrenderer.com and maybe we can find a way to tweak the scenes in a way that doesn't cause the issue. I would personally think it would be much better if you're in production, but of course it's your choice.

Cheers

[riltis]

@Sebastien Blaineau-Ortega

We are using the mayaUSD proxy shape. We are looking at the workaround solve as well as the code fix. For the code fix, I have added the USD_PROCEDURAL_NAME argument to my CMake build, and now the build generates a file called 'usd_cache_proc.dll'. I know you mentioned the location where this file goes is different for Arnold standin or mayaUSD proxy shape, where should I put this .dll?

Thanks!

[Stephen.Blair]

Depends on what version of Maya USD
Under "extensions\usd" there's folders like 2105, 2108, 2111
usd_cache_proc goes in one of those

[riltis]

We're using Maya USD v0.13.0.

[Stephen.Blair]

Ok, so then in the 2108 folder because I was wrong, it's the USD version that is in the name of the folder (and I also see you listed USD 21.08 in your question).

[riltis]

What if there is no 2108 folder? What does MtoA fall back to?

[riltis]

Correct me if I'm wrong, but it seems like if Arnold does not find the correct corresponding USD version in the "extensions\usd" folder, then it falls back to the "plugins\usd_proc.dll".

[riltis]

@Sebastien Blaineau-Ortega @stephen

Noting this here, but I did try updating to MtoA 5.0.0.3 just as a quick test, and I am unable to render any prims that are marked as USD Instanceable in Maya. This doesn't seem to be an issue before MtoA 5.x.x.x. I created an issue on the arnold-usd GitHub regarding this.
Still using MayaUSD v0.13.0 and Maya 2022.3.

https://github.com/Autodesk/arnold-usd/issues/1033

Thanks!

[riltis]

@Sebastien Blaineau-Ortega,

I'm not sure if I interpreted your workaround correctly, but I am still seeing the issue happen when I apply your workaround fix. I interpreted your solve:
```
If it was pointing to e.g. a Xform node in the scene, which has a reference just below its hierarchy, then this randomness should disappear
```
as meaning:

1. Add an Xform to the stage. This will serve as a "prototype" of sorts for the instances. Ensure that instanceable is OFF for this prim.
2. Beneath the new Xform you just created, add another Xform. Ensure that instanceable is OFF for this prim.
3. On this second, child Xform that was created in step 2, add a file reference to the geometry you are using as a prototype. This should point to the actual file on disk.
4. On each instance, remove all references that currently exist, and add an internal reference to the Xform created in step 2. Also, ensure that instanceable is ON for this prim.
5. Render the scene.

Attached, I have an example scene that I used to test your solve as well. Let me know if it works on your end with my software versions.

instanceable_dev_fix.zip

Also here are some screenshots.

Thank you!

[sebastien.blaineau.ortegaHRM3U]

I'm sorry, I assumed USD would behave differently with an intermediate Xform node but I was wrong. I also still have the same issue with the workflow I had in mind.

The issue here, is that when we ask USD what is the "prototype" of an instanceable prim, instead of giving us its actual path in the scene, it will return us a new prim called /__Prototype1, or /__Prototype2, etc... and the index is not guaranteed to be constant (possibly depends on the order of evaluation). This is what causes the randomness.

I'm afraid there's no workaround 😞 You'll really need the fix https://github.com/Autodesk/arnold-usd/pull/1025 that was just merged and that will be shipped in a future release

[riltis]

@Sebastien Blaineau-Ortega,

Thank you for the clarification on that. Do you know if this pull request will make it into the Arnold 7.1.0.0 release or would it be the following release?

Thanks!

[sebastien.blaineau.ortegaHRM3U]

We can't comment on future version numbers, but it's planned to be in the next feature release yes

[riltis]

@Sebastien Blaineau-Ortega,

Gotcha. Thanks for the info. In that case, would you be able to provide more information on the thread above regarding building the arnold-usd repo? We are using mayaUSD proxy shapes and I have generated the ‘usd_cache_proc.dll’ and but I am a bit confused. I am able to actually completely delete the “extensions/usd” folder that holds the different ‘usd_cache_proc.dll’ files for each USD release, and for some reason Arnold is still able to render USD scenes inside of Maya. Doesn’t Arnold rely on these .dll files to correctly interpret USD data into the renderer?

Thanks!

[sebastien.blaineau.ortegaHRM3U]

The usd_cache_proc nodes allow to render the same stage than MayaUSD in Arnold. It's needed for example to support edits in MayaUSD (moving geo, etc...) and still see the result in Arnold. This is why it needs to be built against the same USD cut than mayaUSD.

If no usd_cache_proc is found, then MtoA will render this as a basic usd procedural coming from Arnold (usd_proc.dll). This procedural has its own version of USD and it loads a usd filename. It won't update properly if you do live edits in MayaUSD since it's directly looking up the usd file.

[riltis]

Hi @Sebastien Blaineau-Ortega,

Thank you so much for the explanation of those two .dll files. This information was enough to help me get the repo built and into Maya 2022 successfully. I was also able to get your change to fix the random results when using the Instanceable checkbox into a build.
As far as what my issues where, there were two things.

1. I had accidentally checked out the v20.08 tag of USD when compiling USD, rather than v21.08. When Arnold had tried to load the usd_cache_proc.dll I had generated at render time, it errored saying something like "The specified procedure could not be found". I was unable to get more info from Maya when this error occurred, but after double checking all of my builds, I noticed this issue.
2. I needed to compile USD with Visual Studio 2017 instead of 2019. This is because of the Boost python library dependency in arnold-usd. This boost python library has the name of the C++ binary it's compatible with in the name, for example "boost_python37-vc141-mt-x64-1_70.lib". When I tried to build with Visual Studio 2019, I would get a boost library called "boost_python37-vc142-mt-x64-1_70.lib". Maya, in 2022, (inside of the mayaUSD/lib folder) has the "boost_python37-vc141-mt-x64-1_70.lib" boost library, so I needed to build my USD repo in the same way so it would be compatible with Maya.
As far as this issue, it was also difficult to decipher in Maya, as I was getting an error like "The specified module could not be found". I used a program called Dependencies to help be look at what .dlls the core dll required, and this is how I noticed the filename difference in the boost library.


We are still having issues at our studio with these large datasets right now, but I will end this thread here and create new ones for our new issues.