Getting Composite to Network Render using BB

[miketruly]

Sigh... yet another Composite hassle.

My render farm is made up of multiple machines with different vintages which do not have the same hardware configuration.  As such,  on some systems MAX/Composite is installed on C.  On other systems, MAX/Composite may be installed on D.

I have Composite rendering fine using Foreground or Background rendering.

But when sending the Composite job to BB... many of the slaves will fail because they are looking for the <InstallFolder> which was specified by the submitting workstation (which has Composite installed on C) and these slaves have Composite installed on a different drive.  (Yes, I have found the ridiculous requirement in the outdated helpfile which says Composite should be installed on the same drive on all systems).  The unique slave systems need to know where Composite is installed in order to render.

In Deadline, this type of situation is easliy solved because you can specify multiple paths to be searched in order to make Composite run.  So each unique slave will be able to find the Composite app to run.  But I want to use BB to net-render the Composite job.

Is there a way to set the <InstallFolder> to try multiple paths?  The helpfile is outdated because it references <InstallDir> as well as having very little info on this type of situation.

Thanks!

[miketruly]

These variables that are in the BB Render Command setup such as <InstallFolder>, etc.  Where are these things defined?

Seems like there should be a file I can look in and see what the paths are that are assigned to these variables.

Thanks!

[Steve_Curley]

In Users\[your user name]\toxik\[version] are 3 defaults files. Problem is they're XML without any indentation or proper line breaks, so they're virtually unreadable. Maybe there's something in one of those, I don't know for sure. There doesn't appear to be anything useful in the installation folder either 😞

[miketruly]

Thanks for the ideas Steve!  I looked in there previously and did a search for "install" and it can't find anything in those files.

As is, I can successfully net-render via BB on only those machines where Composite is installed on the C drive.  Those which have Composite installed on the D drive error out because the 'workpath' points to C only.  Or I can change it and render on on systems installed on D but not C.  Silly.

Seeing as how MAX and other apps can be installed on whatever drive and work with BB just fine... this Composite limitation is a real pain.

Thanks again.

[Steve_Curley]

Indeed it is. It really shouldn't take too much work to make it far more user friendly where paths are concerned. I know it's EOL but a little work would go a long way towards making it far more usable.

[miketruly]

Now that Composite is free... would be nice if AD gave away the code for free so someone (smarter than me) could modify the code and get Composite to net-render via BB when render slaves have the app installed on different system hard drives.

Thanks!

[Steve_Curley]

Cant see that happening anytime soon. You can always hope though 😉

[miketruly]

A man's gotta have a dream! 

[PiXeL_MoNKeY]

Mike you should be able to fix this on the erroring machines.

Open Composite > Help > Token Substitutions - This tells you the values that the current Composite installs is pulling.

Look for the entries: <ProjectFolder> and <InstallDir>

The <ProjectFolder> path tells you where your toxik.ini (if it exists) is located. If you choose the INI route below it needs to be created here.

The <InstallDir> is the path on the current system that is the issue.

Open Composite > Help > Initialization Variables - These are the environment or INI values you can use.

In there you will see:

toxik::deployment::installDir

Env: DL_INSTALL_DIR

You have 2 choices set the value in the INI:

toxik::deployment::installDir D:/<some path>

or Create a Windows Environment variable set to the path:

DL_INSTALL_DIR D:\<some path>

Hope that helps,

-Eric

[miketruly]

Eric, thanks very much for the ideas!

I have been following your instructions but I'm a bit confused. 

Am I supposed to edit some variable on the workstation or on the slaves which have the install on D, etc.?

The workstation (and a bunch of the slaves) have the install on C.  But other slaves have the install on D.  Unless multiple paths can be specified and searched, (which is no problem for Deadline as it let's you specify multiple paths) then it must be something to edit on each slave who's install is other than C so all the C installs will work.

The only toxik.ini file has only 3 lines in it.  If I try to add the installDir D:/etc. to the ini, then Composite won't start on the workstation system (because of it's C install).  If I take that line back out, it starts fine.

Thanks again.

[PiXeL_MoNKeY]

As you see on your main workstation the toxik.ini and the toxik::deployment::installDir does work. That needs to be set on all machines that have it installed on the 😧 drive with a path to the installation. The other big note is that because the Project Folder is typically in the user folder you will need to make sure the file goes into the backburner service user account, or whatever account is running the rendering process.

-Eric

[miketruly]

Unfortunately, this is still not working.

I can put the <InstallDir> variable in the toxik.ini file (only when \ are used in the path... if / are used then Composite crashes on startup) but it makes no difference.

The problem is that the workstation system always injects it's specific <InstallDir> path into the command before submitting the job to BB.  Since this system has Composite installed on C... all the other slaves which have Composite installed on D throw errors in BB - 'could not set the workpath, no Composite installed on C, etc.'

What is needed is to have the job submitted to BB without the workpath specified and let each slave use it's own unique <InstallDir> location.

Thanks!

[miketruly]

Here are some screenshots showing what happens when the Render Actions are submitted to BB.

Thanks!

[PiXeL_MoNKeY]

See the help file here:

"If you specify the command line argument, the argument takes priority over the environment variable and the initialization variable. If you specify the environment variable, then it takes priority over the initialization variable."

Setting the parameter in the toxik.ini should override the values. Have you tried it on one of the failing machines? The Enviroment Variable approach is good as it isn't user specific, but those values are also overridden by the toxik.ini values, if they exist.

-Eric

[PiXeL_MoNKeY]

Okay I see what you mean now. From the help file here:

"For network rendering:

1. Ensure Composite is installed in the same path on every workstation that is part of the render farm, for example: C:\Program Files\Autodesk\Autodesk Composite 2011\Composite\
2. Ensure the rendering application is installed on every workstation that is part of the render farm. Note If you are using the Autodesk Backburner network rendering application, ensure that one Backburner Manager is running on one workstation and one Backburner Server is running on each workstation that is part of the render farm. In addition, each Backburner Server should be connected to the Backburner Manager."

What is your setup on your Workstation? Do you have a D drive? What if you set up a common dos "subst" to remap virtually all systems to the same drive? "subst C: Z:\" on machines with C: installation, "subst 😧 Z:\" on machines with 😧 installations. This will set a virtual drive of Z: on all machines, and if Z: is set as the drive of your installdir on the workstation you should be in better shape.

-Eric

[miketruly · Accepted solution]

Eric,

Each of the slaves did not have a toxik.ini file down in the User folder like the workstation did.  On each slave, I created a toxik.ini file with only the installDir path set pointing to the proper install path.  The slaves are still erroring as the workpath that is specified in the BB submit from the workstation forces the wrong install path on the slaves.

If I try to remove the workpath part of the submit, the submit errors and will not submit.  So the question is, how would I NOT specify the command line argument for workpath?

Thanks!

[miketruly]

Thank you Eric.

Using the DOS subst command on each system allows all the slaves to render the Composite comp in BB.  Each slave has the X drive mapped to the local Composite install folder and X is specified in the workpath in the Composite Render Actions for submitting to BB.

Thanks again.

[PiXeL_MoNKeY]

Great, glad we got it working finally.

NOTE: This solution is not permanent so you may need to create a start script in case there is a system reboot. To ensure that every time the system logs in it has it setup.

-Eric

[miketruly]

Thanks again Eric!  The BAT files are already in place to create the mapping upon startup.