Asset Collection and Archive Script

Asset Collection and Archive Script

roy_baker
Contributor Contributor
2,565 Views
11 Replies
Message 1 of 12

Asset Collection and Archive Script

roy_baker
Contributor
Contributor
Hello all,

Does anyone have any suggestion, examples, pointers, etc.. on how to write an Asset Collection and Archive script?

I am in need of a script that will collect all the related assets of a scene, zip them up, transfer to a central server and then unpack. This script will need to have the option of letting the user determin what needs to be collected; scene and or relates assets at a minimum, but a entire list would be cooler. This script will be integrated into our Render Farm pipeline, which is not Backburner by the way.

-Roy.
0 Likes
2,566 Views
11 Replies
Replies (11)
Message 2 of 12

Anonymous
Not applicable
As a reference as to how to collect map files in a scene, check out the most excellent BlurScripts MapCollector.
0 Likes
Message 3 of 12

roy_baker
Contributor
Contributor
Excellent! Thanks Ray!

I downloaded Neil's and tried it out as is and it's a great start!. I have a zip/unzip integration I added to Gavin Greenwalt's script using the Maz and UnMaz commands; which will enhance it for my remote folks. Now I just need to see how it can handle xrefed objects and materials.

Seeing how I used Gavin's script I want to thank him. Thanks! Gavin!

I also found a thread on CGtalk that references the approach by Gavin Greenwalt (which I have enhanced) as well as Neil's.

http://forums.cgsociety.org/showthread.php?f=98&t=527443&page=1&pp=15&highlight=archive

-Roy.
0 Likes
Message 4 of 12

Anonymous
Not applicable
Just to note, I have on my plate a rewrite of MapCollector that I hope to release before the end of the year. Just a heads up, afraid I don't have to confirmed release date. But the new one should do a better job of getting every map.

- Neil
0 Likes
Message 5 of 12

roy_baker
Contributor
Contributor
Thanks Neil,

Have you found that the "deleteItem" form array is buggy? I am finding that the "trim_dups" function in the BSLIB does not always work.

- Roy.
0 Likes
Message 6 of 12

Anonymous
Not applicable
Have you found that the "deleteItem" form array is buggy? I am finding that the "trim_dups" function in the BSLIB does not always work.


Never noticed that myself, but I wouldn't be surpised. Do you have an example file (or example code) on an array that doesn't get properly trimmed?

- Neil
0 Likes
Message 7 of 12

roy_baker
Contributor
Contributor
I have a good scene that has tons of duplicate maps. I added a print statement in the MapCollector script and you will see in the Listener that bitmaps are being copied multiple times. Also, you will notice that it will try to copy some bitmaps that are part of the Architectural material type, but will be unsuccessful. I believe what is happening is that the original location of the bitmap has been moved. The bitmap will still render because its new folder location is part of the External Map Paths for Max. If you go the the shader in the Mat editor and pick the bitmap assignment button if will default to the project folder since the original location doesn't exist.

So I am thinking that the Collector script is reading from the scene where the shader believes the map should be, or just the bitmap name minus its original path since the scene did a virtual lookup via the map paths? I hope I am explaining this right 🙂 I could be way off.

Let me know a good location to send the file and I will shoot it you way. The zip file of the scene and bitmaps is 30 megs.

I have been using your great scripts as a template for a new front-end for our Render Farm. I have replaced the copy with a zip routine and then a copy. From there the farm will unzip so the user workstation can go off-line. Since we are using Qube instead of Backburner I can look and see what assets are not on the central server and send only those. This is great because when non-central server user send a render job recursively, they don't have to collect all their bitmaps and such for each submission.

- Roy.
0 Likes
Message 8 of 12

roy_baker
Contributor
Contributor


So I am thinking that the Collector script is reading from the scene where the shader believes the map should be, or just the bitmap name minus its original path since the scene did a virtual lookup via the map paths? I hope I am explaining this right 🙂 I could be way off.


If the problem is that the bitmap is loaded virtually via the map paths is there a way to look at each map found and evaluate its virtual location? Or will the script need to look at every map and determine if it exist in the defined location? If the file doesn't then the script could read the 3dsmax.ini file for all the Bitmap paths, run a search of them, and if found substitute the new location for the old.

- Roy.
0 Likes
Message 9 of 12

roy_baker
Contributor
Contributor
Woo Hoo!

I got all the kinks out.

For the virtual map path I wrote a small function that uses the mapPaths.getFullFilePath method to ask Max where it found the maps:

fn resolve_paths a =
(
for i = 1 to a.count do
(
a = mapPaths.getFullFilePath (filenameFromPath a.filename)
)
a
)

Note that this function only returns the names and location of the bitmaps and not the entire class objects that were sent in the array.

Instead of using the trim_dups function I wrote a simple function that writes a new array while checking for duplicates:

fn trim_dups_ver2 a = 
(
swap=#()
k = 1
for i = 1 to a.count do
(
second_occurance = false
for j = 1 to swap.count do
(if swap == a do second_occurance = true)

if second_occurance == false and a != "" do
(
swap = a
k = k + 1
)
)
a = swap
a
)


Here are the function calls:

all_bm = resolve_paths all_bm
all_bm = trim_dups_ver2 all_bm


This new code works great to resolve all the virtual map path locations and eliminates trimming problems. Instead of allowing the rest of the MapCollector to perform a copy command I rewote it to use the Maz() mehtod to put all the bitmaps in a zip file and then copy that. I then have the Render Manager do the rest.

- Roy.
0 Likes
Message 10 of 12

Anonymous
Not applicable
Well, looks like you answered your own questions. 🙂 I'll do some poking at this when I get the chance to rewrite the script. If I use any of your code, I'll make sure to put you in the thankyou section of my script.

- Neil
0 Likes
Message 11 of 12

roy_baker
Contributor
Contributor
If it wasn't for your code that I started with I would still be staring at a blank screen . 🙂 I am adding in more code to acquire bitmaps from Mental Ray and V-Ray shaders as well as any FG, Lightcache, GI, etc. type files that we use in the render pipeline. I'll shoot the code your way so you can see if it helps your rewrite of the MapCollector.

- Roy.
0 Likes
Message 12 of 12

PiXeL_MoNKeY
Collaborator
Collaborator
I would say look at the ATSops commands and options. This will give you a fast way to access all assets (maps, xrefs, other external files) as well as remap them from a single core interface. It would mean starting from scratch, but may give you more options in the long run.

-Eric
0 Likes