How to render Alphas/Masks for objects occluding each other?

[Anonymous]

Hey guys,

i have a scene with objects occluding each other. And i need to render them in separate passes because of render performance issues when trying to render them all at once.

Problem is: Asteroids flying around in this scene (lots of them) are occluding a single asteroid in the middle. So i have asteroids in front of the big one; it always changes from frame to frame. But i need to mask them out, otherwise i cannot comp them over.

Afaik a Matte Material should do; as i thought at least. Tried it out, works so far so good, but: A lot of times the matte Material renders object occlusion where there is none. So it renders a hole into the big asteroid but no small asteroid is actually there in the render of that pass.

How do you render alphas/masks for object behind and in front of each other? Is the Matte really the solution? Or do you need a different workflow for that?

I hope you can help me out with this one.

Many thanks in advance!

subbz2k

[Anonymous]

well, solved it in the meantime ...

turning off all lights in the scene, removing the HDR, turning FG off, applying every object a black material and applying a self-illumination material to the object i want the mask for (the asteroid in the middle) ét voila: i get what i need. packing the changes into a state set and now i can easily switch back and forth when i need to.

interesting to see that no one responded to this thread though. i mean: you all do compositing, dont you?! although i have the feeling that the way i do it right now is a bit more like a workaround and not the "proper" way, it works for now. if still someone has a "nicer" solution; would be great to hear from you!

cheers

[PROH]

Hi. Depending of the renderer used, the methods slightly differs, but... Have you tried "Render Selected"? And then save the image in a format that support "with alpha" (i.e. TIF). This way you'll get both the Beaty pass and the alpha at the same time.

Hope it helps

[Anonymous]

Hi PROH,

no, the problem never was having an alpha at all. I already save as TIFF all the time. The problem was that i needed an Alpha with Occlusion, meaning: I need objects occluding a particular object to appear in its Alpha as transparent hull - effectively cutting it out. This in turn creates a Mask for that object that is then used in the Comp to bring it in. Just rendering them alltogether doesnt work because their alphas are then merged. And rendering them separately doesnt work either because i cannot selectively invert the alpha in the post for areas i want to have the occlusion. If i would try it that way it would be a massive pain in the **** because i would have to work with animated masks in the comp and go through the render frame by frame to manually mask out the object where it is occluded. No, thats seriously NOT the way such things are done!

Normally a Matte/Shadow Material should do the trick. As far as i know and from what i've found on the net while searching for that. But it hasnt worked in my case though. While it rendered correct occlusion most of the time it sometimes also rendered occlusion where there was none. Only way to do it for now is with the method described above - still feels a bit like a workaround actually ...

i use mental ray in 3ds max 2016 Student Version. But its the same with the Scanline Renderer since the principle doesnt change.

[Anonymous]

OK, Just a quick and dirty example of what i mean:

Ive attached 3 images to this post.

Imagine the first one (maskexample_object.jpg) is the alpha of your object of interest flying through space. Now imagine the second one (maskexample_occluders.jpg) is the alpha of the objects behind and partially in front of your object of interest. You cannot render them separately because you would get solid alphas for the occluders but you can also not render them at the same time as this way their alphas would be merged.

So you need to create a mask (maskexample_objectmask.jpg) that allows you to comp your object of interest on top of the occluders. Or vice versa, but in this example i did it for the object of interest.

Do you now get what i mean? And this is what you could normally achieve by applying a Matte/Shadow to the Occluders (maskexample_occluders.jpg) which would then leave the holes in the alpha of the object of interest (maskexample_object.jpg) as they pass it. But this hasnt worked all the time in my example and so i now apply a Black material to the Occluders (maskexample_occluders.jpg) and a self-illumination material with pure white and no FG to the object (maskexample_object.jpg) which in turn creates this mask (maskexample_objectmask.jpg). But animated of course and with my real objects, not those quick and dirty example here.

But this is what i mean. If there is a better way to achieve this, please let me know. Would be really nice!

Thx

subb

[PROH]

Hi. Yes that's how I understod it...

As I said, it's render dependent. So the solution will depend on your choise of renderer. I'm using Corona, and I've attached a picture (Render Selected Example) showing whats happening when using Coronas "Render Selected". Isn't it what you want?

It's a long time since I used mr for this, but it can do the same, if "Subset Pixel Rendering" is activated. I've attached a picture showing where to find it in the renderer setup.

Hope it helps

[Anonymous]

Hi PROH,

hmm, yeah that looks exactly like what i need and what i achieve with my rather complicated workaround at the moment. I knew it: There MUST be a better way 🙂

Thanks for this hint! Im not using Corona, i use mental ray all the time. I will definitely have a look at this once the sim is finished that occupies Max at the moment. Ill report back if it worked for me.

[Anonymous]

As announced, i tested the method and it doesnt work!

If you select "Render changes to selected objects only" mental ray really only does exactly this! It still renders almost the whole scene (havent figured out why some objects are cut away yet) but it only updates (shading) changes on the objects you have selected if something changed in that scene that affects the selected object. It does however NOT knock out the object you have selected and renders the rest in black and it also does not knock out the Alpha Channel of that particular object while preserving occlusion from other objects in the scene.

The help says this about that feature:

---

Subset Pixel Rendering groupRender changes to selected objects only

When on, rendering the scene applies only to selected objects. Unlike using the Selected option for rendering, however, using this option takes into account all scene elements that affect its appearance. This includes shadows, reflection, direct and indirect lighting, and so on. Also, unlike Selected, which replaces the entire contents of the Rendered Frame Window (except for selected objects) with the background color, this option replaces only pixels used by the re-rendered selected objects.

Subset pixel rendering is particularly useful when performing iterative rendering and refinement of lighting, shadows, and other scene elements for a particular object or set of objects in the scene. It lets you re-render repeatedly to view the results of isolated changes without disturbing the rest of the rendered output.

This setting is also available on the Rendered Frame Window, as Subset Pixels (of selected objects).

---

And thats really all it does. So it is NOT possible to render correct Masks/Alphas for particular objects you selected and it therefore is NOT what i need. Which forces me to use my more complicated method again 😕

Your Corona Renderer seems to do the job and "Selected Objects" seems to mean other things there as it does in mental ray. However i dont have Corona and i wont change my renderer just because of that.

Anyway - thanks for your hints in that direction. It was definitely something to try out!

Cheers

subb

[PROH]

Hi. I tried to do it with mr, and with the first render I did get the wanted result, by after that I dididn't. The reason is this:

"Selected replaces the entire contents of the Rendered Frame Window (except for selected objects) with the background color, but Subset Pixels replaces only pixels used by the re-rendered, selected objects."

That means that if you start with an empty frame buffer, you will get the result you want. But if theframe buffer isn't empty, then the new subset render will be layed on top of this old render - both beaty and alpha.

This means that you can get the result you want, IF you clean the frame buffer before every new rendering.

When rendering a single frame it isn't a big problem to do this manually, but when rendering a sequence you'll need to find a way to do this automatically. I don't know how to do this in mr, but maybe some one has made a script for this. After all, its not an unusual request...

Hope it helps

[Anonymous]

thx for your reply PROH.

yeah, i havent tested it that far yet (FumeFX is already simulating again and occupies my machine 🙂 ) but if it only works when you clear the framebuffer, automation of a clearing process might be nearly the same hassle as putting separate material settings into separate State Sets for rendering the masks of the objects in charge.

what i've also tested some hours ago was setting the framebuffer's render area to "selected". but that doesnt work either as it really only renders whats selected - ignoring everything else. so you again dont get the correct alphas for the objects.

well, as you said: hopefully someone finds this thread one day and comes up with a solution - like a script.

but apart from that: Thanks for your time PROH and thx that you tried to figure this out with MR as well. its nice to see that someone with proper knowledge in our beloved software packages is still trying to help. as a lot of the people that have knowledge seem to be quite busy these days - not caring so much about most of the forums anymore.

so again: thx for your help PROH

[LinoFloripa]

I know this topic is old, but i had the same problem today. Using V-ray, there's a simple solution using VRAY > IMAGE SAMPLER > RENDER MASK > SELECTED. Here's a video explaining: https://www.youtube.com/watch?v=hLAbssoPsEU