Warped Projection Mapping Issues

[Anonymous]

Hi all,

I'm working in 2016 on a facial animation setup that requires re-projection of an image sequence onto a geometry-tracked face replacement for re-animation and warping of the overlayed footage. I've got my texture reference object set up and working, and the footage is warped nicely in the final render, but I need to be able to see the playback and warping of the footage realtime in the viewport for animation work.

It seems that only the "legacy high quality viewport" renderer will allow the footage to show the updated distortion, but won't play back the sequence at realtime, whereas viewport 2.0 plays the sequence back butter smooth, but won't display the distortion.

The rig we're working with is joint-based. The projection mapping is being done through an FBX camera that is being brought in from PFTrack. The sequences that I'm loading are HD 1080, and I've tried several different sequence types, I'm currently using Jpegs and they still will not play back realtime using the HQ legacy viewport renderer.

Anyone have some recommendations or workarounds for getting both the sequence and the distortion/warping to happen in realtime in the viewport?



thanks,
Lenny

[Anonymous]

So, after further investigation/experimentation, it seems that the problem lies in the projection node itself. If I want to be able to view the projection distortion live in the viewport, I have to first commit the projection to a baked texture sequence, thereby eliminating the projection node from the shading network and committing everything to UVs.

For instance, I have a similar scene set up in Cinema 4D, and all I have to do to get the projection to lock to the surface is apply a "stick texture" tag to the mesh and record a given position. The projection can then be warped/distorted by any animation/deformation that I apply afterwards.  Baking the textures in Maya is adding an additional time consuming step to the process.