Using 3D camera data to drive position, angle and scale of batch paint strokes

[m]

Hello Flame Gurus,

I'm working on ground clean up for a drone shot with a large object in the center of frame as the camera rotates around it, very much like this camera move:

https://www.shutterstock.com/video/clip-1028838746-rusty-old-red-barn-sunny-canadian-winter

The camera angle is more challenging as it's flying low to the ground with more parallax and there is also medium tall grass, much like the grass in this shot:

https://www.shutterstock.com/video/clip-13134527-aerial-view-over-forest

The ask is to remove tire tracks like these replacing them with grass:

https://www.shutterstock.com/video/clip-1017152548-drone-perspective-takeoff-flight-rutted-track-rur...

The tracks in question are both close to camera and behind the large object. Cloning grass close to camera is tricky as the parallax is heavy. Cloning the grass behind the large object is challenging as the possible areas to source clone from either leave frame or become obstructed by center object. In most sections the grass is tall enough that the texture constantly changes. I've tackled the easy patches with a good 3D camera track projecting stills onto 3D shapes. I'm tackling the more challenging patches using batch patch, very much as described in this Autodesk Flame forum post:

https://forums.autodesk.com/t5/flame-forum/flame-object-clean-up-using-batch-paint/m-p/8091727

Since I have a solid camera track, is there a sensible way to use the camera data to drive the position, angle and scale of batch paint strokes? The paint strokes are obviously 2D and the camera data is 3D, so trying to cook up a solution. Perhaps 3D tracking graphic markers into a dummy comp, then 2D tracking the markers output to get data into batch paint animation could work? Then massaging the data with animation editing functions and expressions? Seems clunky, hoping there is a more elegant solution.

Thought I'd ask here in case someone else has a workflow to suggest. I'm sensing the real solution is UVs perspective stabilization, then paint, but can't quite "wrap" my head around it with all the parallax, camera move and object obstruction. Been searching for a good demo out there, thanks in advance for any ideas!

Cheers,

Michael

[phil_man]

if you have a solid 3d-track then you can position dummy geometry.

project the running footage from the position of the tracked camera onto the dummy geometry.

choose some static vantage points and start building your cosmetic patches.

its exponentially easier to use paint and patches when the footage is mostly static.

project your patches on to the dummy geometry and film through the tracked camera.

then fire the drone operator, the location manager and tell the producer that you underbid for the shot and it now costs 10x.