Debayering 4K RED with OLPF painfully slow

[ManChicken]

I've got some RED at 4K HD (3840x2160) and have been playing with the proxy workflow today to figure it out for an upcoming job.

It seems like something is broken in Flame with RED debayering at Full Premium with OLPF turned on.  I'm on a 16-core z820.  With OLPF off, it takes about a second to display a frame at Full Premium.  But when I switch it on, that more than doubles to 3 seconds or worse-- it's usually about 3 but sometimes it will randomly take drastically longer, 5 or 6 seconds a frame.  This is just sitting in the Mediahub and jumping from frame to frame while adjusting the settings, so there's no other effects or anything weird applied (but it behaves the same way once I drag the clip in to the library and try to use it.)

For the heck of it I pulled up the same R3D in Resolve in Windows on a different z800 (a much slower 8-core machine) and set it to Full Premium with OLPF, and the render time is not much different than with OLPF off.. not quite 1.5 sec a frame.  I pulled the same file into Avid DS on that same workstation and did the same thing.. again, under 1.5sec/frame.

??

[jaseo]

One thing to try is to adjust the number of wiretapgateway slaves.

Edit /usr/discreet/wiretapgateway/cfg/wiretapgateway.cfg

look for

#NumLocalSlaves=4

unhash and change value to 8 or 12

ie.

NumLocalSlaves=12

save and then restart WTG (as root)

/etc/init.d/wiretapgateway restart

cheers

Jason

[ManChicken]

Hey Jason.

I stumbled upon that from some Googling a little earlier and gave it a try;  I saw a bunch more gateway processes running, but Full Premium with OLPF turned on didn't seem to change, at all.

If I place a Full Premium 4K clip on an 1080 HD sequence with OLPF turned off, when I hit Render it takes a bout 2.5mins to process a ~1.25min clip.  Doing nothing else than turning OLPF on, the render estimate goes up to nearly 4 hours, watching it tick through one or two frames at a time every 10 seconds or so.  Looking at the system with 'top' I really only see 2 cores working.

Strangely, turning on OLPF with Half Premium doesn't exhibit the comparitvely massive slowdown than with it off, like it does with Full Premium.

It smells a little like a bug.  My understanding is the RED SDK is not inherently multithreaded, and everyone just uses multiple decoder processes to power through.  But it seems like in this one specific instance Flame isn't.  I'll call support Friday afternoon once I'm done with a session and see what they think.

[Anonymous]

Hi,

It does take an age. If it's any consolation, it's painfully slow on our Baselight too; being a very processing heavy option. It can look nice, but I'm not sure it's always worth the wait.

Ant

[Anonymous]

What's the optimal value for number of slaves on a Z820 machine? The nproc command returns 32, but these are not physical cores, right?

I set the number to 32 in the .cgf file and Flame would get stuck at rendering. However, when I reduced it to 12 there is a massive debayering performance increase during renders over the default setting.

[ManChicken]

Hyperthreading is on, so yeah you have 16 physical cores but 32 seen by the OS.

Besides CPU, each instance uses RAM and is obviously reading and writing to disk, so it's a balance of resources, leaving something left for the rest of the system to still run in some capacity.  That said 12 is what I use as well.

[hengy]

After having conversations with others doing RED I've since only used high detail and all other settings off. When in Flame I'll do any denoise and sharpening necessary. The amount of time it takes to render full resolution RED is too costly.

[Anonymous]

I have never seen any benefit of denoise that post tools can't handle faster. OLPF has always been too time costly for us.