Using Scaleform In VR Projects

[tbuchan]

Hi guys,

I've seen this question asked a couple of times but no concrete answer. Is it possible to link a scaleform project to a texture that appears on a plane in front of the face in VR?

We're currently doing tests on a player HUD for our VR projects and if it has to dynamically change then swapping out textures/units is quite laborious, when we already have scaleform that can do that. 

We need to be able to use scaleform to build a HUD that is rendered in front of the face. Is this a feature that's already implemented somehow or planned?

[Shanii2]

Scaleform screen is actually, a screen space. Whatever you do is displayed on screen instead of world. Like you have a screen and you apply a screensaver on top, it works like that, i don't see why you need to render it to texture and then display it on face. Even if you create a plain black background it'll come on top of your scene in screen space. What exactly are you trying to achieve. If you don't want full but small kind of screen to display on face which also appears to be floating then again you can apply 3d transforms on the Hud with ui elements. It'll would create an illusion that its floating in space, and also right at front face. You can swap the texture or image in scaleform you just need to update its bitmap component to load a different image at prior time making sure its in scaleform project files.  And their are ready examples how to do that. Also you can update the 3d transforms of Actor in scaleform with from your gameplay scripting.

[tbuchan]

Screen space UI isnt the way to go for VR, as the eyes can't focus on it correctly. It's detailed in this article here in the Unity help docs:

https://unity3d.com/learn/tutorials/topics/virtual-reality/user-interfaces-vr

So therefore the UI needs to be placed in world space, and currently i cant see how scaleform studio allows you to do that. Since the scaleform is still in screen space, 3d transforms wont work either. 

[Shanii2]

Not necessarily you can watch the stingray VR examples and ready made real projects like pool room. Rollarcoster. Piano and so on. And Ui looks amazing of course it has to be aetheticly pleasing for viewers and that's the artistic side not technical issue. Yes scale form supports 3d transforms. As an example you can push the actor in Z axis backwards and ultimately UI will be seen as in 3d world floating also you can rotate around X axis to make it the 3d view. This prevents the UI from becoming direct face up. Be sure to check out these related projects

[paul.kind]

So for UI Elements in VR there are 2 primary approaches.  Both are fine if you handle them correctly.

1. Scaleform

Generally I use scaleform only for main menus, and non 3D interactive menus.  When used as simple 2d overlays, scaleform just shines as it is rather easy to implement and offers a great deal of control.  It also helps ensure that your menu systems fit the screen.  You could also use your looking (raycast) to affect the menu but this is more tricky.  instead I would recommend using buttons to navigate the menu.

2. In Game Elements

You can easily locate 3D elements in game to your controllers.  Because this is rather easy and is vastly more intuitive, making simple 3d elements and using flow/lua to highlight, manipulate and select elements is an obvious way to go.  This is generally how I recommend keeping the feel of VR.  It just sustains the belief so much better.

Hope this helps.

Paul

[tbuchan]

Thanks guys.

So how would you reccomend that i tackle this going forward? The idea is that i want a constant player HUD to be visible in VR, showing the user where they are in the level as well as containing a little space for any tooltips for when interaction is available. 

I've got this working in the second method you detail Paul, but swapping out the texture for different scenarios seems cumbersome when we have a scaleform that can dynamically handle some of this functionality. 

[paul.kind]

If it is a 2d overlay, then yea, i would use scaleform.  It is perfect for that kind of thing.  😃

[tbuchan]

That's good to know. I've been playing with trying to hook up a scaleform project to my project but no luck to far.

I started with the oculus touch template, and i can see that it has some scaleform projects that are ready to go, but i'm struggling to get them to show!

I've done this before on a regular desktop only project, so there must be something that im missing. 

How do i go about linking my project up to use the scaleform files included in the oculus touch template? 

[paul.kind]

Just as an FYI, you dont need to apply any kind of texture.  You can just present it in scaleform.  Like Shanni said, it overlays your project so if it is just a 2 graphic you want overlaid, then you are good to go.  If you are doing some kind of map, then you will have to do the math or an easy way would be to set triggers in the rooms that get tripped when you walk into them.,  then just highlight the room on the map to show "you are now in this room".  this way you don't need to figure out where you are in the world and translate that to your map coordinates.  Im sure there is a easy way to do that as well, but id have to think on it for a while.

Paul

[paul.kind]

so scaleform loads ON TOP of a level.  You will just need to call the scaleform project with either flow or lua.  It is pretty easy and straight forward.  

Just look for the Scaleform Studio flow nodes.  You should be able to figure it out.  Its fairly easy to get started.

Paul

[tbuchan]

Thanks alot guys. I was looking at this as being far more difficult than it actually is!

I've got it working now, but im having some trouble actually getting my eyes to focus on the UI. 

I've tried using the transform in Z to push the UI back (my value is currently at 5000), but it doesnt seem to have any effect but just make the element smaller. 

[paul.kind]

im sorry, could you explain the issue better?  Im not sure what you are expecting vs what is actually happening.  The UI will always just be a 2d element.  So pushing back the Z would appear smaller, that sounds expected.

[tbuchan]

Ah i understood Shanii's reply earlier in this thread that the Z position could essentially push the element back in the scene and help with the eyes focusing on it. 

In any case i seem to be able to focus on things now, but everything is being squashed horizontally. I don't think my scaleform scene has it's stage size set correctly. I'm currently using 1200x1080, as the total HMD resolution is 2400x1080. 

What stage size should i be using in scaleform?

[paul.kind]

Try using set_view_scale_mode...  

http://help.autodesk.com/view/ScaleformStudio/ENU/?guid=__lua_ref_obj_scaleform_Stage_html

Paul