topic Re: Oculus not seeing camera view in Stingray Forum (Read Only)

Oculus not seeing camera view

cbenner — Mon, 03 Apr 2017 14:13:01 GMT

Hello,

I've run through the roller coaster demo setup, and was able to duplicate the demo in Stingray. As of Friday, playing the level (without Oculus), the viewpoint of the level test was that of the camera. This morning I tried the exact same file with the Oculus plugged in, and while Oculus can SEE the level, it is not seeing it form the same viewpoint.... it's like the camera is above and off to the side.

Is there something in the Oculus Lua that needs to be set to tell it to use the same camera?

Re: Oculus not seeing camera view

cbenner — Mon, 03 Apr 2017 16:30:33 GMT

@paul.kind @dan.matlack

Does this have something to do with positional tracking? Can I disable it somehow? Oculus is stuck in Persective, not looking from the camera called out in the level flow. Help please?

Re: Oculus not seeing camera view

dan.matlack — Mon, 03 Apr 2017 16:46:51 GMT

It is my understanding that the roller coaster demo was set up to have the hmd be set at the camera location based on the project. If you are using a different project or have adjusted the project, that may no longer be the case. You need to set the hmd position to be that of the desired camera position. You can do that by updating it every tick to that position or by linking it to the camera in flow or lua.

Re: Oculus not seeing camera view

paul.kind — Mon, 03 Apr 2017 16:48:29 GMT

Hrm, thats a curious problem.

Is the camera tracking correctly along with the tutorial only it is offset? Forgive me, i have mot personally gone through the tutorial myself, but, i know enough to help you out here. Also, if you move your head around does the oculus move with it?

If the answers above are "no" then you may want to set your tracking space using the tracking space node to wherever you want your "head" to be. You will want to update this every frame.

Lemme know if this helps.

Paul

Re: Oculus not seeing camera view

cbenner — Mon, 03 Apr 2017 16:50:57 GMT

@dan.matlack

@dan.matlack wrote:

It is my understanding that the roller coaster demo was set up to have the hmd be set at the camera location based on the project. If you are using a different project or have adjusted the project, that may no longer be the case. You need to set the hmd position to be that of the desired camera position. You can do that by updating it every tick to that position or by linking it to the camera in flow or lua.

Thank you, but I am a complete newbie, you might have to dumb that down for me.... Sorry.

Re: Oculus not seeing camera view

cbenner — Mon, 03 Apr 2017 16:58:36 GMT

@paul.kind

If I disconnect Oculus completely, and run level test without it, the camera is set as expected based on the tutorial. Which is to say, you are in the car on the track. But as soon as I plug in the Oculus, it jumps back to a perspective view. I'm sure this is Oculus 101, but I'm in over my head here coming from an Inventor background. I have people that want to see this working, and I'm not able to make it happen.

Re: Oculus not seeing camera view

paul.kind — Mon, 03 Apr 2017 21:08:27 GMT

Ok, so it should be pretty easy no matter the case. If you have a few minutes, contact me on Skype, my user name is threedeenut (Paul Kind). Just friend request me and lets have a call together to resolve.

Paul

Re: Oculus not seeing camera view

cbenner — Tue, 04 Apr 2017 12:11:11 GMT

@paul.kind wrote:

Ok, so it should be pretty easy no matter the case. If you have a few minutes, contact me on Skype, my user name is threedeenut (Paul Kind). Just friend request me and lets have a call together to resolve.

Paul

Paul,

Thank you for the offer, but that's not really going to work. This is all on my work PC and I don't have the ability to Skype from here. Thanks anyway.

Re: Oculus not seeing camera view

paul.kind — Tue, 04 Apr 2017 12:44:10 GMT

Do you have any other way I can screen share/meet with you? Teamviewer maybe?

Alternatively you could upload me what you have somewhere and I can take a look and see if I can resolve it for you.

Re: Oculus not seeing camera view

cbenner — Tue, 04 Apr 2017 13:02:21 GMT

@paul.kind

I've sent you a PM... once we get to a resolution I will update here so the solution will be searchable. Thank you for your help!

Re: Oculus not seeing camera view

cbenner — Tue, 04 Apr 2017 19:19:27 GMT

I thought It wouldn't hurt to tag @MatthewDoyleArt in this thread also, since the roller coaster has his name on it.

Re: Oculus not seeing camera view

MatthewDoyleArt — Tue, 04 Apr 2017 20:04:03 GMT

Hi,

I updated the Rollercoaster VR kit to Stingray 1.7. You can try to download the new version from here: https://autodesk.box.com/s/jsutlpm7v6kxx5l817ejybuc025rv1az

The update no longer uses Flow to control the Oculus camera. It is all done in Lua now. I am actually creating a Lua Tips & Tricks video which will go live this week that explains how I did it.

Until then, here's the basic code I did to make it work:

Step 1: Open Project.lua script and specify the level (line 11)

Project.level_names = {
rollercoaster = "content/levels/RollerCoaster_VR/RollerCoaster_VR_Level"
}

Step 2: Add these local variables (line 35)

-- Cart & Player Start
local cart = nil
local player_starts = nil

Step 3: Adjust the SimpleProject.config data (line 43).

Just change the standalone_init_level_name to point to the level defined in Step 1 above:

SimpleProject.config = {
standalone_init_level_name = Project.level_names.rollercoaster,

Step 4: Modify get_player_start_pose() function (line 53)

The entire function should now look like this:

local function get_player_start_pose()
    local pose = stingray.Unit.world_pose(player_starts, 1)
    local scale = stingray.Unit.local_scale(player_starts, 1)
    stingray.Matrix4x4.set_scale(pose, scale)
   return pose
end

Step 5: Modify Project.on_level_load_pre_flow() function (line 84)

function Project.on_level_load_pre_flow()
 
-- Add this code directly after the line: Project.vr_eye_height = vr_eye_height
 
    -- Link player start to cart
    cart = stingray.World.unit_by_name(SimpleProject.world, "Coaster_Camera_B")
    local cartIndex = stingray.Unit.node(cart, "Cart")
    player_starts = stingray.World.unit_by_name(SimpleProject.world, "player_start")
    stingray.World.link_unit(SimpleProject.world, player_starts, cart, cartIndex)

Step 6: Modify Project.update(dt) function (line 161)

function Project.update(dt)

-- Add this code directly after the line: if not OculusVRSystem.is_ensabled(vr_system) then return end

local player_start_pose = get_player_start_pose()
local player_start_position = stingray.Matrix4x4.transform(player_start_pose, stingray.Vector3(0, -0.5, 2))
stingray.Matrix4x4.set_translation(player_start_pose, player_start_position)
OculusVRSystem.set_tracking_space_pose(vr_system, player_start_pose)

Re: Oculus not seeing camera view

cbenner — Tue, 04 Apr 2017 20:21:35 GMT

Wow, thanks Mr. Doyle. I'll have a look at that tomorrow. Gotta say after 20 years in mechanical cad, I feel like a stranger in a strange land... but it is fun to learn something new.

Re: Oculus not seeing camera view

MatthewDoyleArt — Tue, 04 Apr 2017 20:24:23 GMT

You're welcome. If you have any issues with the update kit, let me know. I have tested it with DK2 on my end and it works great. I moved from Flow to Lua, because the camera was laggy in Flow. In Lua, the VR camera maintains a perfect position as the coaster moves.

Re: Oculus not seeing camera view

cbenner — Tue, 04 Apr 2017 20:30:32 GMT

Sounds promising. Where we are taking this is to create a fly through of some of the systems we build, to show off at trade shows and such. The coaster concept, once I get the hang of it, should translate nicely to what we want to do.

Re: Oculus not seeing camera view

MatthewDoyleArt — Tue, 04 Apr 2017 20:34:22 GMT

Cool. Well, let me know if you have trouble downloading the 1.7 kit from that link. If you don't have access, I will try to figure out another place I can put it for you.

Re: Oculus not seeing camera view

cbenner — Fri, 07 Apr 2017 13:16:55 GMT

@MatthewDoyleArt

Hello again. Finally got back to looking at this. I'm trying to take your code lines and apply them to a new project (adjusted of course). The level test keeps failing at this line of code:

This project I am running doesn't have the camera attached to a physical entity like the cart in the coaster, so I made "cart" = to the name of the node in my camera entity. It's the player_start line it seems not to like. taking it out doesn't seem to fix the issue. Any ideas?

Re: Oculus not seeing camera view

cbenner — Fri, 07 Apr 2017 13:55:27 GMT

@MatthewDoyleArt

Digging deeper, the line items are different in this project than in your original coaster lua, so I opened that and put the code you gave me above into the appropriate lines on this new script. The logs keep pointing me to this section:

-- Cart & Player Start
local cart = nil
local player_starts = nil

-- Can provide a config for the basic project, or it will use a default if not.
SimpleProject.config = {
	standalone_init_level_name = Project.level_names.VR_Chem_Plant,
	camera_unit = "core/appkit/units/camera/camera",
	camera_index = 1,
	shading_environment = nil, -- Will override levels that have env set in editor.
	create_free_cam_player = false, -- Project will provide its own player.
	exit_standalone_with_esc_key = true
}

-- check to see if there are player starts in the level, if not return world 0
local function get_player_start_pose()
		if #player_starts > 0 then
		local index = stingray.Math.random(1, #player_starts)
		local pose = stingray.Unit.world_pose(player_starts, 1)
		local scale = stingray.Unit.local_scale(player_starts, 1)
		stingray.Matrix4x4.set_scale(pose, scale)
		return pose
	else
		return stingray.Matrix4x4.identity()
	end
end

The highlighted line seems to be the one failing.

Re: Oculus not seeing camera view

MatthewDoyleArt — Fri, 07 Apr 2017 18:08:12 GMT

I see. The code I provided works with the coaster project due to the way it is set up. I don't know about how your project is set up, but it sounds like you are just using an animated camera, rather than an animated mesh object.

(Update: See my next post for a solution that should work with just an animated camera)

Here's how my setup works:

First up, you must have a player spawn point located at the location you want to start at.

Select Create Tab, and choose Player Start under Misc.
Place the spawn point as I have in this image below.
Name it player_start

The code below is added to: Project.on_level_load_pre_flow()

It gets references to the cart mesh & it's index, as well as the player start, then it links the player start to the cart.

You might not need these lines of code at all if you aren't linking the player spawn point to a moving mesh object, but instead using a camera.

cart = stingray.World.unit_by_name(SimpleProject.world, "Coaster_Camera_B")
local cartIndex = stingray.Unit.node(cart, "Cart")
player_starts = stingray.World.unit_by_name(SimpleProject.world, "player_start")
stingray.World.link_unit(SimpleProject.world, player_starts, cart, cartIndex)

Next, this code in the Project.update(dt) function makes sure the VR camera's tracking space is updated every tick of the game loop:

local player_start_pose = get_player_start_pose()
local player_start_position = stingray.Matrix4x4.transform(player_start_pose, stingray.Vector3(0, -0.25, 2.25))
stingray.Matrix4x4.set_translation(player_start_pose, player_start_position)
OculusVRSystem.set_tracking_space_pose(vr_system, player_start_pose)

Finally, this function replaces the original get_player_start_pose() function in the oculus vr template. It uses the player_starts spawn point to find out where the VR camera should be at any given frame, and is called on by the code above in Project.update(dt):

Notice it gets its pose & scale info from the player_starts object. If you want to use the camera, you'll need to replace these lines with code that gets the pose & scale from the camera instead.

local function get_player_start_pose()
    local pose = stingray.Unit.world_pose(player_starts, 1)
    local scale = stingray.Unit.local_scale(player_starts, 1)
    stingray.Matrix4x4.set_scale(pose, scale)
    return pose
end

Re: Oculus not seeing camera view

MatthewDoyleArt — Fri, 07 Apr 2017 18:15:22 GMT

Here's what I did to get it to work with just a camera:

Assume the camera in my level has the name "myCam" and that the camera is animated.
Add the local variable in Lua:
```
local myCam = nil
```

Change the get_player_start_pose() function to this:

local function get_player_start_pose()
    local pose = stingray.Unit.world_pose(myCam, 1)
    local scale = stingray.Unit.local_scale(myCam, 1)
    stingray.Matrix4x4.set_scale(pose, scale)
    return pose
end

Add this line of code to Project.on_level_load_pre_flow()

myCam = stingray.World.unit_by_name(SimpleProject.world, "myCam")

Add this code to the Project.update(dt) function:

 local player_start_pose = get_player_start_pose()
 OculusVRSystem.set_tracking_space_pose(vr_system, player_start_pose)