Lua Mouse interactivity with Units

[kainevg]

Hi everyone!

I've been exploring how I can implement a function that can return the unit the mouse clicks on.. I've been looking at the functions in the Lua API, so far the ones that have caught my attention have been Camera.world_to_screen() and Camera.screen_to_world(). Also potentially using some form of raycast from the camera, though I can think of a few issues with that. Any ideas or examples on how to accomplish this?

Cheers,

Kaine

[dan.matlack]

Hey Kaine,

There are a couple ways we can do this. I'm investigating now the best practice and will see if I can get you some sample code soon to test out.

[kainevg]

Thanks mate! Looking forward to the answer!

[dan.matlack]

In the project.lua, you can do something like below to get the mouse click:

function Project.update(dt)
    
    if(stingray.Mouse.pressed(stingray.Mouse.button_id("left"))) then
        print "mouse left pressed"
    end

    if(stingray.Mouse.released(stingray.Mouse.button_id("left"))) then
        print "mouse left released"
    end

end

After that you'd need to do a cast out and return whatever object you have hit. You could also do something similar with scaleform and listeners.

You can also get the mouse focus in window as below:

   -- Used to capture the mouse focus so you can interact with your UI using the mouse
    stingray.Window.set_mouse_focus(true)
    stingray.Window.set_clip_cursor(true)
    stingray.Window.set_show_cursor(true)

 Does this help you out at all? 

[kainevg]

Hey Dan, thanks for the response.

I'm not sure how I would cast out to check. Obviously I'd use a ray tracer but I'm not sure how to determine where to cast the ray.. could I have a more specific example on how to accomplish it?

Thanks!

Kaine

[kainevg]

The ray would just be perpendicular to the Camera wouldn't it, now that I actually think about it.. is this correct?

[dan.matlack]

I'm still looking into this and will get back to you with a solution as soon as possible. 🙂

[kainevg]

Thanks Dan!

When I said perpendicular I meant parallel as well. Oops. It's 4:50am here, so I'm going to use that as my excuse.

I believe I'm getting closer to having it working, however it'd be awesome to see how the pros do it, as I want to accomplish it in the most efficient way possible.

Cheers,

Kaine

[kainevg]

Could you also see if you can find out how to get the vector position of the mouse? i.e (x,y,0)

There isn't a lot of info on the Mouse in docs yet.

Thanks again!!

Kaine

[devirosnguyen]

Same question with Kaine, how to get mouse position when mouse pressed/released?

Thanks,

Deviros

[kainevg]

Hey Deviros,

This should do the trick:

stingray.Mouse.axis(stingray.Mouse.axis_id("cursor"), stingray.Mouse.RAW, 3).x,
stingray.Mouse.axis(stingray.Mouse.axis_id("cursor"), stingray.Mouse.RAW, 3).y)

Cheers,

Kaine

[dan.matlack]

Hi Diviros,

You can return the location of the cursor by using the following:

stingray.Mouse.axis(stingray.Mouse.axis_id("cursor"), stingray.Mouse.RAW, 3).x,
stingray.Mouse.axis(stingray.Mouse.axis_id("cursor"), stingray.Mouse.RAW, 3).y)

[devirosnguyen]

Thank Dan and Kaine,

Can you give me any mouse button, example : "left", "right", "middle", "cursor" ... blah blah

Thanks,

Deviros

[devirosnguyen]

Sorry,

I have a question about Screen_to_world function, this line in player.lua:

local mouse = stingray.Mouse.axis(stingray.Mouse.axis_id("cursor"), stingray.Mouse.RAW, 3)
local v3 = stingray.Camera.screen_to_world(self.player_camera, mouse, 1,stingray.Window)

I get error

bad argument #1 to 'screen_to_world' (userdata expected, got table)

So, what is camera parameter in screen_to_world() function?

Thanks!

Deviros 

[kainevg]

Hi!

To check if a Mouse button is being pressed (this also applies to the Keyboard, you just replace the Mouse namespace with the Keyboard namespace) you can do it like this:

local leftMousePressed = Mouse.button(Mouse.button_id("left"))

This will return 1 while the left mouse button is pressed, and 0 when it's not. Keep in mind that for that line to work you have to cache of the namespace (normally at the top of your Lua file) like this:

local Mouse = stingray.Mouse

This is just so you don't have to put stingray.Mouse everytime, and it also has some slight performance gains I believe, though I could be wrong.

Here is a list of all available mouse button IDs (I haven't tested these):

extra__1
extra__1__double
extra__2
extra__2__double
left
left__double
middle
middle__double
right
right__double
wheel__down
wheel__left
wheel__right
wheel__up

The Mouse inherits all the functions under the InputController and by extension the BaseInputController so I recommend reading the documentation on those to see exactly what you can do.

BaseInputController Documentation

For your second question:

If you check the documentation for the screen_to_world() (screen_to_world() Documentation) you'll see that is takes a stingray.Camera object as the parameter. The error you recieved is there for the reason it states; player_camera is only a table which is used to store the camera. In the player.lua script under the init() function you'll see it creates the variable:

self.player_camera = {}

the {} defines a blank table. This variable player_camera is used to store the unit which in turn is storing the stingray.Camera object. If you're going through these templates, or using the Appkit then the easiest way to get the actual Camera instance is by using the built in functionality of the camera_wrapper.lua script. An example of putting it all together would look something like:

local player_camera = self.player_camera	
local camera_wrapper = CameraWrapper.manager:get(player_camera)
local camera = camera_wrapper:get_camera() 

local mouseXPos = Mouse.axis(Mouse.axis_id("cursor"), Mouse.RAW, 3).x
local mouseYPos = Mouse.axis(Mouse.axis_id("cursor"), Mouse.RAW, 3).y
local depth = 1

local mouseWorldCoords = Camera.screen_to_world(camera, Vector3(mouseXPos, mouseYPos, 0), depth)

As you can see, I've left out the Window parameter from the screen_to_world() function as it is an optional parameter. If it is left out then it uses the largest open application window, so if you only have one window in your application, you don't need to worry about it. 

I'd recommened you look through the code from the different Appkit files so you can better understand how it all works together, you can find them under Program Files -> Autodesk -> Stingray -> VersionNumber -> core -> appkit -> lua

Note: Everytime I ommit "stingray." before a namespace i.e on Mouse and Camera, that means I have cached it like was shown above in this post for speed and also readability. I also haven't tested the code so I would just read it as psuedo code. It should work though.

I hope this helps! If you have any issues let me know.

Kaine

[dan.matlack]

Thanks Kaine, very solid explanation 🙂

[_robbs_ · Accepted solution]

I've just been cleaning up the input controller stuff in the docs... Coming next update.

Kaine you have the names right, except you have them with an extra underscore.

Mouse button names

• `left`, `right`, `middle`: Indicates single-clicks on the main mouse buttons.
• `extra_1`, `extra_2`: Indicates single-clicks on the extra mouse buttons, if any.
• `left_double`, `right_double`, `middle_double`, `extra_1_double`, `extra_2_double`: Indicates double-clicks on the mouse buttons.
• `wheel_up`: Indicates when the mouse wheel is rolled upward (i.e. forward, or away from the player).
• `wheel_down`: Indicates when the mouse wheel is rolled downward (i.e. back, or toward the player).
• `wheel_left`: Indicates when the mouse wheel is rolled to the left.
• `wheel_right`: Indicates when the mouse wheel is rolled to the right.

Mouse axis names

• `mouse`: Indicates the difference in the mouse position since the previous frame, in (X,Y) screen coordinates.
• `wheel`: Indicates the movement of the mouse wheel. Positive numbers indicate upward scrolling (i.e. forward, or away from the player). Negative numbers indicate downward scrolling (i.e. back, or toward the player).
• `cursor`: Indicates the (X,Y) position of the mouse cursor in screen coordinates.

[kainevg]

Thanks for clarifying! I got the names from the flow editor, and they appear to have double underscores there, not sure if that's a mistake? Thanks though!

Cheers,

Kaine

[devirosnguyen]

Thanks @_robbs_ @kainevg for this response.

Cheers,

Deviros

[kainevg]

Hi!

I just wanted to clarify why a post was accepted as a solution? We've had some informative posts but the question I asked hasn't been answered yet.

Cheers,

Kaine