Navigating through a Natural User Interface using your palm is quite common – after all, it’s the primary navigation mechanism XBOX uses. Many Windows Kinect applications implement hand tracking for similar purposes. Today, I would like to share a Kinect hand cursor control I developed and you can use for your own apps. This hand cursor control will save you tons of time and you’ll be able to integrate it right into your existing WPF code!

Here is the final result of this handful user control:

Kinect hand cursor example

Using the control in your project is fairly easy. Read on!

Prerequisites

The code

OK, let’s type some quick code now.

Step 1: Download the project from GitHub

Download the source code and build it using Visual Studio. Locate the assembly named KinectControls.dll.

Step 2: Import the assembly to your project

Create a new WPF project and add a reference to the assembly you built previously.

Step 3: Import the assembly to your XAML code

Type the following line of code in your Window, view or user control definition:


xmlns:Controls="clr-namespace:KinectControls;assembly=KinectControls"

You can now drag a Canvas and place the hand cursor inside it:

<Canvas Name="canvas" Width="640" Height="480">
    <Controls:KinectCursor x:Name="cursor" Width="100" Height="100" />
</Canvas>

Step 4: Move the cursor using C#

The KinectCursor class contains a method named Update. You need to call that method whenever you want to move the cursor. The Update method gets the X and Y coordinates of the hand in the 2D color or depth space. Alternatively, you can provide a ColorSpacePoint or DepthSpacePoint that contains the coordinates. I placed this code inside SkeletonFrameReady event handler. You can specify the “active” hand by comparing the Z values of the left and right hands.


// Select the hand that is closer to the sensor.
var activeHand = handRight.Position.Z <= handLeft.Position.Z ? handRight : handLeft;
var position = _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(
                                        activeHand.Position,
                                        ColorImageFormat.RgbResolution640x480Fps30);

cursor.Flip(activeHand);                            
cursor.Update(position);

Also, notice the Flip method? The Flip method mirrors the cursor visual, so to properly match the active hand. That’s it! You can check the complete source code on GitHub.

The hand image is a scalable vector shape I designed. You can change its color, dimensions or shadow effects easily:


<Controls:KinectCursor x:Name="cursor" Width="400" Height="400" Fill="Blue" />

Copyrights

You are free to use the user control as you wish for your personal and commercial projects, just by making a simple attribution in your project.

PS: Vitruvius

If you enjoyed this article, then you’ll love Vitruvius. Vitruvius is a set of powerful Kinect extensions that will help you build stunning Kinect apps in minutes. Vitruvius includes avateering, HD Face, background removal, angle calculations, and more. Check it now.

Want to hire my team for your next cutting-edge fitness app? Drop me an email.

Vangos Pterneas

Author Vangos Pterneas

Vangos Pterneas is a professional software engineer and an award-winning Microsoft Most Valuable Professional (2014-2019). Since 2012, Vangos has been helping Fortune-500 companies and ambitious startups create demanding motion-tracking applications. He’s obsessive about analyzing and modeling every aspect of the human motion using Computer Vision and Mathematics. Kinect programming started as a hobby and quickly evolved into a full-time business (lightbuzz.com). Vangos is sharing his passion by regularly publishing articles and open-source projects that help fellow developers understand the fascinating Kinect technology.

More posts by Vangos Pterneas

Join the discussion 37 Comments

  • Chris says:

    Your “Copyrights” isn’t a copyright message, but rather a simple, non-standard license that actually contradicts the GPL license for your project on GitHub.

    • Hey Chris. Thank you very much for noting that issue! I changed the license to MIT License. You can see the update on GitHub. I just want everyone to use this control for free in personal or commercial projects.

      Have I mentioned that I hate lawyers? 😉

  • tcboy says:

    I didn’t try your code yet
    but I think it only works in your application right?
    will it work outside your application?
    like controlling the windows cursor.

    • Hi tcboy. The code works on any application. You simply provide the required coordinates. It will not control the mouse, as this is not a good practice for most applications.

      If you need a more solid solution, try Microsoft Kinect Toolkit and KinectRegion.

  • Helmut says:

    Hi!

    I didn’t tried you code till now.
    Does your code work also in a Windows store app?

    And is your code compatible with kinect v2 sensor?

    Thx

  • Chris says:

    Hello, it’s July now. When can we see the new book 🙁

  • ken says:

    Hey, I just built the source against the latest Kinect v2.00 pre-release SDK (1406) and it fails to build due to DepthImagePoint & DepthImagePoint structures?

  • Emmanuel says:

    Qué SDK usas? 1.7 ?

  • Fahd Arshad says:

    hello ,

    I am working on 3D Sculpting using kinect, I have got the sculpting aspect covered by creating a javascript WinRT application.
    We are stuck at using kinect to bind with the mouse which is done by your application.

    I would like some help as to how to connect this application with our JavaScript WinRT application

    • Hi Fahd. Can you import C# assemblies into your JavaScript project? As an alternative solution, I suggest you used ControlsBasics. Launch the SDK Browser from the Start Menu and select the “Windows Store” tab. Select the “Controls Basics – HTML” sample. You can configure the built-in hand navigation, along with a few predefined gestures. This will solve your problems.

  • George Alain says:

    Hi
    FIrst of all thanks a lot for the things you post on your blog regading to the kinect and I am waiting for your book.
    I have built your code in VS and included it in my project.
    I am working on a kinect V2 app so it’s worth to mention that the xaml code to include the reference for v2 would look like this

    including the code and the cursor control went just fine and compiled with no errors.

    Now, could you please help me on how to update the cursor in the Kinect V2 code?
    var position = ??
    and should I put this bit in the event where I read a body frame or what?

    thanks.

  • Ravi says:

    Hi,

    this is actually a very nice example. i just want to know whether it is possible for Kinect to detect hand gesture when the Hand position is in horizontal mode, like we actually doing with mouse.
    i am trying to detect button click using hand push like in example but in my scenario the palm will not be facing the kinect and hand will be in horizontal position and move downward instead of forward.

    • Hi Ravi. Are you trying to perform a “push” gesture? That is, moving your hand back and forth very quickly. You could check 15-20 frames and compare the relative distance between the hand and the sensor. 15-20 frames is the window size you’d need to use (aka the “time” between a back-and-forth gesture).

  • FI says:

    Hi Vangos,

    I have a question not exactly related to this article. I have followed this post ( https://social.msdn.microsoft.com/Forums/en-US/970aa6d7-da26-4dc1-9445-90d09d6a15eb/doing-drag-drop-with-handpointers?forum=kinectv2sdk ) to implement drag and drop functionality in my application. I have three user controls (Left,Middle,Right) that are being hosted in one main user control. In my Left UC i have three images so if a user drags and drops any of these three images in Right UC with the help of Hand Cursor, i want the dropped image to be copied in Right UC (or a new instance of the image is created which is visible in Right UC or a snapshot is taken and shown in Right UC which can be viewed later). The original image in Left UC should be at the same place before the drag and drop operation.

    Can you please guide me how can i implement this with Kinect V2 Hand Cursor as there is no support for copy and paste operations as it is for mouse cursor and also no events are fired when the hand cursor is over any control or it enters or leaves a specific control. Thank you.

  • Anand Kumar says:

    Hi Vangos,
    I was just wonderin, where is the KinectControls.dll file? It’s not in the directory you provided.
    Pls send the file to my email at anandskumar927@gmail.com.
    Thnx,
    Anand

  • Jack says:

    Hi Vangos,
    I‘m a beginner and i found your blog fortunately.Your work is wonderful . Currently i use Kinect v2 with SDKv2.0_1409,but i find the source code doesn’t work. VS2013 shows me about 22 errors.What should i do?Thank you!
    Jack

  • hasan says:

    hi
    i am trying to control mouse courser by kinect 360 for some pursuit rotatory games.
    but i dont know any computer language. what can i do. is it possible guide me by pic or send me their codes.

  • ali says:

    Hi! Can I use it for WinForm application?

  • Mauro says:

    Hi! First, I want to thank you for the awesome work you do!

    Second, I would like to know if its possible to change the cursor?

    Thanks!

  • Fernando says:

    hey, can i use this on unity 3D?

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.