NOTE: This is preliminary software and/or hardware and APIs are preliminary and subject to change.
Well, I have been lucky enough since Microsoft selected me for early access to the new Kinect for Windows version 2 sensor. Today, I want to share some facts and figures regarding the new device.
The hardware
The new sensor features a radically different hardware design. First thing to notice is that the tilt motor is now gone. However, the new cameras provide a wider field of view and feature frames of higher resolution. Above, you can see my Developer Preview unit, unboxed. Below, you can watch a quick video I made, demonstrating the new color, depth, infrared and body streams.
And here are the differences between the old and the new sensor.
| Version 1 | Version 2 | |
|---|---|---|
| Depth range | 0.4m โ 4.0m | 0.4m โ 4.5m |
| Color stream | 640×480 | 1920×1080 |
| Depth stream | 320×240 | 512×424 |
| Infrared stream | None | 512×424 |
| Audio stream | 4-mic array | 4-mic array |
| USB | 2.0 | 3.0 |
The software
The new API is quite similar to the old one, however, the projects developed using Kinect SDK v1 will not run as-is on the new sensor. Some class names and methodologies have been changed, so a plain recompile won’t do the job. Do not worry though. Throughout the upcoming series of blog posts, I will try to explain every difference and guide you step-by-step on creating your Kinect v2 projects.
So, here are my highlights:
Better camera streams
Kinect combines a color, a depth and an infrared camera at an affordable price. Version 1 had low specification cameras, even though the results were stunning. The second version of Kinect doubles the color camera resolution and almost-doubles the depth camera resolution. The data are processed faster and the accuracy is radically increased.
More joints
Yeah, the new sensor tracks up to 25 body joints, along with their corresponding orientations. There are some joint renames as well.
Hand tracking
That’s right! Except from joint tracking, the new sensor lets us determine the state of the users’ hands. The state is just an enumeration with values of “Open”, “Closed”, “Lasso”, “Unknown” and “NotTracked”. This is really useful for creating smoother natural user interfaces. For example, we can now determine that the user is performing a wave gesture when the hand joint is moving AND the hand state is “Open”. If you need more advanced finger tracking, consider Candescent NUI.
Facial expressions
Kinect for Windows version 1 could track 40 points of the human face. Kinect for Windows version 2 goes one step further and can even recognize some very basic facial expressions, activities and accessories! Here are supported facial data:
- Eyes closed
- Eyes looking away
- Mouth open
- Mouth moved
- Glasses accessory
- Happy expression
- Neutral expression
Potential? Unlimited! Personally, I control the lights of my house using gestures in front of my Kinect sensor. Now, I can just blink my eye and the living room spots turn on! (More on this at a later blog post)
Overall
Let me clarify that I am not paid by Microsoft, though Kinect for Windows version 2 is my gadget of choice for 2014. Everything has been dramatically improved and new features will be popping all the time. Now, the only limit of software is your imagination.
PS: Vitruvius
If you enjoyed this post, consider checking Vitruvius. Vitruvius is a set of powerful Kinect extensions that will help you build stunning Kinect apps in minutes. Vitruvius includes avateering, HD Face, background removal, angle calculations, and more. Check it now.
About USB 3.0 requirement, I was wondering if a USB 3.0 Card Adapter works for Kinect v2 or not. My machine does not have the hardware requirements right now, and I am not sure if a card adapter and the power supply would be compatible with v2 or not.
Thanks
Hi Nazli,
I have not tested Kinect v2 with a USB 3 card adapter. The release notes require “Built-in USB 3.0 host controller (Intel or Renesas chipset)”.
Currently, Kinect for Windows v2 is under Developer Preview, so the final requirements will most probably change by summer.
Thanks very much for your informative response Vangos.
Hello Vangos,
I’ve just started working with the Kinect SDK 1.8, but I will be switching to the new version later this month. I’m interesting in the facil expressions and looks like the improvement was amazing. From 40 points to 2000, it will really give us way more facial details.
Just one question, the app that you have used to recongnize your left eye and mouth open in your post, is it a sample project available in the new sdk tookit?
Thank you!
– Diego
Hi Diego,
The pre-installed applications do not contain facial expressions demos, though I am planning new blog posts about such features ๐
Oh cool ๐
Already following your blog and looking forward to reading your book ๐
– Diego
Hi Vangos,
I am new with Kinect, and I have problem with missing hands and wrists locations. I am trying to collect data for my project which is gesture detection and recognition using skeleton tracker of Microsoft, but there are lots of missing data. How can I solve the problem? Is there any way that I could correct my previous data as well as changing the code so that I could collect reliable data? Would new connect be helpful?
-Erfan
Hi Erfan. Could you send me a code sample? That tutorial may help you: https://pterneas.com/2014/03/21/kinect-for-windows-version-2-hand-tracking/
Hi Vangos,
I am trying to modify Candescent NUI in order to make it work with Kinect V2. But it turned out very difficult to a starter like me because I don’t have experience in V1 SDK and now I can’t sort out all the new names/places for V1 classes/objects. I konw some of them have been either replaced by or removed from V1, Do you know where I can find a change list for all the difference? For example, what is it in V2 for DepthImageFrame.PlayerIndexBitmask in V1
Hi Jony. There is now equivalent to the PlayerIndexBitmask. Instead, you can use the BodyIndexFrame that is new to Kinect v2. BodyIndexFrame works like the color, depth and infrared frames and provides useful information about the bodies. Check the CoordinateMapping Basics example from the Kinect v2 samples.
hi ..
how could i program Kinect v2 to compare the user movement with pre-stored movement . & how could i store movement & whats the best method to add graphics and with which graphic program ?!
PS : i’m new to programming stuff ๐
Hi Huda,
Microsoft has already provided us with the tools for comparing gestures. Launch the SDK Browser (it was installed along with the SDK), click Tools and select the Visual Gesture Builder. Then click C# and select the Descrete Gesture Basics sample app.
You can watch a full tutorial here: http://channel9.msdn.com/Blogs/k4wdev/Custom-Gestures-End-to-End-with-Kinect-and-Visual-Gesture-Builder
Apologies for my late replies โ I have been sick for the past few daysโฆ
hi
Recently i purchase a new kinect for windows (V2). i need to capture depth data. therefore i installed SDK 2 for this reason. i captured depth data with Depth D2D or WPF in SDK 2. but i see the depth data that kinect captured,is very different from real depth. also we can see that many depth data will change suddenly into a random value of pixel(something like above 238, 140 or 3). And we also find that many black and white pixel appear on the display which also means that there exists an extreme sudden change between zero and a great value.
What makes this happen and how can I filter it if I don’t need this sudden change?
Hi Vahid. I suggest you used the Infrared frame instead. The Infrared frame will give you distances, as an array of ushort. I think it would work much better.
Hi Vangos,
Thanks for writing a truly amazing blog on Kinect!
I am very new to Kinect. I just purchase XBox One + Kinect Sensor. Only I need to purchase Kinect Adapter to connect with PC. I need to start a project which will work on many POS (Point of Sale) with a common data center. The application will capture the human motions, body tracking etc. and send it to the data center.
Can you please suggest me should I go with WPF application or with HTML5 applications. Which one is preferable for my purpose?
I have my PC with WIndows 8.1 and SDK 2.0 installed.
Thanks!
Ashish
Hi Ashish,
Thank you very much for your comment. Kinect uses a lot of CPU resources. So, I suggest you go for a native (WPF) solution. You can transmit data over the web, but that would make your applications slightly slower.
Sir kindly give me link how to show color frame in window store app………..
You can use the code from the ColorBitmapGenerator class.
Hello๏ผVangos. I’d like to make a WPF application to record the audio data, color data, depth data and face points data to the local and I use a ConcurrentQueue to save color, depth, face points arrays converted from the frame in the memory and then write to a bin file, after the window app finished, I extract the bin file to get the color data, depth data and face points data. The audio data is written to the local machine simply using the FileStream class: this.audioFile.Write(audioBuffer, 0, audioBuffer.Length). But I find that the audio is always missing several seconds, how could I save these data while the data won’t get missed? I would be quite apperaciated if you could give me some advice!๐
Hello. You can check the Audio sample of the Microsoft Kinect SDK Browser (bundled with the SDK). I would recommend running the recording process on a separate thread to avoid overloading the main thread.