Azure Kinect in a Nutshell

A few days ago, I had the honor to speak at the IT Pro | Dev Connections 2020 conference. As a software engineer, I love natural user interfaces, body tracking, and motion analysis. It’s no surprise that my presentation is focusing on the Azure Kinect. We’ll explore how the interaction between humans and machines evolved until we reach the era of Kinect and natural user interaction. You’ll have the chance to explore the anatomy of the sensor and investigate its discrete components. Amazing things happen when great hardware meets Artificial Intelligence. Our coding session includes body-tracking, angle calculations, and 3D point clouds! Last but not least, my friend Georgia Makoudi is connecting Kinect to the Cloud and enhances it with object tracking capabilities!

You can watch our 45-minute presentation here. I’m also attaching my presentation and notes below.

We are starting our journey with one of the very first computers – ENIAC. Programming the ENIAC was not trivial. Honestly, the original programmers were truly heroes.

Computers evolved, and so did the interaction with humans. Computer Scientists invented revolutionary means of interaction, such as the keyboard, mouse, and touch screen. In the meantime, however, Science Fiction was always a step forward. Iconic movies & series such as Star Wars, Star Trek, and Minority Report brought us to a world where people interacted with computers via holograms, gestures, and voice commands. What a marvelous world! For many decades, humanity believed that was only a fairy tale.

That’s until Microsoft revealed the Kinect. Unlike what you might think, Kinect is not a single device; instead, it’s a bundle of multiple components grouped together. These components include:

• 1 video camera
• 1 depth sensor
• 7 microphones
• 1 accelerometer & 1 gyroscope

You can configure the video camera to output a resolution of 720p up to 4K. Internally, the output is simply an RGBA array of bytes.

The truly revolutionary module is the depth sensor. The depth sensor is an infrared emitter that can see the world in 3D. It supports a Wide and a Narrow field of view.

The combination of depth data and cutting-edge Artificial Intelligence produces magic. Here’s how the Kinect SDK estimates the coordinates of the human body joints in three-dimensional space…

… allowing us to see the human body from multiple angles!

Things can get even more exciting. What would happen if you combined the color, depth, and body-tracking data all at once? Well, here is a 3D point cloud that will blow your mind:

But, wait, there’s more! Azure Kinect is named “Azure” for a reason. Azure is a powerful cloud solution by Microsoft. Part of its functionality includes a set of APIs that provide remote AI Computer Vision solutions. Connect Kinect to Azure, and you’ll have a ton of additional functionality, such as real-time object detection!