Skip to main content

A few days ago, I had the honor to speak at the IT Pro | Dev Connections 2020 conference. As a software engineer, I love natural user interfaces, body tracking, and motion analysis. It’s no surprise that my presentation is focusing on the Azure Kinect. We’ll explore how the interaction between humans and machines evolved until we reach the era of Kinect and natural user interaction. You’ll have the chance to explore the anatomy of the sensor and investigate its discrete components. Amazing things happen when great hardware meets Artificial Intelligence. Our coding session includes body-tracking, angle calculations, and 3D point clouds! Last but not least, my friend Georgia Makoudi is connecting Kinect to the Cloud and enhances it with object tracking capabilities!

You can watch our 45-minute presentation here. I’m also attaching my presentation and notes below.

We are starting our journey with one of the very first computers – ENIAC. Programming the ENIAC was not trivial. Honestly, the original programmers were truly heroes.

ENIAC Women Programmers

Computers evolved, and so did the interaction with humans. Computer Scientists invented revolutionary means of interaction, such as the keyboard, mouse, and touch screen. In the meantime, however, Science Fiction was always a step forward. Iconic movies & series such as Star Wars, Star Trek, and Minority Report brought us to a world where people interacted with computers via holograms, gestures, and voice commands. What a marvelous world! For many decades, humanity believed that was only a fairy tale.

That’s until Microsoft revealed the Kinect. Unlike what you might think, Kinect is not a single device; instead, it’s a bundle of multiple components grouped together. These components include:

  • 1 video camera
  • 1 depth sensor
  • 7 microphones
  • 1 accelerometer & 1 gyroscope
Azure Kinect - Sensor Anatomy

You can configure the video camera to output a resolution of 720p up to 4K. Internally, the output is simply an RGBA array of bytes.

Azure Kinect Color Frame Pixels

The truly revolutionary module is the depth sensor. The depth sensor is an infrared emitter that can see the world in 3D. It supports a Wide and a Narrow field of view.

Azure Kinect Depth - NFOV and WFOV

The combination of depth data and cutting-edge Artificial Intelligence produces magic. Here’s how the Kinect SDK estimates the coordinates of the human body joints in three-dimensional space…

Azure Kinect Body Tracking 3D

… allowing us to see the human body from multiple angles!

Azure Kinect Body Tracking Unity3D – 3D Skeletons

Things can get even more exciting. What would happen if you combined the color, depth, and body-tracking data all at once? Well, here is a 3D point cloud that will blow your mind:

Azure Kinect Point Cloud

But, wait, there’s more! Azure Kinect is named “Azure” for a reason. Azure is a powerful cloud solution by Microsoft. Part of its functionality includes a set of APIs that provide remote AI Computer Vision solutions. Connect Kinect to Azure, and you’ll have a ton of additional functionality, such as real-time object detection!

Azure Cognitive Services - Computer Vision Object Detection (Living Room)

One more thing…

After leaving my job at Microsoft, I have been helping Fortune-500 companies and innovative startups create amazing body-tracking applications and games. If you are looking to get your business to the next level, get in touch with me.

Sharing is caring!

If you liked this article, remember to share it on social media, so you can help other developers, too! Also, let me know your thoughts in the comments below. ‘Til the next time… keep coding!

Vangos Pterneas

Vangos Pterneas is a software engineer, book author, and award-winning Microsoft Most Valuable Professional (2014-2019). Since 2012, Vangos has been helping Fortune-500 companies and ambitious startups create demanding motion-tracking applications. He's obsessed with analyzing and modeling every aspect of human motion using AI and Maths. Vangos shares his passion by regularly publishing articles and open-source projects to help and inspire fellow developers.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.