Using Azure Kinect in Unity3D

The Azure Kinect SDK received a major update a few days ago. The update introduced great new features, such as support for non-NVIDIA graphics cards. However, it came with a major issue: as people reported in Microsoft’s Kinect repository, it does not work with Unity3D any longer. That’s sad news for the majority of the Kinect developers out there. Thankfully for you, though, I’ve got the solution. Follow this tutorial to make Kinect compatible with Unity3D again.

Of course, if you want to automate the process and have a plug-and-play solution, consider my Asset Store Kinect plugin. Otherwise, read on to learn exactly how to use Azure Kinect with Unity3D

What’s the problem?

Simply put: up until version 1.4.1 of the Azure Kinect Body Tracking SDK, all you needed to do was drag and drop the C++ binary files from the SDK installation folder into your Unity project. Since version 1.4.1, that approach no longer works. You can’t debug Kinect apps in the Unity Editor and you can’t build Windows executables. Why? Microsoft has introduced a lot of new C++ DLL files to make the SDK compatible with a broad range of graphics cards. Those DLLs have conflicts with DLLs used by Unity3D. To overcome the problem, we need to copy the SDK binaries to various locations.

Configuration

To run Azure Kinect in Unity3D, you need the following:

• A PC with an adequate GPU
• Azure Kinect Sensor SDK
• Azure Kinect Body Tracking SDK
• Unity3D 2020 LTS
• Visual Studio 2019
• Windows 10

Fix!

You’ve got everything set up and working? Great! Let’s move to the actual fix. First, we’ll collect the required C# and C++ binaries. Then, we’ll make it work in the Editor, and then we’ll ensure it’s gonna work as a Windows build.

Collecting the binaries

First, you need to collect the required binary files. Here’s where to find them:

• Download and install the Azure Kinect Sensor SDK for Windows.
• Download and install the Azure Kinect Body Tracking SDK for Windows.

The required DLL binaries are located in the following folders:

• C:\Program Files\Azure Kinect Body Tracking SDK\tools
• C:\Program Files\Azure Kinect Body Tracking SDK\sdk\netstandard2.0\release
• C:\Program Files\Azure Kinect SDK v1.4.1\sdk\netstandard2.0\release

Copy all of the .dll and .onnx files from the above locations and move to the next step. You should have 21 .dll files and 2 .onnx files. That’s right: 23 files in total.

Playing in the Editor

To run Azure Kinect apps in the Editor, you need to paste the above files next to your Assets project folder. Do not place them inside the Plugins/x86_64 folder. That’s not gonna work. Here’s how my project folder looks like:

Modifying the Unity installation folder

That’s the tricky part now. You should manually copy the DirectML.dll file into the installation folder of Unity. In my case, it is C:\Program Files\Unity\Hub\Editor\2020.3.0f1\Editor.

It’s recommended to reboot your computer after this step!

Building your project

The above method will make Kinect work in the Editor. After you build and export your program as a Windows executable, you also need to copy the DLL files next to your .exe file.

Remember to set the Build Architecture to x86_64.

Summary

Made it to this point? Congratulations! You can now run the latest Azure Kinect SDK in Unity3D. It’s a cumbersome process, but it’s totally worth it. Again, if you want to avoid the hustle, just download my Asset Store Kinect plugin that automates the process for you. It’s really plug-and-play.

‘Til the next time, keep Kinecting!