OK, you have requested this tutorial over a hundred times. You have been asking for this via email, social media, and private messages. So, today, I’ll show you how to animate a 3D body model using Kinect. To make things easy, we’ll be using Vitruvius throughout this tutorial. Let’s get started.
What is Avateering?
A 3D model can mimic and follow the movements of a person standing in front of a Kinect sensor. A player can control one or multiple 3D models (avatars). Here’s the difference between controlling one or many avatars:
Prerequisites
- Kinect for XBOX v2 sensor with an adapter (or Kinect for Windows v2 sensor)
- Kinect for Windows v2 SDK
- Windows 8.1 or higher
- Visual Studio 2013 or higher
- A dedicated USB 3 port
Source Code
- The complete Unity source code demo is included in Vitruvius.
Step 1: Create a new Unity project
Launch Unity and create a new project, or open an existing Kinect project. We are using Unity because it has awesome 3D support.
Vitruvius includes a handy .unitypackage file. You can double-click or import that package in your project. The package includes Windows.Kinect.dll and LightBuzz.Vitruvius.dll. Those are the required assemblies to use the complete power of Vitruvius.
Step 2: Import the 3D models
Open your scene file in the Editor and import the 3D models you want to animate. Place them whenever you think it’s necessary for your game. I have imported the male and female 3D body models. They are named “male” and “female”, respectively.
This is how my editor looks like:
Step 3: Action!
It’s now time to write your Kinect script. You’ll need to include the following namespaces:
using Windows.Kinect;
using LightBuzz.Vitruvius;
After importing the assemblies, we’ll create the required members to open the sensor, acquire the Body data and update the avatars accordingly.
The separatedPlayers variable indicates whether a single player will control one avatar or multiple avatars.
Members
// Kinect
BodyFrameReader bodyReader;
Body[] users;
// 3D models
Model[] models;
public FBX female;
public FBX male;
// Determines whether
// one player will control all avatars, or
// whether each player will control one avatar
public bool seperatedPlayers;
Start
The Start method initializes Kinect and Vitruvius. It’s important to open the sensor and enable avateering functionality to your 3D models. The code is really simple and self-explanatory:
void Start()
{
// 1. Initialize Kinect
sensor = KinectSensor.GetDefault();
if (sensor != null)
{
bodyReader = sensor.BodyFrameSource.OpenReader();
sensor.Open();
}
// 2. Enable Avateering
Avateering.Enable();
// 3. Specify the 3D models to animate.
models = new Model[]
{
female,
male
};
// 4. Initialize each 3D model.
for (int i = 0; i < models.Length; i++)
{
models[i].Initialize();
}
}
Update
Now, it’s time for the cool part! To see your avatars animated, you simply have to loop through the tracked bodies and update the corresponding avatars. Vitruvius does this with just one line of code.
void Update()
{
if (bodyReader != null)
{
using (var frame = bodyReader.AcquireLatestFrame())
{
if (frame != null)
{
users = frame.Bodies().
Where(b => b.IsTracked).ToArray();
if (seperatedPlayers)
{
// Each user controls a different avatar.
for (int index = 0; index < users.Length; index++)
{
Model model = models[index];
Body user = users[index];
// Yes, this line does ALL of the hard work for you.
Avateering.Update(model, user);
}
} else if (users.Length > 0)
{
// A single user controls all of the avatars.
for (int index = 0; index < models.Length; index++)
{
Model model = models[index];
Body user = users[0];
// Yes, this line does ALL of the hard work for you.
Avateering.Update(model, user);
}
}
}
}
}
}
Dispose
Finally, do not forget to dispose Kinect and clean any resources.
public void Dispose()
{
// 1. Dispose Kinect
if (bodyReader != null)
{
bodyReader.Dispose();
bodyReader = null;
}
if (sensor != null)
{
if (sensor.IsOpen)
{
sensor.Close();
}
sensor = null;
}
// 2. Dispose 3D models
for (int i = 0; i < models.Length; i++)
{
models[i].Dispose();
}
// 3. Disable avateering
Avateering.Disable();
}
This is it! You have just developed an avateering app using Kinect v2!
BONUS – New 3D models
Vitruvius is bundled with nine 3D models you can use in your commercial projects. We are adding new 3D models all the time. Our models are perfectly rigged and work with Kinect just fine. Each model is coming in multiple textures. to modify a texture, simply open the Samples/Models/FBX folder and pick the 3D models and textures of your choice.
The following models are included:
- Female Casual – a beautiful female model with various hair colors and clothes.
- Male Casual – a male model with various clothes.
- Female Athletic – a female model with sports clothing.
- Male Athletic – a male model with sports clothing.
- Kid – a 3D model of a kid.
- Skeleton – a fantastic model of a human skeleton with bones and skull!
- Longsleeve – a shirt with long sleeves and various colors.
- Tshirt – a simple t-shirt model with various colors.
- Pants – a pair of trousers with various styles (jeans, black, formal).
Bodies
Clothes
So, what will you do with Vitruvius? Let me know in the comments below!
Vitruvius
If you enjoyed this post, consider checking Vitruvius. Vitruvius is a set of powerful Kinect extensions that will help you build stunning Kinect apps in minutes. Vitruvius includes avateering, HD Face, background removal, angle calculations, and more. Check it now.
The Vitruvius library has been incredibly easy to use so far in our project to combine Kinect gesture detection with HoloLens. I don’t think it can be simplified more than Vangos has done. I highly recommend it.
Thank you very much Kyle. I’m glad you enjoy Vitruvius. Feel free to email me information about your project when ready, so I can feature it on Vitruvius.
Hello! I’ve purchased the premium version and am trying to follow this tutorial, and imported the packages to Unity. However, I cannot find the free models anywhere, just something called “Codeman.” Where should I be looking? (If it’s worth noting, I don’t have Unity Pro, although it doesn’t seem like anything bad has happened yet).
hi can i use my own models with this plugin
Yes! Any rigged humanoid model can be used with the plugin.
hi before i buy that,i have some Questions :
1. is Avateering updated for Unity Version 2019.1.1f1?
2. do i have an option to use spine bone instead of root bone, i dont like the full avatar Transform with Root bones, looks bad like my tests at least(https://youtu.be/AIbVZZoU6EI)
3. is it scaled to avatar size, so can i control small characters without any ultra long arms and legs?
4. i like your Project, do you still improve it? when did you changed your code at least?
5. can i use it for commercial ?
hello i forget 1 Question : is it possible to stream it in realtime ?
hello, i have got a academic version of avateering, any suggestion to use in multiplayers?
Hello Ian. The Avateering sample included in the Academic version can be used in multiplayer scenarios. All you need to do is use a different Body object as an argument:
Body b1 = frame.BodyData.Left();
Body b2 = frame.BodyData.Right();
model1.DoAvateering(b1);
model2.DoAvateering(b2);
Thank you Vangos, it works well. Btw, Can avatars jump on Vitruvius 5.4.0? Thank you in advance:)
I want to program the visualisation of X,Y and Z of a joints on a able and visualizing them in curves
how can i do that?
I’m beginner and I WANT HELP
Hi Emma. You can get started with the official tutorials here:
http://kinect.github.io/tutorial/