Skip to main content

OK, you have requested this tutorial over a hundred times. You have been asking for this via email, social media, and private messages. So, today, I’ll show you how to animate a 3D body model using Kinect. To make things easy, we’ll be using Vitruvius throughout this tutorial. Let’s get started.

What is Avateering?

A 3D model can mimic and follow the movements of a person standing in front of a Kinect sensor. A player can control one or multiple 3D models (avatars). Here’s the difference between controlling one or many avatars:

Prerequisites

Source Code

  • The complete Unity source code demo is included in Vitruvius.

Step 1: Create a new Unity project

Launch Unity and create a new project, or open an existing Kinect project. We are using Unity because it has awesome 3D support.

Vitruvius includes a handy .unitypackage file. You can double-click or import that package in your project. The package includes Windows.Kinect.dll and LightBuzz.Vitruvius.dll. Those are the required assemblies to use the complete power of Vitruvius.

Step 2: Import the 3D models

Open your scene file in the Editor and import the 3D models you want to animate. Place them whenever you think it’s necessary for your game. I have imported the male and female 3D body models. They are named “male” and “female”, respectively.

This is how my editor looks like:

Vitruvius Avateering Unity Editor

Step 3: Action!

It’s now time to write your Kinect script. You’ll need to include the following namespaces:

using Windows.Kinect;
using LightBuzz.Vitruvius;

After importing the assemblies, we’ll create the required members to open the sensor, acquire the Body data and update the avatars accordingly.

The separatedPlayers variable indicates whether a single player will control one avatar or multiple avatars.

Members

// Kinect
BodyFrameReader bodyReader;
Body[] users;
// 3D models
Model[] models;
public FBX female;
public FBX male;
// Determines whether
// one player will control all avatars, or
// whether each player will control one avatar
public bool seperatedPlayers;

Start

The Start method initializes Kinect and Vitruvius. It’s important to open the sensor and enable avateering functionality to your 3D models. The code is really simple and self-explanatory:

void Start()
{
    // 1. Initialize Kinect
    sensor = KinectSensor.GetDefault();
    if (sensor != null)
    {
        bodyReader = sensor.BodyFrameSource.OpenReader();
        sensor.Open();
    }
    // 2. Enable Avateering
    Avateering.Enable();
    // 3. Specify the 3D models to animate.
    models = new Model[]
    {
        female,
        male
    };
    // 4. Initialize each 3D model.
    for (int i = 0; i < models.Length; i++)
    {
        models[i].Initialize();
    }
}

Update

Now, it’s time for the cool part! To see your avatars animated, you simply have to loop through the tracked bodies and update the corresponding avatars. Vitruvius does this with just one line of code.

void Update()
{
    if (bodyReader != null)
    {
        using (var frame = bodyReader.AcquireLatestFrame())
        {
            if (frame != null)
            {
                users = frame.Bodies().
                        Where(b => b.IsTracked).ToArray();
                if (seperatedPlayers)
                {
                    // Each user controls a different avatar.
                    for (int index = 0; index < users.Length; index++)
                    {
                       Model model = models[index];
                       Body user = users[index];
                       
                       // Yes, this line does ALL of the hard work for you.
                       Avateering.Update(model, user);
                   }
                } else if (users.Length > 0)
                {
                    // A single user controls all of the avatars.
                    for (int index = 0; index < models.Length; index++)
                    {
                        Model model = models[index];
                        Body user = users[0];
                        // Yes, this line does ALL of the hard work for you.
                        Avateering.Update(model, user);
                    }
                }
            }
        }
    }
}

Dispose

Finally, do not forget to dispose Kinect and clean any resources.

public void Dispose()
{
    // 1. Dispose Kinect
    if (bodyReader != null)
    {
        bodyReader.Dispose();
        bodyReader = null;
    }
    if (sensor != null)
    {
        if (sensor.IsOpen)
        {
            sensor.Close();
        }
        sensor = null;
    }
    // 2. Dispose 3D models
    for (int i = 0; i < models.Length; i++)
    {
        models[i].Dispose();
    }
    // 3. Disable avateering
    Avateering.Disable();
}

This is it! You have just developed an avateering app using Kinect v2!

BONUS – New 3D models

Vitruvius is bundled with nine 3D models you can use in your commercial projects. We are adding new 3D models all the time. Our models are perfectly rigged and work with Kinect just fine. Each model is coming in multiple textures. to modify a texture, simply open the Samples/Models/FBX folder and pick the 3D models and textures of your choice.

The following models are included:

  • Female Casual – a beautiful female model with various hair colors and clothes.
  • Male Casual – a male model with various clothes.
  • Female Athletic – a female model with sports clothing.
  • Male Athletic – a male model with sports clothing.
  • Kid – a 3D model of a kid.
  • Skeleton – a fantastic model of a human skeleton with bones and skull!
  • Longsleeve – a shirt with long sleeves and various colors.
  • Tshirt – a simple t-shirt model with various colors.
  • Pants – a pair of trousers with various styles (jeans, black, formal).

Bodies

Vitruvius Kinect Models Bodies

Clothes

Vitruvius Kinect Fitting Room

So, what will you do with Vitruvius? Let me know in the comments below!

Vitruvius

If you enjoyed this post, consider checking Vitruvius. Vitruvius is a set of powerful Kinect extensions that will help you build stunning Kinect apps in minutes. Vitruvius includes avateering, HD Face, background removal, angle calculations, and more. Check it now.

Vangos Pterneas

Vangos Pterneas is a software engineer, book author, and award-winning Microsoft Most Valuable Professional (2014-2019). Since 2012, Vangos has been helping Fortune-500 companies and ambitious startups create demanding motion-tracking applications. He's obsessed with analyzing and modeling every aspect of human motion using AI and Maths. Vangos shares his passion by regularly publishing articles and open-source projects to help and inspire fellow developers.

12 Comments

  • Kyle says:

    The Vitruvius library has been incredibly easy to use so far in our project to combine Kinect gesture detection with HoloLens. I don’t think it can be simplified more than Vangos has done. I highly recommend it.

  • Tony says:

    Hello! I’ve purchased the premium version and am trying to follow this tutorial, and imported the packages to Unity. However, I cannot find the free models anywhere, just something called “Codeman.” Where should I be looking? (If it’s worth noting, I don’t have Unity Pro, although it doesn’t seem like anything bad has happened yet).

  • hi can i use my own models with this plugin

    • Yes! Any rigged humanoid model can be used with the plugin.

      • hi before i buy that,i have some Questions :
        1. is Avateering updated for Unity Version 2019.1.1f1?
        2. do i have an option to use spine bone instead of root bone, i dont like the full avatar Transform with Root bones, looks bad like my tests at least(https://youtu.be/AIbVZZoU6EI)
        3. is it scaled to avatar size, so can i control small characters without any ultra long arms and legs?
        4. i like your Project, do you still improve it? when did you changed your code at least?
        5. can i use it for commercial ?

  • ian says:

    hello, i have got a academic version of avateering, any suggestion to use in multiplayers?

    • Hello Ian. The Avateering sample included in the Academic version can be used in multiplayer scenarios. All you need to do is use a different Body object as an argument:


      Body b1 = frame.BodyData.Left();
      Body b2 = frame.BodyData.Right();

      model1.DoAvateering(b1);
      model2.DoAvateering(b2);

  • emma says:

    I want to program the visualisation of X,Y and Z of a joints on a able and visualizing them in curves
    how can i do that?
    I’m beginner and I WANT HELP

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.