Vitruvius: Developing Kinect Apps in Minutes

By September 26, 2015C#, CodeProject, Kinect, Unity, Videos
Vitruvius Kinect by Vangos Pterneas

During the past few months, I have been quite inactive. It was for a good reason, though. My team and I have been working on an ambitious project that helps Kinect developers build apps faster. I would like to introduce Vitruvius.

What is Vitruvius?

During the past few years, Kinect developers from all over the world have been creating outstanding Kinect apps, utilizing the power of Kinect SDK 2, Computer Vision, complex Mathematics, and Linear Algebra. Kinect development is hard and demanding. Through my blog posts (and your 600 comments), I have been trying to simplify the process and make it easier for students, researchers and engineers.

Kinect development should be fun.

So, after three years of professional experience and a lot of successful commercial projects, I developed a framework that accomplishes hard tasks using a few lines of code. Please welcome Vitruvius.

Vitruvius is a set of tools, utilities, controls and extensions that simplify Kinect development.

Update: Vitruvius is featured in the Official Microsoft Kinect Blog!

Features

Vitruvius is already used by big companies and institutions out there. I was proud to learn that NASA, XEROX and Michigan State University are actively using Vitruvius into their internal research projects.

Here’s exactly what’s included:

Avateering (3D model animation)

Animating a 3D model using Kinect is a very tough process. Not any more. Vitruvius lets you animate your FBX rigid models with one line of code! Never worry about the joint orientations or the complex Mathematics you need to apply. The following line of code animates a 3D model:

Avateering.Update(model, body);

… and this is the result (supports up to 6 people concurrently):

[youtube code=”_UMDyHIYPLE”]

Vitruvius is bundled with 1 female and 1 male 3D model with 10 textures each. Load the FBX models, choose the skin, hair and clothes of your choice and go. Moreover, the samples include a virtual fitting room demo with 3 additional cloth models (a long-sleeve shirt, a pair of pants and a t-shirt) with 3 textures each. You are free to use them in your personal or commercial projects. The 3D models are customizable, so you can keep the internal Kinect skeleton and have your 3D artist modify the look and feel accordingly.

Vitruvius Kinect Fitting Room

Angle calculations

Calculating the angle between selected joints is also a matter of one line of C# code. Calculating an angle will help you take human body measurements with accuracy. You can specify the target axis (X, Y, Z) or simply calculate the angle in the 3D space. Vitruvius also provides you with a handy Arc control to display one or multiple angles on top of a body:

Vitruvius Kinect Angle
Watch video

The Arc control is available for both XAML and Unity.

HD Face Extensions

Kinect SDK includes the most powerful Face API in the world. Face and HD Face APIs lets you access over 1,000 facial points. Vitruvius provides an easy-to-use Face class that is populated with Eyes, Nose, Chin, Cheeks, Jaw and Forehead properties. No need to mess with the thousands of points. Accessing the facial properties has never been more straightforward:

var face = faceFrame.Face();
var forehead = face.Forehead;
var nose = face.Nose;
var jaw = face.Jaw;

Vitruvius Kinect HD Face

Watch video

Bitmap manipulation & Background removal

Converting a Kinect RGB, depth, or infrared frame into an image is, indeed, confusing. Bitmaps are handled by each programming platform. For example, XAML represents bitmaps using the WriteableBitmap class. Unity supports Texture2D. Vitruvius supports everything! We access the raw data of any Kinect frame and convert it to the corresponding image format.

var bitmap = colorFrame.ToBitmap();

But people need more than just displaying an image. Using Vitruvius, you can accurately display the body on top of the Color, Depth, or Infrared frames, or even remove the background.

var bitmap = colorFrame.GreenScreen(depthFrame, bodyIndexFrame);

Vitruvius Kinect - Color, Depth, Infrared, Body

Camera videoBackground Removal video

Coordinate Mapping

Kinect developers have been struggling to convert points between the 3D world-space and the 2D screen-space. Kinect SDK includes CoordinateMapper, a useful tool that converts between camera space and depth/color space. Vitruvius simplifies Coordinate Mapping with the Vector and Point functions:

var joint = body.Joints[JointType.Head].Position;
var point = joint.ToPoint(Visualization.Color);

Body Extensions

Handling body data effectively is a core part of my Kinect philosophy. A lot of people ask me how to identify the closest body, the number of the tracked joints, or the height of the players. The built-in Body class provides a lot of great information, but there is always something missing. Vitruvius fills these gaps and gives you everything you need to create a solid experience for your NUI applications:

// Get the closest body.
var body = frame.Bodies().Closest();

// Calculate the height in meters.
double height = body.Height();

// Find when someone entered or left the scene.
var controller = new PlayersController();
controller.PlayerEntered += Player_Entered;
controller.PlayerLeft += Player_Left;

Gesture detection

Natural User Interfaces are strongly relying on gestures. Gestures help users accomplish tasks easily using their hands or body. Vitruvius supports waving, swiping, and zooming.

void GestureRecognized(object sender, GestureEventArgs e)
{
   var gesture = e.GestureType;
}

Documentation & Samples

Vitruvius supports WPF, Windows Store and Unity3D. There is extensive MSDN-like documentation for every platform.

The following demos are included in the Download section:

  • Avateering
  • Angles
  • Face
  • Fitting Room
  • Frame View
  • Gestures
  • Green Screen

Additionally, Vitruvius integrates amazingly well with your existing projects, too. No need to re-invent the wheel or change your coding style. Simply call any function you want from your existing code. It’s that simple!

Performance

We are taking performance seriously. Vitruvius is designed to use as few resources as possible. It only consumes what’s really necessary. There are no duplicates or unnecessary copies. Everything is real-time. As long as your computer meets the standard Kinect SDK requirements, you can use Vitruvius at no risk.

Who am I?

Vangos Pterneas Kinect MVP

I guess that, since you are reading this blog post, you already know me of my Kinect tutorials, videos, and guides. I’m Vangos Pterneas, Microsoft Kinect Most Valuable Professional. I have been building commercial Kinect apps since 2012. Microsoft recognized my contributions to the open-source Kinect community and awarded me with the MVP title two years ago. My expertise includes C# and NUI app development. Vitruvius is crafted by a dedicated team of engineers and designers.

Ready? Download!

Vitruvius will empower your business or research and let you develop the next cutting-edge Kinect apps. Supports WPF, Windows Store, and Unity3D. NASA (yeap, the space guys), XEROX, and Michigan State University are already including Vitruvius in their workflow. It took me over 6 months to create this product. However, as a Microsoft MVP, I do not want to charge you a ton of money to access the cool features of Vitruvius. Kinect development should be accessible to everyone. This is why I am offering a free version that will let you try some of the aforementioned features at no cost.

  • Students, Professors and Researchers can benefit from the Academic version. The Academic version includes everything: avateering, angles, fitting room, bitmaps, gestures, HD Face. You can use it for research projects, thesis, and labs.
  • Small businesses and independent developers can benefit from Vitruvius Premium. The Premium version also includes everything, plus access to free updates and commercial support. Think about it: using the Premium version, you can spend a tiny amount of money and impress your customers with stunning applications!
  • Finally, bigger corporations can purchase Vitruvius Platinum and access all of the above benefits, plus dedicated telephone support. I am also offering an 1-on-1 consulting session (valued $280) that will help you get your business to the next level.

Free

$ 0

  • Bitmap Generators
  • Background Removal
  • Angle Calculations & Maths
  • Body Extensions
  • Gesture Detection
  • XAML Controls
  • WPF .NET
  • Windows Store
Download Free

Academic

$ 149

  • Bitmap Generators
  • Background Removal
  • Angle Calculations & Maths
  • Body Extensions
  • Gesture Detection
  • XAML Controls
  • WPF .NET
  • Windows Store
  • Unity3D
  • HD Face Extensions
  • Avateering
  • Fitting Room
  • Male & Female 3D models
Download Academic

PremiumMost Popular

$ 299

  • Bitmap Generators
  • Background Removal
  • Angle Calculations & Maths
  • Body Extensions
  • Gesture Detection
  • XAML Controls
  • WPF .NET
  • Windows Store
  • Unity3D
  • HD Face Extensions
  • Avateering
  • Fitting Room
  • Male & Female 3D models
  • COMMERCIAL LICENSE
Download Premium

Platinum

$ 899

  • Bitmap Generators
  • Background Removal
  • Angle Calculations & Maths
  • Body Extensions
  • Gesture Detection
  • XAML Controls
  • WPF .NET
  • Windows Store
  • Unity3D
  • HD Face Extensions
  • Avateering
  • Fitting Room
  • Male & Female 3D models
  • COMMERCIAL LICENSE
  • PHONE SUPPORT
  • 24-HOUR RESPONSE TIME
  • 1-HOUR FREE CONSULTING
Download Platinum

That’s all for now. If you enjoyed my Kinect tutorials, Vitruvius is for you.

Keep coding. Keep Kinecting.

Next Steps

https://youtube.com/watch?v=videoseries%3Flist%3DPLDSXTeae1h8e9uFQXzU3XqXpnAymG0wNG

Author Vangos Pterneas

Vangos Pterneas is an award-winning Microsoft Most Valuable Professional. He is helping companies from all over the world grow their revenue by creating profitable software products. He loves Motion Technology and Mixed Reality. Vangos is the CEO of LightBuzz Inc and author of The Dark Art Of Freelancing.

More posts by Vangos Pterneas

Join the discussion 20 Comments

  • vinayak says:

    hi vangos ,

    i started using your vitruvius frame work its really amazing one you made a-lot off stuff easy am using for my projects its helping a lot thanks for what you have done for kinect developers and am eagerly waiting for your book so when we can except it in the market ? thanks a lot

    vinayak

  • jeni says:

    Really awesome project. I think It make easy work to new students and developers. Is it free to all? Thanks for this. really nice…….

    • Hi Jeni. Thanks a lot for your comment. It really took us over 6 months to integrate all this functionality.

      The basic version (WPF and Windows Store) is free for everyone. If you need more advanced functionality, you can purchase the Academic or Premium version. These versions include Unity3D support, Avateering, and HD Face extensions. There are also 5 awesome 3D models that can be used in commercial projects.

      The Academic version can be used by a team – no need to buy a separate license for each member.

      The Premium version can be used by a company – no need to buy separate licenses.

  • Imran says:

    Hi,

    Can we get Front or back side of person standing in front of Kinect V2.? If possible please explain.

    Regards,

    • Hi Imran. Do you want to check whether a person is facing the sensor? You can use the HD Face and determine whether the Face of the person is visible or not.

      Vitruvius has a Face class that gives all of the required data of an HD Face.

  • Sanket says:

    I have detected multiple bodies (No.6) using your code but the bodies array holds the body in any order i.e first body arrived it put in bodies[4] second to bodies[2] and so on.
    Is this the correct behavior? Because i am facing some issue while performing the operation for the specific one.

    • Hi Sanket. Thanks for your comment. Kinect has no specific order for the bodies. It just gives you an array of 6 Body objects.

      Vitruvius has a useful method to grab the closest body. Simply type the following:

      Body closestBody = frame.Bodies().Closest();

      The Closest() method can be applied to any collection of Body objects.

  • Renivaldo says:

    Is it possible to develop Mirror hum to change clothes virtually ? How much is it?

  • Ambar says:

    Any chance of getting ready to build examples like height and motion tracking?

  • Tinh Phan Trung says:

    Hi Vangos Pterneas,

    I’m Trung Tinh, I come from Vietnam and i’m a student, i really like your project, it’s interesting.
    now i’m developing my project with unity and kinect v2, but i dont have enough money for Academic.
    i want to design a system rehabilitation for older people, please help me, i just need sample code about Unity in Academic.
    You can share me about sample code of Unity. I’m so grateful you and thank you so much.

  • Viraj Gandhi says:

    Hi Vangos Pterneas,

    Thanks for such and Awesome Library.

    While Calculating Angle By following:

    double angle = elbow.Angle(shoulder, wrist);

    The value we get in angle is always less then 180 while the negative values too gets unsigned.
    In your home page illustration(guy running), the angle of elbow is shown as 325.
    Please guide me as how to get full 360 degree for the same.

    Viraj Gandhi
    Student Fab Academy 2017
    India.

    • Hello Viraj. Thanks for your comment! The returned angle is, indeed less than 180 degrees. To get the “big” angle, simply reverse the start and end points:

      double angle = elbow.Angle(wrist, shoulder);

      Let me know if that helps.

  • Huang Qian says:

    Hi, Vangos. When I use your VITRUVIUS there have some questions in the example.
    In the XAML code, there are two codes and . I don’t understand the role of the two statements in the program and how to add it to my XAML program.
    Can you help me?

    controls:KinectViewer x:Name=”viewer” and controls:KinectAngle x:Name=”angle” Opacity=”0.5″

    • Hello, Huang. These are custom user controls by Vitruvius. These user controls provide some XAML functionality for you.

      The KinectViewer control provides a way to display the Color/Depth/Infrared streams, as well as a way to overlay the skeleton:


      viewer.Image = frame.ToBitmap();
      viewer.DrawBody(body);

      The KinectAngle control displays a visual Arc at the specified points:


      angle.Update(point1, point2, point3);

      To learn more about XAML User Controls, check this great article.

  • baher says:

    Hi ,

    can i track the positions of the joints while motion of the body using kinect and Vitruvius or it is just the height of the body , and if so , is it included in the free version or just the academic one .

    Thank you very much for your help .

    • Hello. You can track the positions of the joints using either the Free or the Academic version. The Academic version includes support for Unity3D, video recording, and avateering.

Leave a Reply