Vitruvius: Developing Kinect Apps in Minutes

During the past few months, I have been quite inactive. It was for a good reason, though. My team and I have been working on an ambitious project that helps Kinect developers build apps faster. I would like to introduce Vitruvius.

What is Vitruvius?

During the past few years, Kinect developers from all over the world have been creating outstanding Kinect apps, utilizing the power of Kinect SDK 2, Computer Vision, complex Mathematics, and Linear Algebra. Kinect development is hard and demanding. Through my blog posts (and your 600 comments), I have been trying to simplify the process and make it easier for students, researchers and engineers.

Kinect development should be fun.

So, after three years of professional experience and a lot of successful commercial projects, I developed a framework that accomplishes hard tasks using a few lines of code. Please welcome Vitruvius.

Vitruvius is a set of tools, utilities, controls and extensions that simplify Kinect development.

Update: Vitruvius is featured in the Official Microsoft Kinect Blog!

Features

Vitruvius is already used by big companies and institutions out there. I was proud to learn that NASA, XEROX and Michigan State University are actively using Vitruvius into their internal research projects.

Here’s exactly what’s included:

Avateering (3D model animation)

Animating a 3D model using Kinect is a very tough process. Not any more. Vitruvius lets you animate your FBX rigid models with one line of code! Never worry about the joint orientations or the complex Mathematics you need to apply. The following line of code animates a 3D model:

Avateering.Update(model, body);

… and this is the result (supports up to 6 people concurrently):

[youtube code=”_UMDyHIYPLE”]

Vitruvius is bundled with 1 female and 1 male 3D model with 10 textures each. Load the FBX models, choose the skin, hair and clothes of your choice and go. Moreover, the samples include a virtual fitting room demo with 3 additional cloth models (a long-sleeve shirt, a pair of pants and a t-shirt) with 3 textures each. You are free to use them in your personal or commercial projects. The 3D models are customizable, so you can keep the internal Kinect skeleton and have your 3D artist modify the look and feel accordingly.

[Tweet “Kinect development is FUN with Vitruvius”]

Angle calculations

Calculating the angle between selected joints is also a matter of one line of C# code. Calculating an angle will help you take human body measurements with accuracy. You can specify the target axis (X, Y, Z) or simply calculate the angle in the 3D space. Vitruvius also provides you with a handy Arc control to display one or multiple angles on top of a body:

Watch video

The Arc control is available for both XAML and Unity.

HD Face Extensions

Kinect SDK includes the most powerful Face API in the world. Face and HD Face APIs lets you access over 1,000 facial points. Vitruvius provides an easy-to-use Face class that is populated with Eyes, Nose, Chin, Cheeks, Jaw and Forehead properties. No need to mess with the thousands of points. Accessing the facial properties has never been more straightforward:

var face = faceFrame.Face();
var forehead = face.Forehead;
var nose = face.Nose;
var jaw = face.Jaw;

Watch video

Bitmap manipulation & Background removal

Converting a Kinect RGB, depth, or infrared frame into an image is, indeed, confusing. Bitmaps are handled by each programming platform. For example, XAML represents bitmaps using the WriteableBitmap class. Unity supports Texture2D. Vitruvius supports everything! We access the raw data of any Kinect frame and convert it to the corresponding image format.

var bitmap = colorFrame.ToBitmap();

But people need more than just displaying an image. Using Vitruvius, you can accurately display the body on top of the Color, Depth, or Infrared frames, or even remove the background.

var bitmap = colorFrame.GreenScreen(depthFrame, bodyIndexFrame);

Camera video – Background Removal video

Coordinate Mapping

Kinect developers have been struggling to convert points between the 3D world-space and the 2D screen-space. Kinect SDK includes CoordinateMapper, a useful tool that converts between camera space and depth/color space. Vitruvius simplifies Coordinate Mapping with the Vector and Point functions:

var joint = body.Joints[JointType.Head].Position;
var point = joint.ToPoint(Visualization.Color);

Body Extensions

Handling body data effectively is a core part of my Kinect philosophy. A lot of people ask me how to identify the closest body, the number of the tracked joints, or the height of the players. The built-in Body class provides a lot of great information, but there is always something missing. Vitruvius fills these gaps and gives you everything you need to create a solid experience for your NUI applications:

// Get the closest body.
var body = frame.Bodies().Closest();
// Calculate the height in meters.
double height = body.Height();
// Find when someone entered or left the scene.
var controller = new PlayersController();
controller.PlayerEntered += Player_Entered;
controller.PlayerLeft += Player_Left;

Gesture detection

Natural User Interfaces are strongly relying on gestures. Gestures help users accomplish tasks easily using their hands or body. Vitruvius supports waving, swiping, and zooming.

void GestureRecognized(object sender, GestureEventArgs e)
{
   var gesture = e.GestureType;
}

Documentation & Samples

Vitruvius supports WPF, Windows Store and Unity3D. There is extensive MSDN-like documentation for every platform.

• Documentation for WPF and Windows Store
• Documentation for Unity3D

The following demos are included in the Download section:

• Avateering
• Angles
• Face
• Fitting Room
• Frame View
• Gestures
• Green Screen

Additionally, Vitruvius integrates amazingly well with your existing projects, too. No need to re-invent the wheel or change your coding style. Simply call any function you want from your existing code. It’s that simple!

Performance

We are taking performance seriously. Vitruvius is designed to use as few resources as possible. It only consumes what’s really necessary. There are no duplicates or unnecessary copies. Everything is real-time. As long as your computer meets the standard Kinect SDK requirements, you can use Vitruvius at no risk.

Who am I?

Ready? Download!

Vitruvius will empower your business or research and let you develop the next cutting-edge Kinect apps. Supports WPF, Windows Store, and Unity3D. NASA (yeap, the space guys), XEROX, and Michigan State University are already including Vitruvius in their workflow. It took me over 6 months to create this product. However, as a Microsoft MVP, I do not want to charge you a ton of money to access the cool features of Vitruvius. Kinect development should be accessible to everyone. This is why I am offering a free version that will let you try some of the aforementioned features at no cost.

• Students, Professors and Researchers can benefit from the Academic version. The Academic version includes everything: avateering, angles, fitting room, bitmaps, gestures, HD Face. You can use it for research projects, thesis, and labs.
• Small businesses and independent developers can benefit from Vitruvius Premium. The Premium version also includes everything, plus access to free updates and commercial support. Think about it: using the Premium version, you can spend a tiny amount of money and impress your customers with stunning applications!
• Finally, bigger corporations can purchase Vitruvius Platinum and access all of the above benefits, plus dedicated telephone support. I am also offering an 1-on-1 consulting session (valued $280) that will help you get your business to the next level.

That’s all for now. If you enjoyed my Kinect tutorials, Vitruvius is for you.

Keep coding. Keep Kinecting.

Next Steps

• Visit Vitruvius Website
• Watch Vitruvius Videos

https://youtube.com/watch?v=videoseries%3Flist%3DPLDSXTeae1h8e9uFQXzU3XqXpnAymG0wNG

[Tweet “Download Vitruvius, the most advanced Kinect framework ever.”]