Rob Relyea

Kinect for Windows – Details of API Changes from Beta2 to v1.0 (C#/VB)

robrelyea — Wed, 01 Feb 2012 12:01:29 +0000

[This article covers the details of changes of C#/VB accessible
APIs. Please ensure you read the code
migration post as well.]

Namespace and Assembly Name Changes

Since our Beta 1 (June 2011) release of the Kinect for
Windows SDK , our C#/VB accessible APIs have been provided via
Microsoft.Research.Kinect.dll. The public Types all were organized into 2 namespaces:
Microsoft.Research.Kinect.Nui and Microsoft.Research.Kinect.Audio.
Our team made the decision back in early 2011 that we would mark our beta
releases of APIs as “Research” to indicate the fact that this was an early
version of our APIs, created in concert with Microsoft Research.

On February 1, 2012, we released our final v1.0 SDK. In that
release, our DLL is now called Microsoft.Kinect.dll. All of our public APIs are
in the Microsoft.Kinect namespace.

Release

Assembly Names

Namespaces

Beta 2 (November 2011)

Microsoft.Research.Kinect.dll

Microsoft.Research.Kinect.Nui

Microsoft.Research.Kinect.Audio

V1.0 (Feb 2012)

Microsoft.Kinect.dll

Microsoft.Kinect

Runtime type (Rename to KinectSensor, refactoring)

Significant changes:

Rename of top level class – Runtime à KinectSensor
Removal of Camera class.
Several renames/refactorings.
New guidance for KinectSensor discovery.
Change in initialization of KinectSensor. (Initialize/Uninitialize)
Addition of several Mapping APIs. See details in Mapping API changes details.
Addition of several new KinectStatus values.

KINECTSENSOR API – SUMMARY OF API
CHANGES/REMOVALS/ADDITIONS

<NamespaceOldName=“Microsoft.Research.Kinect.Nui“Name=“Microsoft.Kinect“>

<TypeOldName=“Runtime“Name=“KinectSensor“>

<MemberOldName=“Kinects“Name=“KinectSensors“ />

<MemberOldName=“Initialize“Name=“Start“ />

<MemberOldName=“Uninitialize“Name=“Stop“ />

<MemberName=“Dispose“Added=“True“ />

<MemberOldName=“VideoStream“Name=“ColorStream“ />

<MemberOldName=“VideoFrameReady“Name=“ColorFrameReady“ />

<MemberOldName=“SkeletonEngine“
Name=“SkeletonStream“ />

<MemberName=“AllFramesReady“
Added=“True“ />

<MemberName=“InstanceIndex“
Removed=“True“ />

<MemberOldName=“InstanceName“Name=“DeviceConnectionId“ />

<MemberName=“UniqueKinectId“
Added=“True“ />

<MemberName=“NuiCamera“Removed=“True“/>

<MemberOldName=“ElevationMaximum“Name=“MaxElevationAngle“MovedFrom=“Camera“ />

<MemberOldName=“ElevationMinimum“Name=“MinElevationAngle“MovedFrom=“Camera“ />

<MemberName=“ElevationAngle“
MovedFrom=“Camera“ />

<MemberName=“MapDepthFrameToColorFrame“Added=“True“ />

<MemberName=“MapDepthToColorImagePoint“Added=“True“ />

<MemberName=“MapDepthToSkeletonPoint“Added=“True“ />

<MemberName=“MapSkeletonPointToDepth“Added=“True“ />

<MemberName=“MapSkeletonPointToColor“Added=“True“ />

Type>

<TypeName=“RuntimeOptions“Removed=“True“Message=“Enable the streams from
the KinectSensor class.“ />

<TypeOldName=“KinectDeviceCollection“Name=“KinectSensorCollection“>

<MemberName=“Dispose“Added=“True“ />

Type>

<TypeName=“StatusChangedEventArgs“>

<MemberOldName=“KinectRuntime“
Name=“Sensor“ />

Type>

<TypeName=“Camera“Removed=“True“>

<MemberOldName=“GetColorPixelCoordinatesFromDepthPixel“Name=“MapToColorPixel“MovedTo=“DepthImageFrame“ />

<MemberOldName=“ElevationMaximum“Name=“MaxElevationAngle“MovedTo=“KinectSensor“ />

<MemberOldName=“ElevationMinimum“Name=“MinElevationAngle“MovedTo=“KinectSensor“ />

<MemberName=“ElevationAngle“
MovedTo=“KinectSensor“ />

Type>

<TypeName=“KinectStatus“>

<MemberName=“Undefined“Added=“True“ />

<MemberName=“Initializing“Added=“True“ />

<MemberName=“DeviceNotGenuine“Added=“True“ />

<MemberName=“DeviceNotSupported“Added=“True“ />

<MemberName=“InsufficientBandwidth“Added=“True“ />

Type>

Namespace>

KinectSensor discovery

In beta2 (November 2011), we introduced a KinectSensors property on the Runtime object. We also
introduced a StatusChanged event.

In v1 (Feb 2012), those continue to be the key concepts to
understand when your program needs to work with 1 or more sensors.

It is critical that Kinect-enabled applications can deal
appropriately with the situations that users will hit with their application
(no Kinect sensor connected to computer, Kinect sensor is not powered by A/C
power, insufficient USB bandwidth, etc…)

Please read the documentation under “Programming
Guide/Kinect Sensor/Kinect-Enabled Application”:

Relevant samples (installed with SDK):

ShapeGame (C#/WPF) – KinectSensorChooser component
KinectAudioDemo (C#/WPF) – KinectSensorChooser component
KinectExplorer (C#/WPF)
SkeletonViewer (C++/Direct2D+GDI)

KinectSensor
initialization

Our beta APIs had a set of hard to understand and discover
API calls to configure the services you needed to get from the sensor. We
worked to make this part of our API more approachable. Our design goals were:

Enable a developer to ask once, not several
times for a feature.
Valid settings are determinable via intellisense/compiling, not just runtime errors
Good default behavior.
Dynamic changing of any setting over time.

OLD CODE

EQUIVALENT NEW CODE

runtime.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseColor);

runtime.VideoStream.Open(ImageStreamType.Video, 2,
ImageResolution.Resolution640x480, ImageType.Color);

runtime.DepthStream.Open(ImageStreamType.Depth, 2,
ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex);

kinectSensor.ColorStream.Enable();

kinectSensor.DepthStream.Enable();

kinectSensor.SkeletonStream.Enable();

kinectSensor.Start();

The old way doesn’t meet
many of our design goals for Initialization:

You needed to ask twice – once in Init() and once in Open().

Runtime errors were often are raised.
We had poor defaults; you had to ask for every setting.
We didn’t expose r/w properties for most of those settings, so you
needed to re-init() and/or re-open() to set them to different values.

The ColorImageStream, DepthImageStream,
and SkeletonStream types also have ways to tweak
their behavior. Enable has some optional parameters, and those stream types
have several r/w properties as well.

Types/APIs to explore: KinectSensor, ColorImageStream, DepthImageStream,
SkeletonStream.

KinectSensor uninitialize

OLD CODE	NEW CODE
runtime.Uninitialize();	kinectSensor.Stop();

ColorImage API changes

Significant changes:

Reorganization of colorImage
data. There is no longer a PlanarImage class.
Developer is now responsible for own allocations
of pixelData variables and copying pixelData to that storage. (Color pixelData
is still a byte array with same layout as beta2.)
This enables the Kinect Runtime to better reuse
resources if an application’s data processing falls behind.
Availability of colorImage
data during ColorFrameReady or AllFramesReady
events. (AllFramesReady gives matching color, depth
and/or skeleton frames.)

Relevant samples (installed with SDK):

KinectExplorer (C#/WPF) – KinectColorViewer component
SkeletonViewer (C++/Direct2D+GDI)
ShapeGame (C#/WPF) – KinectColorViewer component

COLORIMAGE API – SUMMARY OF
API CHANGES/REMOVALS/ADDITIONS

<TypeName=“ColorImageStream“Added=“True“>

<MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ />

<MemberName=“Disable“Added=“True“ />

<MemberOldName=“Open“Name=“Enable“ />

<MemberName=“StreamType“Removed=“True“ />

<MemberName=“Resolution“Removed=“True“ />

<MemberName=“Type“Removed=“True“ />

<MemberOldName=“Height“Name=“FrameHeight“ />

<MemberOldName=“Width“Name=“FrameWidth“ />

<MemberName=“CreateCompatibleImageFrame“Removed=“True“ />

Type>

<TypeName=“ImageFrameReadyEventArgs“Removed=“True“Message=“Use ColorImageFrameReadyEventArgs or DepthImageFrameReadyEventArgs
instead of the now removed ImageFrameReadyEventArgs“>

Type>

<TypeName=“ColorImageFrameReadyEventArgs“Added=“True“>

<MemberOldName=“ImageFrame“Name=“OpenColorImageFrame“ />

Type>

<TypeName=“ColorImageFrame“Added=“True“>

<MemberName=“Type“Removed=“True“ />

<MemberName=“Resolution“Removed=“True“ />

<MemberName=“Image“Removed=“True“ />

<MemberName=“ViewArea“Removed=“True“ />

<MemberName=“ImageFrame“Removed=“True“ />

<MemberName=“BytesPerPixel“MovedFrom=“PlanarImage“ />

<MemberName=“Height“MovedFrom=“PlanarImage“ />

<MemberName=“Width“MovedFrom=“PlanarImage“ />

<MemberOldName=“Bits“Name=“PixelData“MovedFrom=“PlanarImage“ />

<MemberName=“CopyPixelDataTo“ />

<MemberName=“SourceStream“Removed=“True“ />

<MemberName=“PixelData“Removed=“True“Message=“use CopyPixelDataTo(…)“/>

<MemberName=“Dispose“Added=“True“ />

Type>

<TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“
/>

OLD CODE

runtime.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);

runtime.VideoFrameReady += new
EventHandler<ImageFrameReadyEventArgs>(ColorImageReady);

void ColorImageReady(object sender, ImageFrameReadyEventArgs e)

{

PlanarImage planarImage
= e.ImageFrame.Image;

// DISPLAY OR PROCESS IMAGE DATA IN planarImage HERE

}

NEW CODE

kinectSensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);

kinectSensor.ColorFrameReady += new
EventHandler<ColorImageFrameReadyEventArgs>(ColorImageReady);

byte[] pixelData;

void ColorImageReady(object sender, ColorImageFrameReadyEventArgs e)

{

bool receivedData
= false;

using (ColorImageFrame colorImageFrame = e.OpenColorImageFrame())

{

if (colorImageFrame != null)

{

if
(pixelData == null)
//allocate the first time

{

pixelData
= new byte[colorImageFrame.PixelDataLength];

}

colorImageFrame.CopyPixelDataTo(pixelData);

receivedData
= true;

}

else

{

// apps
processing of image data is taking too long, it got
more than 2 frames behind.

// the
data is no longer avabilable.

}

if (receivedData)

{

// DISPLAY
OR PROCESS IMAGE DATA IN pixelData HERE

}

DepthImage API changes

Significant changes:

Reorganization of depthImage data. There is no longer a PlanarImage class.
DepthPixel data is now a short[], instead of a byte[]. (Still 2 bytes of data representing each pixel.)
DepthPixel bit layout is now consistent, regardless if you have opted for Depth+Player or just Depth.
There is no longer a need to ask for Depth+Player or Depth. If the SkeletonStream is enabled, you get Depth+Player.
Developer is now responsible for own allocations of pixelData variables and copying pixelData to that storage.

This enables the Kinect Runtime to better reuse resources if an application’s data processing falls behind.

Availability of depthImage data during DepthFrameReady or AllFramesReady
events. (AllFramesReady gives matching color, depth and/or skeleton frames.)

Relevant samples (installed with SDK):

KinectExplorer (C#/WPF) – KinectDepthViewer component
SkeletonViewer (C++/Direct2D+GDI)

DEPTHIMAGE API – SUMMARY OF
API CHANGES/REMOVALS/ADDITIONS

<TypeName=“ImageFrameReadyEventArgs“Removed=“True“Message=“Use ColorImageFrameReadyEventArgs or DepthImageFrameReadyEventArgs
instead of the now removed ImageFrameReadyEventArgs“>

Type>

<TypeName=“DepthImageStream“Added=“True“>

<MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ />

<MemberName=“IsTooFarRangeEnabled“ Added=“True“/>

<MemberName=“Disable“Added=“True“ />

<MemberOldName=“Open“Name=“Enable“ />

<MemberName=“StreamType“Removed=“True“ />

<MemberName=“Resolution“Removed=“True“ />

<MemberName=“Type“Removed=“True“ />

<MemberOldName=“Height“Name=“FrameHeight“ />

<MemberOldName=“Width“Name=“FrameWidth“ />

<MemberName=“CreateCompatibleImageFrame“Removed=“True“ />

Type>

<TypeName=“DepthImageFrameReadyEventArgs“Added=“True“>

<MemberOldName=“ImageFrame“Name=“OpenDepthImageFrame“ />

Type>

<TypeName=“DepthImageFrame“Added=“True“>

<MemberOldName=“GetColorPixelCoordinatesFromDepthPixel“Name=“MapToColorImagePoint“MovedFrom=“DepthImageFrame“ />

<MemberOldName=“SkeletonToDepthImage“Name=“MapFromSkeletonPoint“MovedFrom=“SkeletonStream“ />

<MemberOldName=“DepthImageToSkeleton“Name=“MapToSkeletonPoint“MovedFrom=“SkeletonStream“ />

<MemberName=“PlayerIndexBitmask“Added=“True“ />

<MemberName=“PlayerIndexBitmaskWidth“Added=“True“ />

Type>

<TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“
/>

<TypeName=“ImageDigitalZoom“Removed=“True“ />

<TypeName=“ImageStreamType“Removed=“True“ />

<TypeName=“ImageType“Removed=“True“ />

<TypeName=“ImageResolution“Removed=“True“ />

<TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“
/>

<TypeName=“ImageViewArea“
Removed=“True“ />

<TypeName=“DepthImageFormat“ Added=“True“/>

<TypeName=“DepthRange“ Added=“True“/>

<TypeName=“DepthImagePoint“Added=“True“ />

<TypeName=“DepthImagePointFloat“Added=“True“ />

OLD CODE

runtime.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240,

ImageType.DepthAndPlayerIndex);

runtime.DepthFrameReady += new
EventHandler<ImageFrameReadyEventArgs>(DepthImageReady);

private void
DepthImageReady(object
sender, ImageFrameReadyEventArgs
e)

{

PlanarImage planarImage
= e.ImageFrame.Image;

// DISPLAY OR
PROCESS IMAGE DATA IN pixelData HERE

}

NEW CODE

kinectSensor.DepthStream.Enable(DepthImageFormat.Resolution320x240Fps30);

kinectSensor.DepthFrameReady += new
EventHandler<DepthImageFrameReadyEventArgs>(DepthImageReady);

short[] pixelData;

private void
DepthImageReady(object
sender, DepthImageFrameReadyEventArgs
e)

{

bool receivedData
= false;

using (DepthImageFrame depthImageFrame = e.OpenDepthImageFrame())

{

if (depthImageFrame != null)

{

if (pixelData == null) //allocate the first time

{

pixelData
= new short[depthImageFrame.PixelDataLength];

}

depthImageFrame.CopyPixelDataTo(pixelData);

receivedData
= true;

}

else

{

// apps
processing of image data took too long; it got more than 2 frames behind.

// the
data is no longer avabilable.

}

if (receivedData)

{

// DISPLAY
OR PROCESS IMAGE DATA IN pixelData HERE

}

Skeleton API improvements

Significant changes:

Renamed SkeletonEngine -> SkeletonStream
Renamed SkeletonData -> Skeleton
Renamed JointID -> Joint
Renamed JointsCollection -> JointCollection
Renamed Vector -> SkeletonPoint
Renamed SkeletonQuality -> FrameEdges
Developer is now responsible for own allocations of skeleton array variables and copying skeletons to that storage.

This enables the Kinect Runtime to better reuse
resources if an application’s data processing falls behind.

Availability of skeleton data during SkeletonFrameReady or AllFramesReady
events. (AllFramesReady gives matching color, depth
and/or skeleton frames.)
New SkeletonStream.AppChoosesSkeletons property + ChooseSkeletons() methods.

Relevant samples (installed with SDK):

KinectExplorer (C#/WPF) – KinectSkeletalViewer component
SkeletonViewer (C++/Direct2D+GDI)
ShapeGame (C#/WPF)

SKELETON API – SUMMARY OF API
CHANGES/REMOVALS/ADDITIONS

<TypeOldName=“SkeletonEngine“
Name=“SkeletonStream“>

<MemberName=“Disable“Added=“True“ />

<MemberName=“Enable“Added=“True“ />

<MemberName=“ChooseSkeletons“Added=“True“ />

<MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ />

<MemberName=“DepthImageToSkeleton“MovedTo=“DepthImageFrame“ />

<MemberName=“SkeletonToDepthImage“MovedTo=“DepthImageFrame“ />

Type>

<TypeName=“SkeletonFrameReadyEventArgs“>

<MemberOldName=“SkeletonFrame“
Name=“OpenSkeletonFrame“ />

Type>

<TypeName=“SkeletonFrame“>

<MemberOldName=“TimeStamp“Name=“Timestamp“ />

<MemberOldName=“Skeletons“Name=“CopySkeletonDataTo“ />

<MemberName=“Dispose“Added=“True“ />

<MemberName=“Quality“Removed=“True“ />

Type>

<TypeOldName=“SkeletonData“Name=“Skeleton“ >

<MemberOldName=“TrackingID“Name=“TrackingId“ />

<MemberName=“Count“Removed=“True“ />

Type>

<TypeName=“JointType“Added=“True“/>

<TypeOldName=“JointID“Name=“Joint“>

<MemberOldName=“ID“Name=“JointType“ />

<MemberOldName=“Vector“Name=“Position“ />

<MemberName=“Parent“Removed=“True“ />

Type>

<TypeOldName=“JointsCollection“Name=“JointCollection“ />

<TypeOldName=“Vector“Name=“SkeletonPoint“>

<MemberName=“W“Removed=“True“ />

Type>

<TypeOldName=“SkeletonQuality“Name=“FrameEdges“>

<MemberOldName=“ClippedLeft“Name=“Left“ />

<MemberOldName=“ClippedTop“Name=“Top“ />

<MemberOldName=“ClippedRight“Name=“Right“ />

<MemberOldName=“ClippedBottom“
Name=“Bottom“ />

<MemberName=“None“Added=“True“ />

Type>

<TypeName=“SkeletonFrameQuality“Removed=“True“ />

OLD CODE

runtime.SkeletonFrameReady += new
EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);

private void
nui_SkeletonFrameReady(object
sender, SkeletonFrameReadyEventArgs
e)

{

SkeletonFrame skeletonFrame
= e.SkeletonFrame;

}

NEW CODE

kinectSensor.SkeletonFrameReady += new
EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);

Skeleton[] skeletons;

private void
nui_SkeletonFrameReady(object
sender, SkeletonFrameReadyEventArgs
e)

{

bool receivedData
= false;

using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())

{

if (skeletonFrame != null)

{

if
(skeletons == null) //allocate
the first time

{

skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];

}

receivedData
= true;

}

else

{

// apps
processing of skeleton data took too long; it got more than 2 frames behind.

// the
data is no longer avabilable.

}

if (receivedData)

{

// DISPLAY
OR PROCESS IMAGE DATA IN skeletons HERE

}

Mapping API improvements (Skeleton
-> Depth, Depth -> Color)

Significant changes:

Moved these members to DepthImageFrame and KinectSensor
Added a new “entire frame” instead of per pixel function. (KinectSensor.MapDepthFrameToColorFrame)
Added a new Skeleton to Color method, which is similar to calling SkeletonToDepth and then DepthToColor.
The DepthImageFrame versions of these methods take fewer parameter, since
it has the appropriate depthImage.
New return types: SkeletonPoint (replacing Vector) and DepthImagePoint (replacing 2 “out” parameters).

Relevant samples (installed with SDK):

KinectExplorer (C#/WPF) –KinectSkeletalViewer component
SkeletonViewer (C++/Direct2D+GDI)

OLD APIs

public class Camera
{
    public void GetColorPixelCoordinatesFromDepthPixel(ImageResolution

colorResolution, ImageViewArea
viewArea, int
depthX, int
depthY, short depthValue, out int
colorX, out int
colorY);

}
 
publicclass SkeletonEngine
{
    public Vector DepthImageToSkeleton(float depthX, float depthY, short depthValue);
    public void SkeletonToDepthImage(Vector point, out float depthX, out float depthY);
    public void SkeletonToDepthImage(Vector point, out float depthX, out float depthY, out short depthValue);
}

NEW APIs

publicsealedclass DepthImageFrame : ImageFrame

    public DepthImagePoint MapFromSkeletonPoint(SkeletonPoint

skeletonPoint);

    public ColorImagePoint MapToColorImagePoint(int depthX, int depthY, ColorImageFormat colorImageFormat);

    public SkeletonPoint MapToSkeletonPoint(int depthX, int depthY);

publicsealedclass KinectSensor : IDisposable
{
    public void MapDepthFrameToColorFrame(DepthImageFormat depthImageFormat, short[] depthPixelData, ColorImageFormat colorImageFormat, ColorImagePoint[] colorCoordinates);
    public ColorImagePoint MapDepthToColorImagePoint(DepthImageFormat depthImageFormat, int depthX, int depthY, short depthPixelValue, ColorImageFormat colorImageFormat);
    public SkeletonPoint MapDepthToSkeletonPoint(DepthImageFormat depthImageFormat, int depthX, int depthY, short depthPixelValue);
    public ColorImagePoint MapSkeletonPointToColor(SkeletonPoint skeletonPoint, ColorImageFormat colorImageFormat);
    public DepthImagePoint MapSkeletonPointToDepth(SkeletonPoint skeletonPoint, DepthImageFormat depthImageFormat);
}

Audio API improvements

Significant changes:

Relevant samples (installed with SDK):

AUDIO API – SUMMARY OF API CHANGES/REMOVALS/ADDITIONS

Type>

Namespace>

OLD
CODE

KinectAudioSource audioSource
= newKinectAudioSource();

audioSource.SystemMode = SystemMode.OptibeamArrayOnly;

audioSource.FeatureMode = true;

audioSource.AutomaticGainControl = false;

audioSource.MicArrayMode = MicArrayMode.MicArrayAdaptiveBeam;

var kinectStream
= audioSource.Start();

NEW
CODE

KinectAudioSource audioSource = kinectSensor.AudioSource;

audioSource.EchoCancellationMode = EchoCancellationMode.CancellationOnly;

audioSource.AutomaticGainControlEnabled = false;

audioSource.BeamAngleMode = BeamAngleMode.Adaptive;

var audioStream
= audioSource.Start();

Speech API improvements

Significant changes:

Relevant samples (installed with SDK):

KinectAudioDemo (C#/WPF)
ShapeGame (C#/WPF)
Speech (C#/console)

Kinect for Windows – Code Migration from Beta2 to v1.0 (C#/VB)

robrelyea — Wed, 01 Feb 2012 12:00:23 +0000

[This article covers the C#/VB accessible APIs. It has a peer article that covers the C++ accessible APIs.]

There have been a number of significant changes and improvements in our APIs since Beta2. This set of documents attempts to detail the changes to facilitate code migration from beta 2 to v1.

[Update - 9:44am PST, 2/1 - fixed link to migration dll below. If you still have problems, clearing browser cache may help.]

API Change Details

[these links take you to a seperate post with details]

Adapting to API Changes

In order to adapt to this change, C#/VB developers should:

Backup code projects (if not using source control, such as TFS)
Uninstall Beta 2 SDK (including speech runtime 10.x components).
Install Kinect for Windows SDK v1.
Migrate code to use v1 APIs and best practices via one of the migration techniques below
If you need help with the answer, go the Kinect for Windows forums. Search for other similar issues first. If needed, create a new issue. Please prefix the title of the forum post with “Migration issue:”. (for example: “Migration Issue: Runtime.Sensors replacement?”)
If you figured out the answer, and want to suggest improvements to the details and techniques in these documents, please leave a comment (We generally will respond to that feedback, but won’t show the comments, as it may get unwieldy).

Where to go for help

Code Migration Techniques (C#/VB)

There are three major approaches to migration

1) RECOMMENDED: Use a migration reference assembly to assist with learning about API changes

2) RECOMMENDED: Go back to the examples you may have started from, find the newest versions of those, and transfer your improvements to this new version.

3) NOT RECOMMENDED: Change reference assembly, (recompile -> search for info -> change -> adapt -> repeat)

Using the Migration reference assembly (RECOMMENDED)

Backup code projects (if not using source control, such as TFS)
Uninstall Beta 2 SDK (including speech runtime 10.x components).
Install Kinect for Windows SDK v1.
Remove Microsoft.Research.Kinect.dll references
Download (link is fixed now, may need to clear browser cache if still having problems) and add reference to Microsoft.Kinect.Migration.dll
[Ensure all projects, including binary dependencies, are migrated]
Build and evaluate errors/warnings

After changing all the “usings” of the old namespace to “using Microsoft.Kinect;”, most of the errors/warnings will point you to renames of types/members:

Look for guidance in errors/warnings
Look at sample code in SDK to see changes in event handling code necessary (there have been significant changes with ImageFrame and SkeletonFrame).
Change code as necessary -> recompile -> (repeat)
After building successfully with no errors and warnings (related to name changes)
- Remove reference to Microsoft.Kinect.Migration.dll
- Add reference to Microsoft.Kinect.dll
- Run application and debug as necessary

Doing it all manually (NOT RECOMMENDED)

[WE STRONGLY RECOMMEND YOU FOLLOW THE MIGRATION DLL TECHNIQUE ABOVE…AND DON’T FOLLOW THE MANUAL TECHNIQUE]

Backup code projects (if not using source control, such as TFS)
Uninstall Beta 2 SDK (including speech runtime 10.x components).
Install Kinect for Windows SDK v1.
Remove Microsoft.Research.Kinect.dll references
Add reference to Microsoft.Kinect.dll
[Ensure all projects, including binary dependencies, are migrated]
Build and evaluate errors/warnings
Look at sample code in SDK to see changes in event handling code necessary (there have been significant changes with ImageFrame and SkeletonFrame).
Change code as necessary -> recompile -> (repeat)
Run application and debug as necessary

Kinect for Windows Support Info Coming Soon

robrelyea — Wed, 01 Feb 2012 11:59:51 +0000

[Temporary placeholder page]

Normally, you would have been taken to the Kinect for Windows support site for “more info” about the situation the Kinect sensor UI was in.

The support site with end-user help for issues is coming soon.

Kinect Apps – ensuring Kinect Runtime is installed

robrelyea — Wed, 11 Jan 2012 16:50:41 +0000

Kinect for Windows 1.0 enabled apps should ensure that the Kinect Runtime is installed wherever the app is installed.

[Note: Kinect for Windows 1.0's latest public preview is beta 2. Parts of this blog post may be applicable to beta 2, but is primarily focused on the final v1.0 version, coming February 1st. Since v1.0 is not yet released, information I give here may change when it does release. I also am filtering this information to ensure that I am not giving away details that we don't yet want to release.]

Guidelines

App installers should install its dependencies, including Kinect Runtime – CRITICAL

Kinect for Windows 1.0 will have a KinectRuntime-v1.0-Setup.exe that your app installer MUST chain install. In addition to installing Kinect specific software (drivers + runtime), KinectRuntime-v1.0-Setup.exe will ensure the following dependencies are installed:

VCRT x86 and/or x64
.NET 4 client profile (or later 4.x versions like .NET 4.5)
Microsoft Speech Runtime v11 x86 and/or x64

Running an app should provide a decent user experience if Kinect Runtime isn’t installed – RECOMMENDED

.NET Applications

If your Kinect App is a .NET 4.x based app (WPF, XNA, WindowsForms, etc…), code like the following sample may help improve your apps user experience in case the Kinect Runtime 1.0 is not installed.

    //App.xaml.cs
    public partial class App : Application
    {
        protected override void OnStartup(StartupEventArgs e)
        {
            if (IsKinectRuntimeInstalled)
            {
                //go ahead and load the StartupUri as defined in App.xaml
                base.OnStartup(e);
            }
            else
            {
                MessageBox.Show(Assembly.GetExecutingAssembly().FullName + " is not able to excecute."
                + "An important dependency should have been installed by its setup program: Microsoft Kinect Runtime 1.0");
            }
        }

        public bool IsKinectRuntimeInstalled {
            get
            {
                bool isInstalled;
                try
                {
                    TestForKinectTypeLoadException();
                    isInstalled = true;
                }
                catch (FileNotFoundException)
                {
                    isInstalled = false;
                }
                return isInstalled;
            }
        }

        // This Microsoft.Kinect.dll based type, must be isolated in its own method
        // as the CLR will attempt to load the Microsoft.Kinect.dll assembly it when this method is executed.
        private void TestForKinectTypeLoadException()
        {
#pragma warning disable 219 //ignore the fact that status is unused code after this set.
            var status = KinectStatus.Disconnected;
#pragma warning restore 219
        }
    }

Native Applications

If your Kinect App is a Native Windows based app (C++ using GDI/GDI+/D2D/DirectX/etc..), your application should handle a missing Kinect10.dll in a way that informs the user of the problem and how to fix it.

Examples of depth API improvements coming in v1

robrelyea — Sat, 17 Dec 2011 15:48:23 +0000

As a program manager working on Kinect for Windows, I’m always excited to see feedback about Kinect for Windows. It is even better when we already have those issues fixed.

Piers7 posted “Gotchas with the Kinect SDK for Windows” recently. I’m happy to say that we’ve already got fixes for the first 3 gotchas he lists:

DepthData in v1 will no longer be mirrored improperly depending if you ask for Depth or Depth+Player.
DepthData data representation in v1 will no longer be different if you ask for Depth or Depth+Player. We always will use 3 bits to represent player id even if you haven’t asked for Depth+Player. In that case, they will just be “0″
APIs for Mapping from Skeleton->Depth, Depth->Skeleton, Depth->Color in v1 will no longer require bit shifting of the depth data.

Our v1 release in early 2012 will have these improvements and more…

We’re also investigating the best way to make GreenScreening easy and approachable, which is his 4th gotcha

Mobile developers and Kinect

robrelyea — Sun, 13 Nov 2011 22:57:38 +0000

I’ll Be Working on the Railroad

I recently had a rowing regatta in Portland. I had to mix some work in that weekend, so I took the train. I booked a business class ticket ($15 more each way). I was working on slides and demos for my DevConnections talk on 11/2 in Las Vegas about Kinect beta 2, v1 and beyond.

Here is a picture of my laptop and Kinect plugged in, and a demo app running on the train:

SkeletonViewer-WPF App

The app running in that photo is an evolution of our SDK’s SkeletonViewer-WPF app. Beta 2 included this new componentized SkeletonViewer app. KinectColorViwer and KinectDepthViewer are side by side in the app window.

On that trip I evolved that app to have a new component – KinectSkeletonViewer which overlapped both the color image and the depth image and rendered the skeleton on top of the color or depth image. In the beta 2 version of that app, the skeleton viewer code is part of KinectDiagnosticViewer. This new componentization work didn’t get to ship in Kinect for Windows v1-beta2, but we’ll get it out to devs in the future.

Recording/Playback of Kinect Data

One of the things I demoed at DevConnections was an Xbox SDK tool to record and playback Kinect data. That would have been easier for my train based development. I could have just used the pre-recorded Kinect data (color, depth, etc…) instead of bringing the Kinect with me. I’ll be excited when we can deliver that for Kinect for Windows developers too.

My shift from WPF-XAML to Kinect for Windows

robrelyea — Sun, 13 Nov 2011 16:42:17 +0000

New Team @ Microsoft for Rob

I’m 6 months into my new team (Kinect for Windows) and I haven’t even mentioned it on my blog. My twitter self (@rrelyea) announced my new gig on June 20^th, 2011:

(in November 2011, we moved to use @KinectWindows and #kinectforwindows)

Perhaps I haven’t blogged about it yet because twitter is a place where you can say less. It was difficult to talk about some of these things due to the fact that some key facts were Microsoft confidential until the Build conference in September.

When I announced my team move, I had already been working almost full time on Kinect for about a month. That day was just the official start date. By total coincidence, that day was the day that (reportedly) Soma sent an email announcing changes for the XAML, WPF and Silverlight teams. Although I knew org changes might happen in the future in that space, I had thought that it would wait until after Build.

WPF 4.5 Project

After WPF 4 shipped in April 2010, I was asked to move from my XAML Architect role to leading the Program Management team for WPF 4.5. That role also had me working closely with the heads of the Developer and Test team. The 3 of us were referred to as the WPF Triad – the leadership of the WPF team. The role was an exciting, and challenging one for me. It stretched me in new directions. We were leading the team doing the planning (see pictures of post it notes from my office) and execution of the entire release. That was a new scope for me – which I enjoyed.

I did a PDC 2010 talk about WPF Today and Tomorrow. There I shared some of our plans for WPF 4.5. Build 2011 was rightly focused primarily on Windows 8, so there was no talk focused on improvements in WPF. However, it was during Build, that a preview of WPF 4.5 first became available (in the .NET 4.5 Developer Preview). I’m excited about WPF 4.5 for existing customers and also excited about the fact that Windows 8 has a strong focus on leveraging HTML or XAML skillsets.

Rob 3.0?

In early months of this year, I had decided to do soul searching to figure out what was next for me. We had reached code complete on WPF 4.5. Although I was enjoying much about my work, it was the first time I had found myself in a position that was deemed important, but much less important than several other efforts.

I thought about the following options:

Delay a change until WPF 4.5 ships.
Join a sister team of WPF. The XAML team had lots of needs/opportunities, as the Windows 8 based project needed significant more focus in its v1.
Join a XAML ecosystem related team – Expression, Visual Studio, etc…
Join a HTML ecosystem related team – a platform or tools team (leveraging my earlier roots from 1998-2001).
Join an entirely different team inside of Microsoft – Windows, Windows Phone, Bing, Azure, Xbox, …

As part of this soul search, I even considered leaving Microsoft – in my 18^th year, it was the first time I ever seriously thought about that. I talked with 2 friends who had left the company for other prestigious companies. After dipping my toe in the water, I decided that I would focus on finding a great role at Microsoft.

Effectively, I was trying to decide whether it was time for Rob 2.1 or 3.0. Rob 1.0 was my 5 years working as a Developer Evangelist-ish for Microsoft in Chicago. Rob 2.0 was my 13 years working on HTML programmability, XAML UI programmability, and the XAML language and engines. I had last done a major soul search in 1998 when I joined the IE team. During next 13 years I had never changed my group – IE, “Avalon”, WPF, and XAML – the groups may have reorged and changed focuses, but I never even considered a job shift.

Kinect for Windows is it!

In the end, I picked the Kinect for Windows role. I was lucky and excited to get a great opportunity to work on something so exciting. It had an exciting new focus for me – Natural User Interface (NUI) and the magic of Kinect. The Kinect Effect shows some of the possibilities. Couldn’t be enjoying it more…

Starting another blog – WindowsClient.net

robrelyea — Sun, 10 Feb 2008 15:18:32 +0000

I’m not positive what I’m going to do with this blog, but I’m heavily considering moving most of my blogging over to WindowsClient.net. Just did my first post: Rob start his WindowsClient.net Era – 3rd blogs a charm (http://blogs.windowsclient.net/rob_relyea/archive/2008/02/10/3rd-time-is-a-charm.aspx)

Josh Twist: Xaml Editing with Intellisense with no VS Designer

robrelyea — Fri, 08 Feb 2008 17:42:43 +0000

I knew you could change the default from WPF Designer to XML Editor (and lose all the intellisense), but I didn’t know this tip that Josh shows about Xaml editing in VS…

Visualization of progress for self measurement and public pressure/support

robrelyea — Fri, 08 Feb 2008 14:03:59 +0000

The other day I blogged in "My car runs on oil from a baby seal" that people would be able to conserve gasoline, reduce driving, etc.. better if they could visualize the amount of gallons they used per day (GPD).

Now I tripped over this site that talks about a similar measurement, visualization and motivation technique that Jerry Seinfeld used (and maybe still does).

Rob Relyea

Kinect for Windows – Details of API Changes from Beta2 to v1.0 (C#/VB)

Namespace and Assembly Name Changes

Runtime type (Rename to KinectSensor, refactoring)

KINECTSENSOR API – SUMMARY OF API CHANGES/REMOVALS/ADDITIONS

KinectSensor discovery

KinectSensor initialization

KinectSensor uninitialize

ColorImage API changes

DepthImage API changes

Skeleton API improvements

Mapping API improvements (Skeleton -> Depth, Depth -> Color)

Audio API improvements

Speech API improvements

Kinect for Windows – Code Migration from Beta2 to v1.0 (C#/VB)

API Change Details

Adapting to API Changes

Where to go for help

Code Migration Techniques (C#/VB)

Kinect for Windows Support Info Coming Soon

Kinect Apps – ensuring Kinect Runtime is installed

Guidelines

App installers should install its dependencies, including Kinect Runtime – CRITICAL

Running an app should provide a decent user experience if Kinect Runtime isn’t installed – RECOMMENDED

.NET Applications

Native Applications

Examples of depth API improvements coming in v1

Mobile developers and Kinect

My shift from WPF-XAML to Kinect for Windows

Starting another blog – WindowsClient.net

Josh Twist: Xaml Editing with Intellisense with no VS Designer

Visualization of progress for self measurement and public pressure/support

KINECTSENSOR API – SUMMARY OF API
CHANGES/REMOVALS/ADDITIONS

KinectSensor
initialization

Mapping API improvements (Skeleton
-> Depth, Depth -> Color)