Kinect for Windows – Details of API Changes from Beta2 to v1.0 (C#/VB)

[This article covers the details of changes of C#/VB accessible
APIs. Please ensure you read the code
migration post as well.]

Namespace and Assembly Name Changes

Since our Beta 1 (June 2011) release of the Kinect for
Windows SDK , our C#/VB  accessible APIs have been provided via
Microsoft.Research.Kinect.dll. The public Types all were organized into 2 namespaces:
Microsoft.Research.Kinect.Nui and Microsoft.Research.Kinect.Audio.
Our team made the decision back in early 2011 that we would mark our beta
releases of APIs as “Research” to indicate the fact that this was an early
version of our APIs, created in concert with Microsoft Research.

On February 1, 2012, we released our final v1.0 SDK. In that
release, our DLL is now called Microsoft.Kinect.dll. All of our public APIs are
in the Microsoft.Kinect namespace.

Release	Assembly Names	Namespaces
Beta 2 (November 2011)	Microsoft.Research.Kinect.dll	Microsoft.Research.Kinect.Nui Microsoft.Research.Kinect.Audio
V1.0 (Feb 2012)	Microsoft.Kinect.dll	Microsoft.Kinect

Runtime type (Rename to KinectSensor, refactoring)

Significant changes:

• Rename of top level class – Runtime à KinectSensor
• Removal of Camera class.
• Several renames/refactorings.
• New guidance for KinectSensor discovery.
• Change in initialization of KinectSensor. (Initialize/Uninitialize)
• Addition of several Mapping APIs. See details in Mapping API changes details.
• Addition of several new KinectStatus values.

KINECTSENSOR API – SUMMARY OF API CHANGES/REMOVALS/ADDITIONS

<NamespaceOldName=“Microsoft.Research.Kinect.Nui“Name=“Microsoft.Kinect“>     <TypeOldName=“Runtime“Name=“KinectSensor“>         <MemberOldName=“Kinects“Name=“KinectSensors“ />         <MemberOldName=“Initialize“Name=“Start“ />         <MemberOldName=“Uninitialize“Name=“Stop“ />         <MemberName=“Dispose“Added=“True“ />           <MemberOldName=“VideoStream“Name=“ColorStream“ />         <MemberOldName=“VideoFrameReady“Name=“ColorFrameReady“ />           <MemberOldName=“SkeletonEngine“ Name=“SkeletonStream“ />           <MemberName=“AllFramesReady“ Added=“True“ />           <MemberName=“InstanceIndex“ Removed=“True“ />         <MemberOldName=“InstanceName“Name=“DeviceConnectionId“ />         <MemberName=“UniqueKinectId“ Added=“True“ />           <MemberName=“NuiCamera“Removed=“True“/>         <MemberOldName=“ElevationMaximum“Name=“MaxElevationAngle“MovedFrom=“Camera“ />         <MemberOldName=“ElevationMinimum“Name=“MinElevationAngle“MovedFrom=“Camera“ />         <MemberName=“ElevationAngle“ MovedFrom=“Camera“ />           <MemberName=“MapDepthFrameToColorFrame“Added=“True“ />         <MemberName=“MapDepthToColorImagePoint“Added=“True“ />         <MemberName=“MapDepthToSkeletonPoint“Added=“True“ />         <MemberName=“MapSkeletonPointToDepth“Added=“True“ />         <MemberName=“MapSkeletonPointToColor“Added=“True“ />     </Type>     <TypeName=“RuntimeOptions“Removed=“True“Message=“Enable the streams from the KinectSensor class.“ />     <TypeOldName=“KinectDeviceCollection“Name=“KinectSensorCollection“>         <MemberName=“Dispose“Added=“True“ />     </Type>     <TypeName=“StatusChangedEventArgs“>         <MemberOldName=“KinectRuntime“ Name=“Sensor“ />     </Type>     <TypeName=“Camera“Removed=“True“>         <MemberOldName=“GetColorPixelCoordinatesFromDepthPixel“Name=“MapToColorPixel“MovedTo=“DepthImageFrame“ />         <MemberOldName=“ElevationMaximum“Name=“MaxElevationAngle“MovedTo=“KinectSensor“ />         <MemberOldName=“ElevationMinimum“Name=“MinElevationAngle“MovedTo=“KinectSensor“ />         <MemberName=“ElevationAngle“ MovedTo=“KinectSensor“ />     </Type>     <TypeName=“KinectStatus“>         <MemberName=“Undefined“Added=“True“ />         <MemberName=“Initializing“Added=“True“ />         <MemberName=“DeviceNotGenuine“Added=“True“ />         <MemberName=“DeviceNotSupported“Added=“True“ />         <MemberName=“InsufficientBandwidth“Added=“True“ />     </Type> </Namespace>

KinectSensor discovery

In beta2 (November 2011), we introduced a KinectSensors property on the Runtime object. We also
introduced a StatusChanged event.

In v1 (Feb 2012), those continue to be the key concepts to
understand when your program needs to work with 1 or more sensors.

It is critical that Kinect-enabled applications can deal
appropriately with the situations that users will hit with their application
(no Kinect sensor connected to computer, Kinect sensor is not powered by A/C
power, insufficient USB bandwidth, etc…)

Please read the documentation under “Programming
Guide/Kinect Sensor/Kinect-Enabled Application”:

Relevant samples (installed with SDK):

• ShapeGame (C#/WPF) – KinectSensorChooser component
• KinectAudioDemo (C#/WPF) – KinectSensorChooser component
• KinectExplorer (C#/WPF)
• SkeletonViewer (C++/Direct2D+GDI)

KinectSensor
initialization

Our beta APIs had a set of hard to understand and discover
API calls to configure the services you needed to get from the sensor. We
worked to make this part of our API more approachable. Our design goals were:

• Enable a developer to ask once, not several
  times for a feature.
• Valid settings are determinable via intellisense/compiling, not just runtime errors
• Good default behavior.
• Dynamic changing of any setting over time.

OLD CODE	EQUIVALENT NEW CODE
runtime.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseColor); runtime.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color); runtime.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex);	kinectSensor.ColorStream.Enable(); kinectSensor.DepthStream.Enable(); kinectSensor.SkeletonStream.Enable(); kinectSensor.Start();
The old way doesn’t meet many of our design goals for Initialization: You needed to ask twice – once in Init() and once in Open(). Runtime errors were often are raised. We had poor defaults; you had to ask for every setting. We didn’t expose r/w properties for most of those settings, so you needed to re-init() and/or re-open() to set them to different values.	The ColorImageStream, DepthImageStream, and SkeletonStream types also have ways to tweak their behavior. Enable has some optional parameters, and those stream types have several r/w properties as well.

Types/APIs to explore: KinectSensor, ColorImageStream, DepthImageStream,
SkeletonStream.

KinectSensor uninitialize

OLD CODE	NEW CODE
runtime.Uninitialize();	kinectSensor.Stop();

ColorImage API changes

Significant changes:

• Reorganization of colorImage
  data. There is no longer a PlanarImage class.
• Developer is now responsible for own allocations
  of pixelData variables and copying pixelData to that storage. (Color pixelData
  is still a byte array with same layout as beta2.)
• This enables the Kinect Runtime to better reuse
  resources if an application’s data processing falls behind.
• Availability of colorImage
  data during ColorFrameReady or AllFramesReady
  events. (AllFramesReady gives matching color, depth
  and/or skeleton frames.)

Relevant samples (installed with SDK):

• KinectExplorer (C#/WPF) – KinectColorViewer component
• SkeletonViewer (C++/Direct2D+GDI)
• ShapeGame (C#/WPF) – KinectColorViewer component

COLORIMAGE API – SUMMARY OF API CHANGES/REMOVALS/ADDITIONS

<TypeName=“ColorImageStream“Added=“True“>     <MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ />     <MemberName=“Disable“Added=“True“ />     <MemberOldName=“Open“Name=“Enable“ />     <MemberName=“StreamType“Removed=“True“ />     <MemberName=“Resolution“Removed=“True“ />     <MemberName=“Type“Removed=“True“ />     <MemberOldName=“Height“Name=“FrameHeight“ />     <MemberOldName=“Width“Name=“FrameWidth“ />     <MemberName=“CreateCompatibleImageFrame“Removed=“True“ /> </Type> <TypeName=“ImageFrameReadyEventArgs“Removed=“True“Message=“Use ColorImageFrameReadyEventArgs or DepthImageFrameReadyEventArgs instead of the now removed ImageFrameReadyEventArgs“> </Type> <TypeName=“ColorImageFrameReadyEventArgs“Added=“True“>     <MemberOldName=“ImageFrame“Name=“OpenColorImageFrame“ /> </Type> <TypeName=“ColorImageFrame“Added=“True“>     <MemberName=“Type“Removed=“True“ />     <MemberName=“Resolution“Removed=“True“ />     <MemberName=“Image“Removed=“True“ />     <MemberName=“ViewArea“Removed=“True“ />     <MemberName=“ImageFrame“Removed=“True“ />     <MemberName=“BytesPerPixel“MovedFrom=“PlanarImage“ />     <MemberName=“Height“MovedFrom=“PlanarImage“ />     <MemberName=“Width“MovedFrom=“PlanarImage“ />     <MemberOldName=“Bits“Name=“PixelData“MovedFrom=“PlanarImage“ />     <MemberName=“CopyPixelDataTo“ />     <MemberName=“SourceStream“Removed=“True“ />     <MemberName=“PixelData“Removed=“True“Message=“use CopyPixelDataTo(…)“/>     <MemberName=“Dispose“Added=“True“ /> </Type> <TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“ />

OLD CODE

runtime.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);   runtime.VideoFrameReady += new EventHandler<ImageFrameReadyEventArgs>(ColorImageReady);   void ColorImageReady(object sender, ImageFrameReadyEventArgs e) {     PlanarImage planarImage = e.ImageFrame.Image;       // DISPLAY OR PROCESS IMAGE DATA IN planarImage HERE }

NEW CODE

kinectSensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);   kinectSensor.ColorFrameReady += new EventHandler<ColorImageFrameReadyEventArgs>(ColorImageReady); byte[] pixelData;   void ColorImageReady(object sender, ColorImageFrameReadyEventArgs e) {     bool receivedData = false;     using (ColorImageFrame colorImageFrame = e.OpenColorImageFrame())     {         if (colorImageFrame != null)         {             if (pixelData == null) //allocate the first time             {                 pixelData = new byte[colorImageFrame.PixelDataLength];             }             colorImageFrame.CopyPixelDataTo(pixelData);             receivedData = true;         }         else         {             // apps processing of image data is taking too long, it got more than 2 frames behind.             // the data is no longer avabilable.         }     }     if (receivedData)     {         // DISPLAY OR PROCESS IMAGE DATA IN pixelData HERE     } }

DepthImage API changes

Significant changes:

• Reorganization of depthImage data. There is no longer a PlanarImage class.
• DepthPixel data is now a short[], instead of a byte[]. (Still 2 bytes of data representing each pixel.)
• DepthPixel bit layout is now consistent, regardless if you have opted for Depth+Player or just Depth.
• There is no longer a need to ask for Depth+Player or Depth. If the SkeletonStream is enabled, you get Depth+Player.
• Developer is now responsible for own allocations of pixelData variables and copying pixelData to that storage.

This enables the Kinect Runtime to better reuse resources if an application’s data processing falls behind.

• Availability of depthImage data during DepthFrameReady or AllFramesReady
  events. (AllFramesReady gives matching color, depth and/or skeleton frames.)

Relevant samples (installed with SDK):

• KinectExplorer (C#/WPF) – KinectDepthViewer component
• SkeletonViewer (C++/Direct2D+GDI)

DEPTHIMAGE API – SUMMARY OF API CHANGES/REMOVALS/ADDITIONS

<TypeName=“ImageFrameReadyEventArgs“Removed=“True“Message=“Use ColorImageFrameReadyEventArgs or DepthImageFrameReadyEventArgs instead of the now removed ImageFrameReadyEventArgs“> </Type> <TypeName=“DepthImageStream“Added=“True“>     <MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ />     <MemberName=“IsTooFarRangeEnabled“  Added=“True“/>     <MemberName=“Disable“Added=“True“ />     <MemberOldName=“Open“Name=“Enable“ />     <MemberName=“StreamType“Removed=“True“ />     <MemberName=“Resolution“Removed=“True“ />     <MemberName=“Type“Removed=“True“ />     <MemberOldName=“Height“Name=“FrameHeight“ />     <MemberOldName=“Width“Name=“FrameWidth“ />     <MemberName=“CreateCompatibleImageFrame“Removed=“True“ /> </Type> <TypeName=“DepthImageFrameReadyEventArgs“Added=“True“>     <MemberOldName=“ImageFrame“Name=“OpenDepthImageFrame“ /> </Type> <TypeName=“DepthImageFrame“Added=“True“>     <MemberOldName=“GetColorPixelCoordinatesFromDepthPixel“Name=“MapToColorImagePoint“MovedFrom=“DepthImageFrame“ />     <MemberOldName=“SkeletonToDepthImage“Name=“MapFromSkeletonPoint“MovedFrom=“SkeletonStream“ />     <MemberOldName=“DepthImageToSkeleton“Name=“MapToSkeletonPoint“MovedFrom=“SkeletonStream“ />     <MemberName=“PlayerIndexBitmask“Added=“True“ />     <MemberName=“PlayerIndexBitmaskWidth“Added=“True“ /> </Type> <TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“ /> <TypeName=“ImageDigitalZoom“Removed=“True“ /> <TypeName=“ImageStreamType“Removed=“True“ /> <TypeName=“ImageType“Removed=“True“ /> <TypeName=“ImageResolution“Removed=“True“ /> <TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“ /> <TypeName=“ImageViewArea“ Removed=“True“ /> <TypeName=“DepthImageFormat“  Added=“True“/> <TypeName=“DepthRange“  Added=“True“/> <TypeName=“DepthImagePoint“Added=“True“ /> <TypeName=“DepthImagePointFloat“Added=“True“ />

OLD CODE

runtime.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240,                      ImageType.DepthAndPlayerIndex);   runtime.DepthFrameReady += new EventHandler<ImageFrameReadyEventArgs>(DepthImageReady);   private void DepthImageReady(object sender, ImageFrameReadyEventArgs e) {     PlanarImage planarImage = e.ImageFrame.Image;       // DISPLAY OR PROCESS IMAGE DATA IN pixelData HERE }

NEW CODE

kinectSensor.DepthStream.Enable(DepthImageFormat.Resolution320x240Fps30);   kinectSensor.DepthFrameReady += new EventHandler<DepthImageFrameReadyEventArgs>(DepthImageReady); short[] pixelData;   private void DepthImageReady(object sender, DepthImageFrameReadyEventArgs e) {     bool receivedData = false;     using (DepthImageFrame depthImageFrame = e.OpenDepthImageFrame())     {         if (depthImageFrame != null)         {             if (pixelData == null) //allocate the first time             {                 pixelData = new short[depthImageFrame.PixelDataLength];             }             depthImageFrame.CopyPixelDataTo(pixelData);             receivedData = true;         }         else         {             // apps processing of image data took too long; it got more than 2 frames behind.             // the data is no longer avabilable.         }     }     if (receivedData)     {         // DISPLAY OR PROCESS IMAGE DATA IN pixelData HERE     } }

Skeleton API improvements

Significant changes:

• Renamed SkeletonEngine -> SkeletonStream
• Renamed SkeletonData -> Skeleton
• Renamed JointID -> Joint
• Renamed JointsCollection -> JointCollection
• Renamed Vector -> SkeletonPoint
• Renamed SkeletonQuality -> FrameEdges
• Developer is now responsible for own allocations of skeleton array variables and copying skeletons to that storage.

This enables the Kinect Runtime to better reuse
resources if an application’s data processing falls behind.

• Availability of skeleton data during SkeletonFrameReady or AllFramesReady
  events. (AllFramesReady gives matching color, depth
  and/or skeleton frames.)
• New SkeletonStream.AppChoosesSkeletons  property + ChooseSkeletons() methods.

Relevant samples (installed with SDK):

• KinectExplorer (C#/WPF) – KinectSkeletalViewer component
• SkeletonViewer (C++/Direct2D+GDI)
• ShapeGame (C#/WPF)

SKELETON API – SUMMARY OF API CHANGES/REMOVALS/ADDITIONS

<TypeOldName=“SkeletonEngine“ Name=“SkeletonStream“>     <MemberName=“Disable“Added=“True“ />     <MemberName=“Enable“Added=“True“ />     <MemberName=“ChooseSkeletons“Added=“True“ />     <MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ />     <MemberName=“DepthImageToSkeleton“MovedTo=“DepthImageFrame“ />     <MemberName=“SkeletonToDepthImage“MovedTo=“DepthImageFrame“ /> </Type> <TypeName=“SkeletonFrameReadyEventArgs“>     <MemberOldName=“SkeletonFrame“ Name=“OpenSkeletonFrame“ /> </Type> <TypeName=“SkeletonFrame“>     <MemberOldName=“TimeStamp“Name=“Timestamp“ />     <MemberOldName=“Skeletons“Name=“CopySkeletonDataTo“ />     <MemberName=“Dispose“Added=“True“ />     <MemberName=“Quality“Removed=“True“ /> </Type> <TypeOldName=“SkeletonData“Name=“Skeleton“ >     <MemberOldName=“TrackingID“Name=“TrackingId“ /> <MemberName=“Count“Removed=“True“ /> </Type> <TypeName=“JointType“Added=“True“/> <TypeOldName=“JointID“Name=“Joint“>     <MemberOldName=“ID“Name=“JointType“ />     <MemberOldName=“Vector“Name=“Position“ />     <MemberName=“Parent“Removed=“True“ /> </Type> <TypeOldName=“JointsCollection“Name=“JointCollection“ /> <TypeOldName=“Vector“Name=“SkeletonPoint“>     <MemberName=“W“Removed=“True“ /> </Type> <TypeOldName=“SkeletonQuality“Name=“FrameEdges“>     <MemberOldName=“ClippedLeft“Name=“Left“ />     <MemberOldName=“ClippedTop“Name=“Top“ />     <MemberOldName=“ClippedRight“Name=“Right“ />     <MemberOldName=“ClippedBottom“ Name=“Bottom“ />     <MemberName=“None“Added=“True“ /> </Type> <TypeName=“SkeletonFrameQuality“Removed=“True“ />

OLD CODE

runtime.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);   private void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e) {     SkeletonFrame skeletonFrame = e.SkeletonFrame;   }

NEW CODE

kinectSensor.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);   Skeleton[] skeletons;   private void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e) {     bool receivedData = false;     using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())     {         if (skeletonFrame != null)         {             if (skeletons == null) //allocate the first time             {                 skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];             }             receivedData = true;         }         else         {             // apps processing of skeleton data took too long; it got more than 2 frames behind.             // the data is no longer avabilable.         }     }     if (receivedData)     {         // DISPLAY OR PROCESS IMAGE DATA IN skeletons HERE     } }

Mapping API improvements (Skeleton
-> Depth, Depth -> Color)

Significant changes:

• Moved these members to DepthImageFrame and KinectSensor
• Added a new “entire frame” instead of per pixel function. (KinectSensor.MapDepthFrameToColorFrame)
• Added a new Skeleton to Color method, which is similar to calling SkeletonToDepth and then DepthToColor.
• The DepthImageFrame versions of these methods take fewer parameter, since
  it has the appropriate depthImage.
• New return types: SkeletonPoint (replacing Vector) and DepthImagePoint (replacing 2 “out” parameters).

Relevant samples (installed with SDK):

• KinectExplorer (C#/WPF) –KinectSkeletalViewer component
• SkeletonViewer (C++/Direct2D+GDI)

OLD APIs

public class Camera {     public void GetColorPixelCoordinatesFromDepthPixel(ImageResolution colorResolution, ImageViewArea viewArea, int depthX, int depthY, short depthValue, out int colorX, out int colorY); }   publicclass SkeletonEngine {     public Vector DepthImageToSkeleton(float depthX, float depthY, short depthValue);     public void SkeletonToDepthImage(Vector point, out float depthX, out float depthY);     public void SkeletonToDepthImage(Vector point, out float depthX, out float depthY, out short depthValue); }

NEW APIs

publicsealedclass DepthImageFrame : ImageFrame {     public DepthImagePoint MapFromSkeletonPoint(SkeletonPoint skeletonPoint);     public ColorImagePoint MapToColorImagePoint(int depthX, int depthY, ColorImageFormat colorImageFormat);     public SkeletonPoint MapToSkeletonPoint(int depthX, int depthY); } publicsealedclass KinectSensor : IDisposable {     public void MapDepthFrameToColorFrame(DepthImageFormat depthImageFormat, short[] depthPixelData, ColorImageFormat colorImageFormat, ColorImagePoint[] colorCoordinates);     public ColorImagePoint MapDepthToColorImagePoint(DepthImageFormat depthImageFormat, int depthX, int depthY, short depthPixelValue, ColorImageFormat colorImageFormat);     public SkeletonPoint MapDepthToSkeletonPoint(DepthImageFormat depthImageFormat, int depthX, int depthY, short depthPixelValue);     public ColorImagePoint MapSkeletonPointToColor(SkeletonPoint skeletonPoint, ColorImageFormat colorImageFormat);     public DepthImagePoint MapSkeletonPointToDepth(SkeletonPoint skeletonPoint, DepthImageFormat depthImageFormat); }

Audio API improvements

Significant changes:

• KinectAudioSource no longer has a constructor, you must access the AudioSource property from KinectSensor. In multiple KinectSensor scenarios, it will now ensure that you get the audio source that is from that sensor.
• API renames
• API refactoring with simplification in mind.
• Removal of many less useful members.
• Properties that return angles now return angles in degrees instead of radians. This is  consistent with the motor’s ElevationAngle.

Relevant samples (installed with SDK):

• KinectAudioDemo (C#/WPF)
• ShapeGame (C#/WPF)
• Speech (C#/console)
• AudioCaptureRaw (C++/console)
• MFAudioFilter (C++/console)
• MicArrayEchoCancellation (C++/console)
• KinectExplorer (C#/WPF) – KinectAudioViewer component

AUDIO API – SUMMARY OF API CHANGES/REMOVALS/ADDITIONS

<Namespace OldName=“Microsoft.Research.Kinect.Audio“ Name=“Microsoft.Kinect“>     <Type Name=“AudioDeviceInfo“ Removed=“True“ />     <Type Name=“KinectAudioSource“>         <Member Name=“FindCaptureDevices“ Removed=“True“ />         <Member Name=“RetrieveTsStats“ Removed=“True“ />         <Member Name=“QualityMetrics“ Removed=“True“ />         <Member Name=“DevicePairGuid“ Removed=“True“ />         <Member Name=“KinectAudioSource“ Removed=“True“ />         <Member OldName=“AutomaticGainControl“ Name=“AutomaticGainControlEnabled“ />         <Member OldName=“SoundSourcePosition“ Name=“SoundSourceAngle“ Message=“Now using degrees, instead of Radians.“/>         <Member OldName=“SoundSourcePositionConfidence“ Name=“SoundSourceAngleConfidence“ />         <Member OldName=“MicArrayMode“ Name=“BeamAngleMode“ />         <Member OldName=“MicArrayBeamAngle“ Name=“ManualBeamAngle“ />         <Member OldName=“BeamChanged“ Name=“BeamAngleChanged“ />         <Member OldName=“SpeakerIndex“ Name=“EchoCancellationSpeakerIndex“ Message=“Automatically set to default Speaker on PC now.“ />         <Member Name=“AcousticEchoSuppression“ Removed=“True“ Message=“Replaced by setting kinectAudioSource.EchoCancellationMode = EchoCancellationMode.CancellationAndSuppression“ />         <Member Name=“CenterClip“ Removed=“True“/>         <Member Name=“EchoLength“ Removed=“True“/>         <Member Name=“FeatureMode“ Removed=“True“ Message=“Just remove the line of code that set FeatureMode. It was useless.“/>         <Member Name=“FrameSize“ Removed=“True“/>         <Member Name=“GainBounder“ Removed=“True“/>         <Member Name=“MicArrayPreprocess“ Removed=“True“/>         <Member Name=“MicrophoneIndex“ Removed=“True“/>         <Member Name=“NoiseFill“ Removed=“True“/>         <Member Name=“SourceMode“ Removed=“True“/>         <Member Name=“VoiceActivityDetector“ Removed=“True“/>         <Member OldName=“SystemMode“ Name=“EchoCancellationMode“ Message=“For ArrayOnly, use EchoCancellationMode.None. For ArrayAndEchoCancellation, use EchoCancellationMode.CancellationOnly.“ />         <Member Name=“Dispose“ Removed=“True“ />     </Type>     <Type Name=“MicArrayMode“ Removed=“True“ />     <Type Name=“SystemMode“ Removed=“True“ Message=“Use EchoCancellationMode property instead.“ />     <Type OldName=“BeamChangedEventArgs“ Name=“BeamAngleChangedEventArgs“/>     <Type Name=“SoundSourceAngleChangedEventArgs“ Added=“True“ />     <Type Name=“EchoCancellationMode“ Added=“True“ />     <Type Name=“BeamAngleMode“ Added=“True“ /> </Namespace>

 

OLD CODE

KinectAudioSource audioSource = newKinectAudioSource(); audioSource.SystemMode = SystemMode.OptibeamArrayOnly; audioSource.FeatureMode = true; audioSource.AutomaticGainControl = false; audioSource.MicArrayMode = MicArrayMode.MicArrayAdaptiveBeam; var kinectStream = audioSource.Start();

NEW CODE

KinectAudioSource audioSource = kinectSensor.AudioSource; audioSource.EchoCancellationMode = EchoCancellationMode.CancellationOnly; audioSource.AutomaticGainControlEnabled = false; audioSource.BeamAngleMode = BeamAngleMode.Adaptive; var audioStream = audioSource.Start();

Speech API improvements

Significant changes:

• Beta 2 of Kinect for Windows used Speech SDK v10.2. We now use Speech SDK v11
• Our new runtime setup installs Speech runtimes (x86 and/or x64) and the Kinect language pack (en-us).

Relevant samples (installed with SDK):

• KinectAudioDemo (C#/WPF)
• ShapeGame (C#/WPF)
• Speech (C#/console)

Kinect for Windows – Code Migration from Beta2 to v1.0 (C#/VB)

[This article covers the C#/VB accessible APIs. It has a peer article that covers the C++ accessible APIs.]

There have been a number of significant changes and improvements in our APIs since Beta2. This set of documents attempts to detail the changes to facilitate code migration from beta 2 to v1.

[Update – 9:44am PST, 2/1 – fixed link to migration dll below. If you still have problems, clearing browser cache may help.]

API Change Details

[these links take you to a seperate post with details]

• Namespace and Assembly Name changes
• Runtime type (Rename to KinectSensor, refactoring)
• ColorImage API changes
• DepthImage API changes
• Skeleton API changes
• Mapping API changes (Skeleton -> Depth, Depth -> Color)
• Audio API changes
• Speech API changes

Adapting to API Changes

In order to adapt to this change, C#/VB developers should:

• Backup code projects (if not using source control, such as TFS)
• Uninstall Beta 2 SDK (including speech runtime 10.x components).
• Install Kinect for Windows SDK v1.
• Migrate code to use v1 APIs and best practices via one of the migration techniques below
• If you need help with the answer, go the Kinect for Windows forums. Search for other similar issues first. If needed, create a new issue. Please prefix the title of the forum post with “Migration issue:”. (for example: “Migration Issue: Runtime.Sensors replacement?”)
• If you figured out the answer, and want to suggest improvements to the details and techniques in these documents, please leave a comment (We generally will respond to that feedback, but won’t show the comments, as it may get unwieldy).

Where to go for help

Code Migration Techniques (C#/VB)

There are three major approaches to migration

1)      RECOMMENDED: Use a migration reference assembly to assist with learning about API changes

2)      RECOMMENDED: Go back to the examples you may have started from, find the newest versions of those, and transfer your improvements to this new version.

3)      NOT RECOMMENDED: Change reference assembly, (recompile -> search for info -> change -> adapt -> repeat)

Using the Migration reference assembly (RECOMMENDED)

Backup code projects (if not using source   control, such as TFS)   Uninstall Beta 2 SDK (including speech runtime   10.x components).   Install Kinect for Windows SDK v1.   Remove Microsoft.Research.Kinect.dll   references   Download (link is fixed now, may need to clear browser cache if still having problems) and add reference to Microsoft.Kinect.Migration.dll   [Ensure all projects, including binary dependencies,   are migrated]   Build and evaluate errors/warnings After changing all the “usings” of the old namespace to “using   Microsoft.Kinect;”, most of the errors/warnings will point you to renames of   types/members:   Look for guidance in errors/warnings   Look at sample code in SDK to see changes in   event handling code necessary (there have been significant changes with   ImageFrame and SkeletonFrame).   Change code as necessary -> recompile ->   (repeat)   After building successfully with no errors and   warnings (related to name changes)   Remove reference to   Microsoft.Kinect.Migration.dll   Add reference to Microsoft.Kinect.dll   Run application and debug as necessary

Doing it all manually (NOT RECOMMENDED)

[WE STRONGLY RECOMMEND YOU   FOLLOW THE MIGRATION DLL TECHNIQUE ABOVE…AND DON’T FOLLOW THE MANUAL   TECHNIQUE]   Backup code projects (if not using source   control, such as TFS)   Uninstall Beta 2 SDK (including speech runtime   10.x components).   Install Kinect for Windows SDK v1.   Remove Microsoft.Research.Kinect.dll   references   Add reference to Microsoft.Kinect.dll   [Ensure all projects, including binary dependencies,   are migrated]   Build and evaluate errors/warnings   Look at sample code in SDK to see changes in   event handling code necessary (there have been significant changes with   ImageFrame and SkeletonFrame).   Change code as necessary -> recompile ->   (repeat)   Run application and debug as necessary

Kinect for Windows Support Info Coming Soon

[Temporary placeholder page]

Normally, you would have been taken to the Kinect for Windows support site for “more info” about the situation the Kinect sensor UI was in.

The support site with end-user help for issues is coming soon.

Kinect Apps – ensuring Kinect Runtime is installed

Kinect for Windows 1.0 enabled apps should ensure that the Kinect Runtime is installed wherever the app is installed.

[Note: Kinect for Windows 1.0’s latest public preview is beta 2. Parts of this blog post may be applicable to beta 2, but is primarily focused on the final v1.0 version, coming February 1st. Since v1.0 is not yet released, information I give here may change when it does release. I also am filtering this information to ensure that I am not giving away details that we don’t yet want to release.]

Guidelines

App installers should install its dependencies, including Kinect Runtime – CRITICAL

Kinect for Windows 1.0 will have a KinectRuntime-v1.0-Setup.exe that your app installer MUST chain install. In addition to installing Kinect specific software (drivers + runtime), KinectRuntime-v1.0-Setup.exe will ensure the following dependencies are installed:

• VCRT x86 and/or x64
• .NET 4 client profile (or later 4.x versions like .NET 4.5)
• Microsoft Speech Runtime v11 x86 and/or x64

Running an app should provide a decent user experience if Kinect Runtime isn’t installed – RECOMMENDED

• .NET Applications

If your Kinect App is a .NET 4.x based app (WPF, XNA, WindowsForms, etc…), code like the following sample may help improve your apps user experience in case the Kinect Runtime 1.0 is not installed.

    //App.xaml.cs
    public partial class App : Application
    {
        protected override void OnStartup(StartupEventArgs e)
        {
            if (IsKinectRuntimeInstalled)
            {
                //go ahead and load the StartupUri as defined in App.xaml
                base.OnStartup(e);
            }
            else
            {
                MessageBox.Show(Assembly.GetExecutingAssembly().FullName + " is not able to excecute."
                + "An important dependency should have been installed by its setup program: Microsoft Kinect Runtime 1.0");
            }
        }

        public bool IsKinectRuntimeInstalled {
            get
            {
                bool isInstalled;
                try
                {
                    TestForKinectTypeLoadException();
                    isInstalled = true;
                }
                catch (FileNotFoundException)
                {
                    isInstalled = false;
                }
                return isInstalled;
            }
        }

        // This Microsoft.Kinect.dll based type, must be isolated in its own method
        // as the CLR will attempt to load the Microsoft.Kinect.dll assembly it when this method is executed.
        private void TestForKinectTypeLoadException()
        {
#pragma warning disable 219 //ignore the fact that status is unused code after this set.
            var status = KinectStatus.Disconnected;
#pragma warning restore 219
        }
    }

• Native Applications

If your Kinect App is a Native Windows based app (C++ using GDI/GDI+/D2D/DirectX/etc..), your application should handle a missing Kinect10.dll in a way that informs the user of the problem and how to fix it.

Examples of depth API improvements coming in v1

As a program manager working on Kinect for Windows, I’m always excited to see feedback about Kinect for Windows. It is even better when we already have those issues fixed.

Piers7 posted “Gotchas with the Kinect SDK for Windows” recently. I’m happy to say that we’ve already got fixes for the first 3 gotchas he lists:

1. DepthData in v1 will no longer be mirrored improperly depending if you ask for Depth or Depth+Player.
2. DepthData data representation in v1 will no longer be different if you ask for Depth or Depth+Player. We always will use 3 bits to represent player id even if you haven’t asked for Depth+Player. In that case, they will just be “0”
3. APIs for Mapping from Skeleton->Depth, Depth->Skeleton, Depth->Color in v1 will no longer require bit shifting of the depth data.

Our v1 release in early 2012 will have these improvements and more…

We’re also investigating the best way to make GreenScreening easy and approachable, which is his 4th gotcha

Mobile developers and Kinect

I’ll Be Working on the Railroad

I recently had a rowing regatta in Portland. I had to mix some work in that weekend, so I took the train. I booked a business class ticket ($15 more each way). I was working on slides and demos for my DevConnections talk on 11/2 in Las Vegas about Kinect beta 2, v1 and beyond.

Here is a picture of my laptop and Kinect plugged in, and a demo app running on the train:

 

SkeletonViewer-WPF App

The app running in that photo is an evolution of our SDK’s SkeletonViewer-WPF app. Beta 2 included this new componentized SkeletonViewer app. KinectColorViwer and KinectDepthViewer are side by side in the app window.

On that trip I evolved that app to have a new component – KinectSkeletonViewer which overlapped both the color image and the depth image and rendered the skeleton on top of the color or depth image. In the beta 2 version of that app, the skeleton viewer code is part of KinectDiagnosticViewer. This new componentization work didn’t get to ship in Kinect for Windows v1-beta2, but we’ll get it out to devs in the future.

Recording/Playback of Kinect Data

One of the things I demoed at DevConnections was an Xbox SDK tool to record and playback Kinect data. That would have been easier for my train based development. I could have just used the pre-recorded Kinect data (color, depth, etc…) instead of bringing the Kinect with me. I’ll be excited when we can deliver that for Kinect for Windows developers too.

My shift from WPF-XAML to Kinect for Windows

New Team @ Microsoft for Rob

I’m 6 months into my new team (Kinect for Windows) and I haven’t even mentioned it on my blog. My twitter self (@rrelyea) announced my new gig on June 20^th, 2011:

(in November 2011, we moved to use @KinectWindows and #kinectforwindows)

Perhaps I haven’t blogged about it yet because twitter is a place where you can say less. It was difficult to talk about some of these things due to the fact that some key facts were Microsoft confidential until the Build conference in September.

When I announced my team move, I had already been working almost full time on Kinect for about a month. That day was just the official start date. By total coincidence, that day was the day that (reportedly) Soma sent an email announcing changes for the XAML, WPF and Silverlight teams. Although I knew org changes might happen in the future in that space, I had thought that it would wait until after Build.

WPF 4.5 Project

After WPF 4 shipped in April 2010, I was asked to move from my XAML Architect role to leading the Program Management team for WPF 4.5. That role also had me working closely with the heads of the Developer and Test team. The 3 of us were referred to as the WPF Triad – the leadership of the WPF team. The role was an exciting, and challenging one for me. It stretched me in new directions. We were leading the team doing the planning (see pictures of post it notes from my office) and execution of the entire release. That was a new scope for me – which I enjoyed.

I did a PDC 2010 talk about WPF Today and Tomorrow. There I shared some of our plans for WPF 4.5. Build 2011 was rightly focused primarily on Windows 8, so there was no talk focused on improvements in WPF. However, it was during Build, that a preview of WPF 4.5 first became available (in the .NET 4.5 Developer Preview). I’m excited about WPF 4.5 for existing customers and also excited about the fact that Windows 8 has a strong focus on leveraging HTML or XAML skillsets.

Rob 3.0?

In early months of this year, I had decided to do soul searching to figure out what was next for me. We had reached code complete on WPF 4.5. Although I was enjoying much about my work, it was the first time I had found myself in a position that was deemed important, but much less important than several other efforts.

I thought about the following options:

• Delay a change until WPF 4.5 ships.
• Join a sister team of WPF. The XAML team had lots of needs/opportunities, as the Windows 8 based project needed significant more focus in its v1.
• Join a XAML ecosystem related team – Expression, Visual Studio, etc…
• Join a HTML ecosystem related team – a platform or tools team (leveraging my earlier roots from 1998-2001).
• Join an entirely different team inside of Microsoft – Windows, Windows Phone, Bing, Azure, Xbox, …

As part of this soul search, I even considered leaving Microsoft – in my 18^th year, it was the first time I ever seriously thought about that. I talked with 2 friends who had left the company for other prestigious companies. After dipping my toe in the water, I decided that I would focus on finding a great role at Microsoft.

Effectively, I was trying to decide whether it was time for Rob 2.1 or 3.0. Rob 1.0 was my 5 years working as a Developer Evangelist-ish for Microsoft in Chicago. Rob 2.0 was my 13 years working on HTML programmability, XAML UI programmability, and the XAML language and engines. I had last done a major soul search in 1998 when I joined the IE team. During next 13 years I had never changed my group – IE, “Avalon”, WPF, and XAML – the groups may have reorged and changed focuses, but I never even considered a job shift.

Kinect for Windows is it!

In the end, I picked the Kinect for Windows role. I was lucky and excited to get a great opportunity to work on something so exciting. It had an exciting new focus for me – Natural User Interface (NUI) and the magic of Kinect. The Kinect Effect shows some of the possibilities. Couldn’t be enjoying it more…

Starting another blog – WindowsClient.net

I’m not positive what I’m going to do with this blog, but I’m heavily considering moving most of my blogging over to WindowsClient.net.  Just did my first post: Rob start his WindowsClient.net Era – 3rd blogs a charm (http://blogs.windowsclient.net/rob_relyea/archive/2008/02/10/3rd-time-is-a-charm.aspx)

Josh Twist: Xaml Editing with Intellisense with no VS Designer

I knew you could change the default from WPF Designer to XML Editor (and lose all the intellisense), but I didn’t know this tip that Josh shows about Xaml editing in VS…

Visualization of progress for self measurement and public pressure/support

The other day I blogged in "My car runs on oil from a baby seal" that people would be able to conserve gasoline, reduce driving, etc.. better if they could visualize the amount of gallons they used per day (GPD).

Now I tripped over this site that talks about a similar measurement, visualization and motivation technique that Jerry Seinfeld used (and maybe still does).