Kinect for Windows – Details of API Changes from Beta2 to v1.0 (C#/VB)
[This article covers the details of changes of C#/VB accessible
APIs. Please ensure you read the code
migration post as well.]
Namespace and Assembly Name Changes
Since our Beta 1 (June 2011) release of the Kinect for
Windows SDK , our C#/VB accessible APIs have been provided via
Microsoft.Research.Kinect.dll. The public Types all were organized into 2 namespaces:
Microsoft.Research.Kinect.Nui and Microsoft.Research.Kinect.Audio.
Our team made the decision back in early 2011 that we would mark our beta
releases of APIs as “Research” to indicate the fact that this was an early
version of our APIs, created in concert with Microsoft Research.
On February 1, 2012, we released our final v1.0 SDK. In that
release, our DLL is now called Microsoft.Kinect.dll. All of our public APIs are
in the Microsoft.Kinect namespace.
Release |
Assembly Names |
Namespaces |
Beta 2 (November 2011) |
Microsoft.Research.Kinect.dll |
Microsoft.Research.Kinect.Nui Microsoft.Research.Kinect.Audio |
V1.0 (Feb 2012) |
Microsoft.Kinect.dll |
Microsoft.Kinect |
Runtime type (Rename to KinectSensor, refactoring)
Significant changes:
-
Rename of top level class – Runtime à KinectSensor
-
Removal of Camera class.
-
Several renames/refactorings.
-
Change in initialization of KinectSensor. (Initialize/Uninitialize)
-
Addition of several Mapping APIs. See details in Mapping API changes details.
-
Addition of several new KinectStatus values.
KINECTSENSOR API – SUMMARY OF API
|
<NamespaceOldName=“Microsoft.Research.Kinect.Nui“Name=“Microsoft.Kinect“> <TypeOldName=“Runtime“Name=“KinectSensor“> <MemberOldName=“Kinects“Name=“KinectSensors“ /> <MemberOldName=“Initialize“Name=“Start“ /> <MemberOldName=“Uninitialize“Name=“Stop“ /> <MemberName=“Dispose“Added=“True“ />
<MemberOldName=“VideoStream“Name=“ColorStream“ /> <MemberOldName=“VideoFrameReady“Name=“ColorFrameReady“ />
<MemberOldName=“SkeletonEngine“
<MemberName=“AllFramesReady“
<MemberName=“InstanceIndex“ <MemberOldName=“InstanceName“Name=“DeviceConnectionId“ /> <MemberName=“UniqueKinectId“
<MemberName=“NuiCamera“Removed=“True“/> <MemberOldName=“ElevationMaximum“Name=“MaxElevationAngle“MovedFrom=“Camera“ /> <MemberOldName=“ElevationMinimum“Name=“MinElevationAngle“MovedFrom=“Camera“ /> <MemberName=“ElevationAngle“
<MemberName=“MapDepthFrameToColorFrame“Added=“True“ /> <MemberName=“MapDepthToColorImagePoint“Added=“True“ /> <MemberName=“MapDepthToSkeletonPoint“Added=“True“ /> <MemberName=“MapSkeletonPointToDepth“Added=“True“ /> <MemberName=“MapSkeletonPointToColor“Added=“True“ /> </Type> <TypeName=“RuntimeOptions“Removed=“True“Message=“Enable the streams from <TypeOldName=“KinectDeviceCollection“Name=“KinectSensorCollection“> <MemberName=“Dispose“Added=“True“ /> </Type> <TypeName=“StatusChangedEventArgs“> <MemberOldName=“KinectRuntime“ </Type> <TypeName=“Camera“Removed=“True“> <MemberOldName=“GetColorPixelCoordinatesFromDepthPixel“Name=“MapToColorPixel“MovedTo=“DepthImageFrame“ /> <MemberOldName=“ElevationMaximum“Name=“MaxElevationAngle“MovedTo=“KinectSensor“ /> <MemberOldName=“ElevationMinimum“Name=“MinElevationAngle“MovedTo=“KinectSensor“ /> <MemberName=“ElevationAngle“ </Type> <TypeName=“KinectStatus“> <MemberName=“Undefined“Added=“True“ /> <MemberName=“Initializing“Added=“True“ /> <MemberName=“DeviceNotGenuine“Added=“True“ /> <MemberName=“DeviceNotSupported“Added=“True“ /> <MemberName=“InsufficientBandwidth“Added=“True“ /> </Type> </Namespace> |
KinectSensor discovery
In beta2 (November 2011), we introduced a KinectSensors property on the Runtime object. We also
introduced a StatusChanged event.
In v1 (Feb 2012), those continue to be the key concepts to
understand when your program needs to work with 1 or more sensors.
It is critical that Kinect-enabled applications can deal
appropriately with the situations that users will hit with their application
(no Kinect sensor connected to computer, Kinect sensor is not powered by A/C
power, insufficient USB bandwidth, etc…)
Please read the documentation under “Programming
Guide/Kinect Sensor/Kinect-Enabled Application”:
Relevant samples (installed with SDK):
-
ShapeGame (C#/WPF) – KinectSensorChooser component
-
KinectAudioDemo (C#/WPF) – KinectSensorChooser component
-
KinectExplorer (C#/WPF)
-
SkeletonViewer (C++/Direct2D+GDI)
KinectSensor
initialization
Our beta APIs had a set of hard to understand and discover
API calls to configure the services you needed to get from the sensor. We
worked to make this part of our API more approachable. Our design goals were:
-
Enable a developer to ask once, not several
times for a feature. -
Valid settings are determinable via intellisense/compiling, not just runtime errors
-
Good default behavior.
-
Dynamic changing of any setting over time.
OLD CODE |
EQUIVALENT NEW CODE |
runtime.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseColor); runtime.VideoStream.Open(ImageStreamType.Video, 2, runtime.DepthStream.Open(ImageStreamType.Depth, 2, |
kinectSensor.ColorStream.Enable(); kinectSensor.DepthStream.Enable(); kinectSensor.SkeletonStream.Enable(); kinectSensor.Start(); |
The old way doesn’t meet You needed to ask twice – once in Init() and once in Open().
|
The ColorImageStream, DepthImageStream, |
Types/APIs to explore: KinectSensor, ColorImageStream, DepthImageStream,
SkeletonStream.
KinectSensor uninitialize
OLD CODE |
NEW CODE |
runtime.Uninitialize(); |
kinectSensor.Stop(); |
ColorImage API changes
Significant changes:
-
Reorganization of colorImage
data. There is no longer a PlanarImage class. -
Developer is now responsible for own allocations
of pixelData variables and copying pixelData to that storage. (Color pixelData
is still a byte array with same layout as beta2.) -
This enables the Kinect Runtime to better reuse
resources if an application’s data processing falls behind. -
Availability of colorImage
data during ColorFrameReady or AllFramesReady
events. (AllFramesReady gives matching color, depth
and/or skeleton frames.)
Relevant samples (installed with SDK):
-
KinectExplorer (C#/WPF) – KinectColorViewer component
-
SkeletonViewer (C++/Direct2D+GDI)
-
ShapeGame (C#/WPF) – KinectColorViewer component
COLORIMAGE API – SUMMARY OF |
<TypeName=“ColorImageStream“Added=“True“> <MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ /> <MemberName=“Disable“Added=“True“ /> <MemberOldName=“Open“Name=“Enable“ /> <MemberName=“StreamType“Removed=“True“ /> <MemberName=“Resolution“Removed=“True“ /> <MemberName=“Type“Removed=“True“ /> <MemberOldName=“Height“Name=“FrameHeight“ /> <MemberOldName=“Width“Name=“FrameWidth“ /> <MemberName=“CreateCompatibleImageFrame“Removed=“True“ /> </Type> <TypeName=“ImageFrameReadyEventArgs“Removed=“True“Message=“Use ColorImageFrameReadyEventArgs or DepthImageFrameReadyEventArgs </Type> <TypeName=“ColorImageFrameReadyEventArgs“Added=“True“> <MemberOldName=“ImageFrame“Name=“OpenColorImageFrame“ /> </Type> <TypeName=“ColorImageFrame“Added=“True“> <MemberName=“Type“Removed=“True“ /> <MemberName=“Resolution“Removed=“True“ /> <MemberName=“Image“Removed=“True“ /> <MemberName=“ViewArea“Removed=“True“ /> <MemberName=“ImageFrame“Removed=“True“ /> <MemberName=“BytesPerPixel“MovedFrom=“PlanarImage“ /> <MemberName=“Height“MovedFrom=“PlanarImage“ /> <MemberName=“Width“MovedFrom=“PlanarImage“ /> <MemberOldName=“Bits“Name=“PixelData“MovedFrom=“PlanarImage“ /> <MemberName=“CopyPixelDataTo“ /> <MemberName=“SourceStream“Removed=“True“ /> <MemberName=“PixelData“Removed=“True“Message=“use CopyPixelDataTo(…)“/> <MemberName=“Dispose“Added=“True“ /> </Type> <TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“
|
OLD CODE |
runtime.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);
runtime.VideoFrameReady += new
void ColorImageReady(object sender, ImageFrameReadyEventArgs e) { PlanarImage planarImage
// DISPLAY OR PROCESS IMAGE DATA IN planarImage HERE }
|
NEW CODE |
kinectSensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
kinectSensor.ColorFrameReady += new byte[] pixelData;
void ColorImageReady(object sender, ColorImageFrameReadyEventArgs e) { bool receivedData using (ColorImageFrame colorImageFrame = e.OpenColorImageFrame()) { if (colorImageFrame != null) { if { pixelData } colorImageFrame.CopyPixelDataTo(pixelData); receivedData } else { // apps // the } } if (receivedData) { // DISPLAY } }
|
DepthImage API changes
Significant changes:
-
Reorganization of depthImage data. There is no longer a PlanarImage class.
-
DepthPixel data is now a short[], instead of a byte[]. (Still 2 bytes of data representing each pixel.)
-
DepthPixel bit layout is now consistent, regardless if you have opted for Depth+Player or just Depth.
-
There is no longer a need to ask for Depth+Player or Depth. If the SkeletonStream is enabled, you get Depth+Player.
-
Developer is now responsible for own allocations of pixelData variables and copying pixelData to that storage.
This enables the Kinect Runtime to better reuse resources if an application’s data processing falls behind.
-
Availability of depthImage data during DepthFrameReady or AllFramesReady
events. (AllFramesReady gives matching color, depth and/or skeleton frames.)
Relevant samples (installed with SDK):
-
KinectExplorer (C#/WPF) – KinectDepthViewer component
-
SkeletonViewer (C++/Direct2D+GDI)
DEPTHIMAGE API – SUMMARY OF |
<TypeName=“ImageFrameReadyEventArgs“Removed=“True“Message=“Use ColorImageFrameReadyEventArgs or DepthImageFrameReadyEventArgs </Type> <TypeName=“DepthImageStream“Added=“True“> <MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ /> <MemberName=“IsTooFarRangeEnabled“ Added=“True“/> <MemberName=“Disable“Added=“True“ /> <MemberOldName=“Open“Name=“Enable“ /> <MemberName=“StreamType“Removed=“True“ /> <MemberName=“Resolution“Removed=“True“ /> <MemberName=“Type“Removed=“True“ /> <MemberOldName=“Height“Name=“FrameHeight“ /> <MemberOldName=“Width“Name=“FrameWidth“ /> <MemberName=“CreateCompatibleImageFrame“Removed=“True“ /> </Type> <TypeName=“DepthImageFrameReadyEventArgs“Added=“True“> <MemberOldName=“ImageFrame“Name=“OpenDepthImageFrame“ /> </Type> <TypeName=“DepthImageFrame“Added=“True“> <MemberOldName=“GetColorPixelCoordinatesFromDepthPixel“Name=“MapToColorImagePoint“MovedFrom=“DepthImageFrame“ /> <MemberOldName=“SkeletonToDepthImage“Name=“MapFromSkeletonPoint“MovedFrom=“SkeletonStream“ /> <MemberOldName=“DepthImageToSkeleton“Name=“MapToSkeletonPoint“MovedFrom=“SkeletonStream“ /> <MemberName=“PlayerIndexBitmask“Added=“True“ /> <MemberName=“PlayerIndexBitmaskWidth“Added=“True“ /> </Type> <TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“ <TypeName=“ImageDigitalZoom“Removed=“True“ /> <TypeName=“ImageStreamType“Removed=“True“ /> <TypeName=“ImageType“Removed=“True“ /> <TypeName=“ImageResolution“Removed=“True“ /> <TypeName=“PlanarImage“Removed=“True“Message=“Data moved to ColorImageFrame/DepthImageFrame“ <TypeName=“ImageViewArea“ <TypeName=“DepthImageFormat“ Added=“True“/> <TypeName=“DepthRange“ Added=“True“/> <TypeName=“DepthImagePoint“Added=“True“ /> <TypeName=“DepthImagePointFloat“Added=“True“ /> |
OLD CODE |
runtime.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex);
runtime.DepthFrameReady += new
private void { PlanarImage planarImage
// DISPLAY OR } |
NEW CODE |
kinectSensor.DepthStream.Enable(DepthImageFormat.Resolution320x240Fps30);
kinectSensor.DepthFrameReady += new short[] pixelData;
private void { bool receivedData using (DepthImageFrame depthImageFrame = e.OpenDepthImageFrame()) { if (depthImageFrame != null) { if (pixelData == null) //allocate the first time { pixelData } depthImageFrame.CopyPixelDataTo(pixelData); receivedData } else { // apps // the } } if (receivedData) { // DISPLAY } }
|
Skeleton API improvements
Significant changes:
-
Renamed SkeletonEngine -> SkeletonStream
-
Renamed SkeletonData -> Skeleton
-
Renamed JointID -> Joint
-
Renamed JointsCollection -> JointCollection
-
Renamed Vector -> SkeletonPoint
-
Renamed SkeletonQuality -> FrameEdges
-
Developer is now responsible for own allocations of skeleton array variables and copying skeletons to that storage.
This enables the Kinect Runtime to better reuse
resources if an application’s data processing falls behind.
-
Availability of skeleton data during SkeletonFrameReady or AllFramesReady
events. (AllFramesReady gives matching color, depth
and/or skeleton frames.) -
New SkeletonStream.AppChoosesSkeletons property + ChooseSkeletons() methods.
Relevant samples (installed with SDK):
-
KinectExplorer (C#/WPF) – KinectSkeletalViewer component
-
SkeletonViewer (C++/Direct2D+GDI)
-
ShapeGame (C#/WPF)
SKELETON API – SUMMARY OF API |
<TypeOldName=“SkeletonEngine“ <MemberName=“Disable“Added=“True“ /> <MemberName=“Enable“Added=“True“ /> <MemberName=“ChooseSkeletons“Added=“True“ /> <MemberOldName=“GetNextFrame“Name=“OpenNextFrame“ /> <MemberName=“DepthImageToSkeleton“MovedTo=“DepthImageFrame“ /> <MemberName=“SkeletonToDepthImage“MovedTo=“DepthImageFrame“ /> </Type> <TypeName=“SkeletonFrameReadyEventArgs“> <MemberOldName=“SkeletonFrame“ </Type> <TypeName=“SkeletonFrame“> <MemberOldName=“TimeStamp“Name=“Timestamp“ /> <MemberOldName=“Skeletons“Name=“CopySkeletonDataTo“ /> <MemberName=“Dispose“Added=“True“ /> <MemberName=“Quality“Removed=“True“ /> </Type> <TypeOldName=“SkeletonData“Name=“Skeleton“ > <MemberOldName=“TrackingID“Name=“TrackingId“ />
</Type> <TypeName=“JointType“Added=“True“/> <TypeOldName=“JointID“Name=“Joint“> <MemberOldName=“ID“Name=“JointType“ /> <MemberOldName=“Vector“Name=“Position“ /> <MemberName=“Parent“Removed=“True“ /> </Type> <TypeOldName=“JointsCollection“Name=“JointCollection“ /> <TypeOldName=“Vector“Name=“SkeletonPoint“> <MemberName=“W“Removed=“True“ /> </Type> <TypeOldName=“SkeletonQuality“Name=“FrameEdges“> <MemberOldName=“ClippedLeft“Name=“Left“ /> <MemberOldName=“ClippedTop“Name=“Top“ /> <MemberOldName=“ClippedRight“Name=“Right“ /> <MemberOldName=“ClippedBottom“ <MemberName=“None“Added=“True“ /> </Type> <TypeName=“SkeletonFrameQuality“Removed=“True“ />
|
OLD CODE |
runtime.SkeletonFrameReady += new
private void { SkeletonFrame skeletonFrame
}
|
NEW CODE |
kinectSensor.SkeletonFrameReady += new
Skeleton[] skeletons;
private void { bool receivedData using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame()) { if (skeletonFrame != null) { if { skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength]; } receivedData } else { // apps // the } } if (receivedData) { // DISPLAY } }
|
Mapping API improvements (Skeleton
-> Depth, Depth -> Color)
Significant changes:
-
Moved these members to DepthImageFrame and KinectSensor
-
Added a new “entire frame” instead of per pixel function. (KinectSensor.MapDepthFrameToColorFrame)
-
Added a new Skeleton to Color method, which is similar to calling SkeletonToDepth and then DepthToColor.
-
The DepthImageFrame versions of these methods take fewer parameter, since
it has the appropriate depthImage. -
New return types: SkeletonPoint (replacing Vector) and DepthImagePoint (replacing 2 “out” parameters).
Relevant samples (installed with SDK):
-
KinectExplorer (C#/WPF) –KinectSkeletalViewer component
-
SkeletonViewer (C++/Direct2D+GDI)
Audio API improvements
Relevant samples (installed with SDK):
<Namespace OldName=“Microsoft.Research.Kinect.Audio“ <Type Name=“AudioDeviceInfo“ Removed=“True“ /> <Type Name=“KinectAudioSource“> <Member Name=“FindCaptureDevices“ Removed=“True“ /> <Member Name=“RetrieveTsStats“ Removed=“True“ /> <Member Name=“QualityMetrics“ Removed=“True“ /> <Member Name=“DevicePairGuid“ Removed=“True“ /> <Member Name=“KinectAudioSource“ Removed=“True“ /> <Member OldName=“AutomaticGainControl“ Name=“AutomaticGainControlEnabled“ /> <Member OldName=“SoundSourcePositionConfidence“ Name=“SoundSourceAngleConfidence“ /> <Member OldName=“MicArrayMode“ Name=“BeamAngleMode“ /> <Member OldName=“MicArrayBeamAngle“ Name=“ManualBeamAngle“ /> <Member OldName=“BeamChanged“ Name=“BeamAngleChanged“ /> <Member Name=“CenterClip“ Removed=“True“/> <Member Name=“EchoLength“ Removed=“True“/> <Member Name=“FrameSize“ Removed=“True“/> <Member Name=“GainBounder“ Removed=“True“/> <Member Name=“MicArrayPreprocess“ Removed=“True“/> <Member Name=“MicrophoneIndex“ Removed=“True“/> <Member Name=“NoiseFill“ Removed=“True“/> <Member Name=“SourceMode“ Removed=“True“/> <Member Name=“VoiceActivityDetector“ Removed=“True“/> <Member Name=“Dispose“ Removed=“True“ <Type Name=“MicArrayMode“ Removed=“True“ /> <Type Name=“SystemMode“ Removed=“True“ Message=“Use EchoCancellationMode <Type OldName=“BeamChangedEventArgs“ Name=“BeamAngleChangedEventArgs“/> <Type Name=“SoundSourceAngleChangedEventArgs“ Added=“True“ /> <Type Name=“EchoCancellationMode“ Added=“True“ /> |
KinectAudioSource audioSource audioSource.SystemMode = SystemMode.OptibeamArrayOnly; audioSource.FeatureMode = true; audioSource.AutomaticGainControl = false; audioSource.MicArrayMode = MicArrayMode.MicArrayAdaptiveBeam; |
KinectAudioSource audioSource = kinectSensor.AudioSource; audioSource.EchoCancellationMode = EchoCancellationMode.CancellationOnly; audioSource.AutomaticGainControlEnabled = false; |
Speech API improvements
Relevant samples (installed with SDK):
-
KinectAudioDemo (C#/WPF)
-
ShapeGame (C#/WPF)
-
Speech (C#/console)
Your old code/equivalent new code is unreadable. I can’t see more than 20 characters of the new code.
Would you be able to post VB code for the skelton sample project using the new API?
@Phil: if you have a Touch Pad or Mouse with Horizontal Scrolling, you can scroll to see more of the code.
Alternatively, you can just select the code from start to finish then copy-paste to notepad to see the whole code.
This just because a CSS glitch causing the browser to not show a scroll bar :S
hey, could you do a simple tutorial for beginners with this new API?
I can’t found tutorials in internet for this new version.
thanks
Do you know where I can find the KinectExplorer sample in VB?
Great but how do i capture frames one by one like a picture in code? Thats what im trying to do but I think im doing something wrong because I have the depth stream like above in code but its not working and the depth stream displays correctly on image control if I remove my section of code from received data.
Im using gifbitmap encoder.
Jamie take a look at kinect mouse for skeleton tracking because they do skeleton tracking in their sample code and its in v1. Remember to remove readme.txt file because it cause you to not be able to compile the project.
hi everyone!
Sorry for my bad English.
void SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
if (e.SkeletonFrame.Skeletons.Count() == 0) return;
SkeletonFrame skeletonSet = e.SkeletonFrame;
SkeletonData firstPerson = (from s in skeletonSet.Skeletons
where s.TrackingState == SkeletonTrackingState.Tracked
orderby s.UserIndex descending
select s).FirstOrDefault();
if (firstPerson == null) return;
JointsCollection joints = firstPerson.Joints;
Joint rightHand = joints[JointID.HandRight];
Joint leftHand = joints[JointID.HandLeft];
//find which hand is being used for cursor – is the user right handed or left handed?
var joinCursorHand = (rightHand.Position.Y > leftHand.Position.Y)
? rightHand
: leftHand;
float posX = joinCursorHand.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight).Position.X;
float posY = joinCursorHand.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight).Position.Y;
Joint scaledCursorJoint = new Joint
{
TrackingState = JointTrackingState.Tracked,
Position = new Microsoft.Research.Kinect.Nui.Vector
{
X = posX,
Y = posY,
Z = joinCursorHand.Position.Z
}
};
OnButtonLocationChanged(kinectButton, buttons, (int)scaledCursorJoint.Position.X, (int)scaledCursorJoint.Position.Y);
}
would like u guys, anyone know how to convert these code from BETA 2 to SDK V1.
thanks in advance!
Hi Rob,
I’m rather new at this… I was using the beta version with the runtime class in a .NET environment (runtime.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseColor). We have now bought a Kinect for Windows and I want to use the new SDK driver. The question is that when I select de KinectSensor class I’m told there is not a “public constructor” and I receive an error 1172. I guess I need that public constructor so as I can use the members Start, Stop, VideoStream, etc… What should I do?