使用Kinect传感器识别坐姿(译文)
By robot-v1.0
本文链接 https://www.kyfws.com/games/sitting-posture-recognition-with-kinnect-sensor-zh/
版权声明 本博客所有文章除特别声明外,均采用 BY-NC-SA 许可协议。转载请注明出处!
- 8 分钟阅读 - 3958 个词 阅读量 0使用Kinect传感器识别坐姿(译文)
原文地址:https://www.codeproject.com/Articles/260741/Sitting-posture-recognition-with-Kinnect-sensor
原文作者:Gavrilov Alexey
译文由本站 robot-v1.0 翻译
前言
Recognition of concentrating, non-concentrating, sleeping, and raise-hand postures.
识别集中注意力,不集中注意力,睡觉和举手姿势.
- 下载使用Microft Kinect Research SDK进行识别的源代码-13.3 KB(Download source code for recognition using Microft Kinect Research SDK - 13.3 KB)
- 下载用于使用OpenNi/PrimeSense Nite进行识别的源代码-39.9 KB(Download source code for recognition using OpenNi/PrimeSense Nite - 39.9 KB)
介绍(Introduction)
许多专家预测,在不久的将来,信息技术将发生新的革命.这场革命将与新的计算机功能相联系,以分割,跟踪和理解人类的姿势,手势和情感表达.为此,计算机必须开始使用提供3D视频的新型视频传感器. Kinect传感器是此类新型传感器中的第一个. Kinect传感器有两个摄像头:传统的彩色摄像机和测量深度,位置和运动的红外光传感器. Kinnect传感器大约在一年前开始作为XBox 360游戏系统的传感器,但几乎立即,许多软件开发人员开始尝试将其用于识别人类的姿势和手势.有关更多信息,请参阅(Many specialists are predicting that in the closer future, a new revolution in information technologies will occur. This revolution will be connected with new computer abilities to segment, track, and understand pose, gestures, and emotional expressions of humans. For this, computers must begin to use new types of video sensors that will provide 3D-videos. The Kinect sensor is the first of such new types of sensors. The Kinect sensor has two cameras: a traditional color video camera and an infrared light sensor that measures depth, position, and motion. The Kinnect sensor started as a sensor for the XBox 360 game system about an year ago, but almost immediately many software developers began to try to use it for recognition of human poses and gestures. More information about can be read from) www.kinecthacks.com(www.kinecthacks.com) .(.)
我的文章专门研究坐姿识别.坐姿识别基于人体骨骼跟踪.有三种可以使用Kinect传感器产生人体骨骼跟踪的软件包:OpenNi/PrimeSense Nite库,Micosoft Kinnect Research SDK和Libfreenet库.我用了前两个.在他们的基础上,我开发了C#WPF应用程序,在其中组合了彩色视频流和骨架图像.(My article is devoted to research of sitting posture recognition. Sitting posture recognition is based on human skeleton tracking. There are three software packages that may produce human skeleton tracking with the Kinect sensor: OpenNi/PrimeSense Nite library, Micosoft Kinnect Research SDK, and the Libfreenet library. I have used the first two. On their basis, I developed C# WPF applications where I combined color video streams and skeleton images.) 这些应用程序在Microsoft Windows 7和.NET Framework 4.0下运行.对于它们的编译,您需要Microsoft Visual Studio2010.您可以在以下位置找到安装OpenNi/PrimeSense Nite库和Microsoft Kinect Research SDK的说明.(These applications run under Microsoft Windows 7 and .NET Framework 4.0. For their compilation, you need Microsoft Visual Studio 2010. You may find instructions to install the OpenNi/PrimeSense Nite library and the Microsoft Kinect Research SDK at) www.kinecthacks.com(www.kinecthacks.com) .(.)
背景(Background)
坐姿识别算法基于人体骨骼跟踪并获得三个坐标(x(Sitting posture recognition algorithm is based on human skeleton tracking and obtaining three coordinates (x)s(s),Y(, y)s(s)z(, z)s(s)), (X(), (x)H(h),Y(, y)H(h)z(, z)H(h))和(x(), and (x)ķ(k),Y(, y)ķ(k)z(, z)ķ(k))人类的位置() of the positions of the human)肩(Shoulder)(表示为S),((denoted as S),)臀围(Hip)(表示为H),以及((denoted as H), and)膝盖(Knee)(表示为K).((denoted as K).) 坐姿与(A sitting posture is related to the)角度(angle)**一种(a)**在HK线(从臀部到膝盖)和HS线(从臀部到肩膀)之间.(between the line HK (from hip to knee) and the line HS (from hip to shoulder).) 我们将区分左身体部位(We will distinguish the left body part)角度(angle)一种(a)-“中髋部至左膝"向量和"中髋部至中肩"向量与右身体部位之间的角度(- angle between “center hip to left knee” vector and “center hip to center shoulder” vector, and right body part)角度(angle)一种(a)-“中心臀部至左膝盖"向量与"中心臀部至中肩"向量之间的角度.(- angle between “center hip to left knee” vector and “center hip to center shoulder” vector.) 从(From)角度(angle)**一种(a)**以及手的位置,可以总结出人类的坐姿,并将其归为以下四种指定类型之一-睡觉,集中注意力,举手和不集中注意力,如下表所示.(and the hand’s position, the human sitting posture can be concluded and classified as one of four specified types - sleeping, concentrating, raising hand, and non-focusing, as given in the table below.)
角度,(Angle,)一种(a) | 手姿势(Hand posture) | 坐姿(Sitting posture) |
---|---|---|
0〜40(0 ~ 40) | down | sleeping |
40~80 | down | non-concentrating |
80~100 | down | concentrating |
up | 举手(raising hand) | |
100~180 | down | non-concentrating |
使用代码(Using the Code)
我在组合彩色视频流和骨架图像时遇到两个问题.(I had two problems combining a color video stream and a skeleton image.)
第一个问题是如何在窗口的一个控件中简单地定位它们.对于此问题,我对两个包含(The first problem was how to locate them simply in one control in a window. For this problem, I used a simple WPF form for both applications that contain a) StatusBar
控制和(control and a) Grid
面板.的(panel. The) Grid
面板包含一个(panel contains an) Image
和(and) Canvas
尺寸相同的控件.(control with the same size.)
<Window x:Class="RecognitionPose.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Title="User tracking with Microsoft SDK" Height="600"
Width="862" Loaded="Window_Loaded"
DataContext="{Binding}">
<DockPanel LastChildFill="True">
<StatusBar Name="statusBar"
MinHeight="40" DockPanel.Dock="Bottom">
<StatusBarItem>
<TextBlock Name="textBlock"
Background="LemonChiffon"
FontSize='10'> Ready </TextBlock>
</StatusBarItem>
</StatusBar>
<Grid DockPanel.Dock="Top">
<Image Name="imgCamera" Width="820"
ClipToBounds="True" Margin="10,0" />
<Canvas Width="820" Height="510"
Name="skeleton" ClipToBounds="True"/>
</Grid>
</DockPanel>
</Window>
同时使用OpenNI/PrimeSense Nite和Microsoft SDK的第二个问题是刷新视频帧和框架帧的事件是非同步发生的.(The second problem of working with both OpenNI/PrimeSense Nite and the Microsoft SDK is that the events of refreshing video frames and skeleton frames occur non-synchronously.)
对于此问题,对于Microsoft SDK,我调用main方法(For this problem, for the Microsoft SDK case, I call the main method) RecognizePose
我的(of my) Recognition
中的课程(class in the) SkeletonFrameReady
之后的事件处理程序(event handler after) imgCamera
并刷新骨架控件.的(and the skeleton controls are refreshed. The) SkeletonFrameRead
事件处理程序只需与(event handler simply synchronizes with the) VideoFrameReady
事件处理程序,方法是将当前视频帧复制到平面图像temp变量中:(event handler by copying the current video frame in the planar image temp variable:)
planarImage = ImageFrame.Image;
然后将这个临时变量复制到(and then copying this temp variable to) imgCamera.Source
在里面(in the) SkeletonFrameReady
事件处理程序:(event handler:)
imgCamera.Source = BitmapSource.Create(planarImage.Width, planarImage.Height,
194,194,PixelFormats.Bgr32, null, planarImage.Bits,
planarImage.Width * planarImage.BytesPerPixel);
对于OpenNi/PrimeSense Nite,我使用NuiVision库(For the OpenNi/PrimeSense Nite case, I use the NuiVision library) http://www.codeproject.com/Articles/169161/Kinect-and-WPF-Complete-body-tracking(http://www.codeproject.com/Articles/169161/Kinect-and-WPF-Complete-body-tracking) 撰写者(written by) Vangos蕨类植物(Vangos Pterneas) 用于视频帧和骨骼识别事件的同步.我称(for synchronization of the video frame and skeleton recognition events. I call the) RecognizePose
中的方法(method in the) UsersUpdated
该库的事件处理程序.(event handler of this library.)
对于坐姿识别,主要问题是为人类找到相对于识别稳定的Kinect传感器的距离和角度.为此,我在应用程序设置中添加了五个参数来控制算法行为:(For sitting posture recognition, the main problem was to find for the human, the distance and angle relative to the Kinect sensor where recognition is stable. For this purpose, I added five parameters in the application settings to control the algorithm behavior:)
-
isDebug
-如果为true,则在状态栏上显示有关当前人员位置的信息;(-if true then show information about the current human location on the status bar;) -
confidenceAngle
-控制左身体和右身体之间的差异(- control differences between the left part body and right part body)角度(angles)一种(a);如果该角度对于给定级别更大,则我们认为识别不稳定.(; if this angle is more for the given level, we assume that the recognition isn’t stable.) -
standPoseFactor
-控制坐姿和站立姿势之间的差异;如果当前人的身高乘以该因子大于站立姿势中的初始人类身高,则我们假设当前姿势也是站立姿势.(- control differences between the sitting and standing pose; if the current human height multiplication on this factor is more than the initial human height in standing pose, we assume that the current pose is standing pose too.) -
isAutomaticChoiceAngle
-自动定义之间的选择(- choice between automatic definition)角度(angle)**一种(a)**最接近相机(真)和计算(as nearest to camera (true) and calculation)角度(angle)**一种(a)**左身体和右身体之间的平均值(假)(as average (false) between the left part body and right part body)角度a(angles a);(;) -
shiftAngle
-从中减去移位角(- shift angle subtracted from)角度(angle)**一种(a)**删除骨架识别错误.(to delete skeleton recognitions error.) 我发现最稳定的就座识别发生在这些参数具有以下值时:(I found that the most stable sitting recognition occurs when these parameters have these values:) -
confidenceAngle=50 degree;
-
standPoseFactor=1.1;
-
isAutomaticChoiceAngle=true;
-
shiftAngle=20.
Kinect传感器位于地板上,Kinect传感器与就座的人之间的距离约为2米,并且人体相对于传感器呈45度角旋转.(The Kinect sensor is located on the floor, the distance between the Kinect sensor and sitting human is about 2 meters, and the human body in turned on a 45-degree angle relative to the sensor.)
人体位置的优势在于,Kinect传感器可以不断跟踪识别所需的人体部位:(Advantage of a sitting human location is that the Kinect sensor may constantly track the parts of the human body that are necessary for recognition:)
- 两个膝盖;(two knees;)
- 一只臀部(one hip;)
- 两个肩膀(two shoulders;)
- 两只手(two hands;)
- 头(head) 对于其他人工位置,情况并非如此.例如,对于正面位置,传感器确实无法跟踪臀部;对于轮廓位置,传感器仅跟踪身体的一部分:右或左.(For other human locations, this isn’t so. For example, for frontal location, the sensor really doesn’t track the hip; for profile location, the sensor tracks only one part of the body: right or left.)
兴趣点(Points of Interest)
我拍了两部关于使用这两个应用程序的电影:(I made two movies about using these two applications:)
- http://www.youtube.com/watch?v=OIoVTSreR0w(http://www.youtube.com/watch?v=OIoVTSreR0w) (使用Microsoft Research Kinect SDK识别)((recognition using Microsoft Research Kinect SDK))
- http://www.youtube.com/watch?NR=1&v=1-etFuPg7J4(http://www.youtube.com/watch?NR=1&v=1-etFuPg7J4) (使用PrimeSense Nite库识别)((recognition using PrimeSense Nite library)) 从电影中,我们可以得出结论,识别对于两个软件包都适用.但是,通过扩展识别稳定的就座人类位置区域,可以显着改善应用程序.为此,我们不能使用一个或两个以上的Kinect传感器.(From the movies, we can conclude that recognition works well for both software packages. However, the applications may be improved significantly by extending the sitting human location zone where the recognition is stable. For this, we must use not one but two or more Kinect sensors.) 我认为这些应用程序可用于需要控制坐姿的人类行为的任何区域.对于人的状态变得不专心或睡眠的情况,可以通过添加一些将发送警报,警报或紧急信号的反馈来增强应用程序.另一方面,此应用程序可用于大学,以收集有关研讨会和实验室期间学生活动的统计信息.这些应用程序将计算学生在研讨会期间集中精力或不集中精力的平均时间,举手的次数,并且教授可以在与学生进行的个人作品中解释此统计信息.(I think that these applications may be used in any area where it is necessary to control human behavior in sitting pose. For cases when human state becomes non-concentrating or sleeping, the applications may be enhanced by adding some feedback that will send an alarm, alert, or emergency signal. On the other hand, this application may be used in universities to collect statistics about student activity during seminars and labs. These applications will calculate the average time a student is concentrating or not-concentrating during a seminar, the number of times they are raising hands, and the professor can account this statistics in personal works with the student.)
许可
本文以及所有相关的源代码和文件均已获得The Code Project Open License (CPOL)的许可。
C# Kinect WPF Microsoft 新闻 翻译