Dynamic hand gesture recognition using rgb - D images for human-machine interaction

MINISTRY OF EDUCATION AND TRAINING
HANOI UNIVERSITY OF SCIENCE AND TECNOLOGY
THI HUONG GIANG DOAN
DYNAMIC HAND GESTURE RECOGNITION USING RGB-D
IMAGES FOR HUMAN-MACHINE INTERACTION
DOCTORAL THESIS OF
CONTROL ENGINEERING AND AUTOMATION
Hanoi 12−2017
MINISTRY OF EDUCATION AND TRAINING
HANOI UNIVERSITY OF SCIENCE AND TECNOLOGY
THI HUONG GIANG DOAN
DYNAMIC HAND GESTURE RECOGNITION USING
RGB-D IMAGES FOR HUMAN-MACHINE
INTERACTION
Specialty: Control Engineering and Automation
Specialty Code: 62520216
DOCTORAL THESIS OF
CONTROL ENGINEERING AND AUTOMATION
SUPERVISORS:
1. Dr. Hai Vu
2. Dr. Thanh Hai Tran
Hanoi 12−2017
DECLARATION OF AUTHORSHIP
I, Thi Huong Giang Doan, declare that the thesis titled, “Dynamic Hand Gesture
Recognition Using RGB-D Images for Human-Machine Interaction”, and the works
presented in it are my own. I confirm that:
 This work was done wholly or mainly while in candidature for a Ph.D. research
degree at Hanoi University of Science and Technology.
 Where any part of this thesis has previously been submitted for a degree or any
other qualification at Hanoi University of Science and Technology or any other
institution, this has been clearly stated.
 Where I have consulted the published work of others, this is always clearly at-
tributed.
 Where I have quoted from the work of others, the source is always given. With
the exception of such quotations, this thesis is entirely my own work.
 I have acknowledged all main sources of help.
 Where the thesis is based on work done by myself jointly with others, I have
made exactly what was done by others and what I have contributed myself.
Hanoi, December 2017
PhD STUDENT
Thi Huong Giang DOAN
SUPERVISORS
Dr. Hai VU Dr. Thi Thanh Hai TRAN
i
ACKNOWLEDGEMENT
This thesis was written during my doctoral study at International Research In-
stitute Multimedia, Information, Communication and Applications (MICA), Hanoi
University of Science and Technology (HUST). It is my great pleasure to thank all the
people who supported me for completing this work.
First, I would like to express my sincere gratitude to my advisors Dr. Hai Vu and
Dr. Thi Thanh Hai Tran for the continuous support of my Ph.D. study and related re-
search, for their patience, motivation, and immense knowledge. Their guidance helped
me in all the time of research and writing of this thesis. I could not have imagined
having a better advisor and mentor for my Ph.D. study.
Besides my advisors, I would like to thank the scientists and the authors of the
published works which are cited in this thesis, and I am provided with valuable infor-
mation resources from their works for my thesis. The attention at scientific conferences
have always been a great experience for me to receive many the useful comments.
In the process of implementation and completion of my research, I have received
many supports from the board of MICA directors. My sincere thanks go to Prof. Yen
Ngoc Pham, Prof. Eric Castelli and Dr. Son Viet Nguyen, who provided me with an
opportunity to join researching works in MICA institute, and who gave access to the
laboratory and research facilities. Without their precious support would it have been
being impossible to conduct this research.
As a Ph.D. student of 911 programme, I would like to thanks 911 programme for
their financial support during my Ph.D course. I also gratefully acknowledge the finan-
cial support for publishing papers and conference fees from research projects T2014-100,
T2016-PC-189, and T2016-LN-27. I would like to thank my colleagues at Computer
Vision Department and Multi-Lab of MICA institute over the years both at work and
outside of work.
Special thanks to my family. Words can not express how grateful I am to my
mother and father for all of the sacrifices that they have made on my behalf. I would
also like to thank my beloved husband. Thank you for supporting me for everything.
Hanoi, December 2017
Ph.D. Student
Thi Huong Giang DOAN
ii
CONTENTS
DECLARATION OF AUTHORSHIP i
ACKNOWLEDGEMENT ii
CONTENTS vi
SYMBOLS vii
LIST OF TABLES xi
LIST OF FIGURES xvi
1 LITERATURE REVIEW 8
1.1 Completed hand gesture recognition systems for controlling home appli-
ances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.1.1 GUI device dependent systems . . . . . . . . . . . . . . . . . . . 8
1.1.2 GUI device independent systems . . . . . . . . . . . . . . . . . 14
1.2 Hand detection and segmentation . . . . . . . . . . . . . . . . . . . . . 18
1.2.1 Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.2.2 Shape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.2.3 Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.2.4 Depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.2.5 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.3 Hand gesture spotting system . . . . . . . . . . . . . . . . . . . . . . . 24
1.3.1 Model-based approaches . . . . . . . . . . . . . . . . . . . . . . 25
1.3.2 Feature-based approaches . . . . . . . . . . . . . . . . . . . . . 27
1.3.3 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.4 Dynamic hand gesture recognition . . . . . . . . . . . . . . . . . . . . . 29
1.4.1 HMM-based approach . . . . . . . . . . . . . . . . . . . . . . . 30
1.4.2 DTW-based approach . . . . . . . . . . . . . . . . . . . . . . . 31
1.4.3 SVM-based approach . . . . . . . . . . . . . . . . . . . . . . . . 33
1.4.4 Deep learning-based approach . . . . . . . . . . . . . . . . . . . 34
1.4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
1.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 35
2 A NEW DYNAMIC HAND GESTURE SET OF CYCLIC MOVE-
MENT 37
iii
2.1 Defining dynamic hand gestures . . . . . . . . . . . . . . . . . . . . . . 37
2.2 The existing dynamic hand gesture datasets . . . . . . . . . . . . . . . 38
2.2.1 The published dynamic hand gesture datasets . . . . . . . . . . 38
2.2.1.1 The RGB hand gesture datasets . . . . . . . . . . . . . 38
2.2.1.2 The Depth hand gesture datasets . . . . . . . . . . . . 40
2.2.1.3 The RGB and Depth hand gesture datasets . . . . . . 41
2.2.2 The non-published hand gesture datasets . . . . . . . . . . . . . 44
2.2.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3 Definition of the closed-form pattern of gestures and phasing issues . . 47
2.3.1 A conducting commands of a dynamic hand gestures set . . . . 47
2.3.2 Definition of the closed-form pattern of gestures and phasing issues 48
2.3.3 Characteristics of dynamic hand gesture set . . . . . . . . . . . 50
2.4 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.4.1 MICA1 dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.4.2 MICA2 dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.4.3 MICA3 dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.4.4 MICA4 dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 55
3 HAND DETECTION AND GESTURE SPOTTING WITH USER-
GUIDE SCHEME 56
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.2 Heuristic user-guide scheme . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2.2 Proposed framework . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2.3 Estimating heuristic parameters . . . . . . . . . . . . . . . . . . 60
3.2.3.1 Estimating parameters of background model for body
detection . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2.3.2 Estimating the distance from hand to the Kinect sensor
for extracting hand candidates . . . . . . . . . . . . . 62
3.2.3.3 Estimating skin color parameters for pruning hand regions 63
3.2.4 Hand detection phase using heuristic parameters . . . . . . . . . 65
3.2.4.1 Hand detection . . . . . . . . . . . . . . . . . . . . . . 65
3.2.4.2 Hand posture recognition . . . . . . . . . . . . . . . . 66
3.3 Dynamic hand gesture spotting . . . . . . . . . . . . . . . . . . . . . . 66
3.3.1 Catching buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.3.2 Spotting dynamic hand gesture . . . . . . . . . . . . . . . . . . 67
3.4 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.4.1 The required learning time for end-users . . . . . . . . . . . . . 71
iv
3.4.2 The computational time for hand segmentation and recognition 73
3.4.3 Performance of the hand region segmentations . . . . . . . . . . 75
3.4.3.1 Evaluate the hand segmentation . . . . . . . . . . . . 75
3.4.3.2 Compare the hand posture recognition results . . . . . 75
3.4.4 Performance of the gesture spotting algorithm . . . . . . . . . . 76
3.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.5.1 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.5.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4 DYNAMIC HAND GESTURE REPRESENTATION AND RECOG-
NITION USING SPATIAL-TEMPORAL FEATURES 79
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2 Proposed framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.2.1 Hand representation from spatial and temporal features . . . . . 81
4.2.1.1 Temporal features extraction . . . . . . . . . . . . . . 81
4.2.1.2 Spatial features extraction using linear reduction space 83
4.2.1.3 Spatial features extraction using non-linear reduction
space . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.2.2 DTW-based phase synchronization and KNN-based classification 86
4.2.2.1 Dynamic Time Warping for phase synchronization . . 86
4.2.2.2 Dynamic hand gesture recognition using K-NN method 88
4.2.3 Interpolation-based synchronization and SVM Classification . . 89
4.2.3.1 Dynamic hand gesture representation . . . . . . . . . . 89
4.2.3.2 Quasi-periodic dynamic hand gesture pattern . . . . . 91
4.2.3.3 Phase synchronization using hand posture interpolation 94
4.2.3.4 Dynamic hand gesture recognition using difference clas-
sifications . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.3 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.3.1 Influence of temporal resolution on recognition accuracy . . . . 97
4.3.2 Tunning kernel scale parameters RBF-SVM classifier . . . . . . 98
4.3.3 Performance evaluation of the proposed method . . . . . . . . . 99
4.3.4 Impacts of the phase normalization . . . . . . . . . . . . . . . . 100
4.3.5 Further evaluations on public datasets . . . . . . . . . . . . . . 101
4.4 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.4.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.4.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5 CONTROLLING HOME APPLIANCES USING DYNAMIC HAND
GESTURES 105
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
v
5.2 Deployment of control systems using hand gestures . . . . . . . . . . . 105
5.2.1 Assignment of hand gestures to commands . . . . . . . . . . . . 105
5.2.2 Different modes of operations carried out by hand gestures . . . 107
5.2.2.1 Different states of lamp and their transitions . . . . . . 107
5.2.2.2 Different states of fan and their transition . . . . . . . 108
5.2.3 Implementation of the control system . . . . . . . . . . . . . . . 108
5.2.3.1 Main components of the control system using hand ges-
tures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.2.3.2 Integration of hand gesture recognition modules . . . . 109
5.3 Experiments of control systems using hand gestures . . . . . . . . . . . 115
5.3.1 Environment and material setup . . . . . . . . . . . . . . . . . . 115
5.3.2 Pre-built script . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.3.3 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . 117
5.3.3.1 Evaluation of hand gesture recognition . . . . . . . . . 118
5.3.3.2 Evaluation of time costs . . . . . . . . . . . . . . . . . 119
5.3.4 Evaluation of usability . . . . . . . . . . . . . . . . . . . . . . . 120
5.4 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.4.1 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.4.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Bibliography 126
vi
ABBREVIATIONS
TT Abbreviation Meaning
1 ANN Artifical Neural Network
2 ASL American Sign Language
3 BB Bounding Box
4 BGS Background Subtraction
5 BW Baum Welch
6 BOW Bag Of Words
7 C3D Convolutional 3D
8 CD Compact Disc
9 CIF Common Intermediate Format
10 CNN Convolution Neural Network
11 CPU Central Processing Unit
12 CRFs Conditional Random Fields
13 CSI Channel State Information
14 DBN Deep Belief Network
15 DDNN Deep Dynamic Neural Networks
16 DoF Degree of Freedom
17 DT Decision Tree
18 DTM Dense Trajectories Motion
19 DTW Dynamic Time Warping
20 FAR False Acceptance Rate
21 FD Fourier Descriptor
22 FP False Positive
23 FN False Negative
24 FSM Finite State Machine
25 fps frame per second
26 GA Genetic Algorithm
27 GMM Gaussian Mixture Model
28 GT Ground True
29 GUI Graphic User Interface
30 HCI Human Computer Interaction
vii
31 HCRFs Hidden Conditional Random Fields
32 HNN Hopfield Neural Network
33 HMM Hidden Markov Model
34 HOG Histogram of Oriented Gradient
35 HSV Hue Saturation Value
36 ID IDentification
37 IP Internet Protocol
38 IR InfRared
39 ISOMAP ISOmetric MAPing
40 JI Jaccard Index
41 KLT Kanade Lucas Tomasi
42 KNN K Nearest Neighbors
43 LAN Local Area Network
44 LE Laplacian Eigenmaps
45 LLE Locally Linear Embedding
46 LRB Left Right Banded
47 MOG Mixture of Gaussian
48 MFC Microsoft Founding Classes
49 MSC Mean Shift Clustering
50 MR Magic Ring
51 NB Naive Bayesian
52 PC Persional Computer
53 PCA Principal Component Analysis
54 PDF Probability Distribution Function
55 PNG Portable Network Graphics
56 QCIF Quarter Common Intermediate Format
57 RAM Random Acess Memory
58 RANSAC RANdom SAmple Consensus
59 RBF Radial Basic Function
60 RF Random Forest
61 RGB Red Green Blue
62 RGB-D Red Green Blue Depth
63 RMSE Root Mean Square Error
64 ROI Region of Interest
65 RNN Recurrent Neural Network
viii
66 SIFT Scale Ivariant Feature Transform
67 SVM Support Vector Machine
68 STE Short Time Energy
69 STF Spatial Temporal Feature
70 ToF Time of Flight
71 TN True Negative
72 TP True Positive
73 TV TeleVion
74 XML Xextensible Markup Languag ... teraction based on visual recognition using volumegrams of local binary patterns .
In Proceedings of The IEEE International Conference on Consumer Electronics
(ICCE), pp. 583–584.
[83] Marcel S., Bernier O., Viallet J.E., and Collobert D. (2000). Hand gesture recog-
nition using input-output hidden Markov models . In Proceedings of The 4th IEEE
International Conference on Automatic Face and Gesture Recognition (FG), pp.
456–461.
[84] Martin J., Devin V., and Crowley J.L. (1998). Active hand tracking . In Proceed-
ings of The 3rd IEEE International Conference on Automatic Face and Gesture
Recognition (FG), pp. 573–578.
[85] Matnani P. (2015). Glove Based and Accelerometer based Gesture Control: A
Literature. International Journal of Research and Engineering (IJRE), 3(6):pp.
216–221.
[86] McGuinness K. and O Connor N.E. (2010). A comparative evaluation of interac-
tive segmentation algorithms . Pattern Recognition Journal , 43(2):pp. 434–444.
[87] Molchanov P., Yang X., Gupta S., Kim K., Tyree S., and Kautz J. (2016). Online
detection and classification of dynamic hand gestures with recurrent 3d convolu-
tional neural networks . In Proceedings of The IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), pp. 4207–4215.
[88] Molina J., Alessandro M.E.v., Pardas M., Ferran C., Bescos J., Marques F., and
Martinez J.M. (2013). Real-time user independent hand gesture recognition from
time-of-flight camera video using static and dynamic models . Machine Vision
and Applications , 24:pp. 187–204.
[89] Morguet P. and Lang M. (1998). Spotting dynamic hand gestures in video im-
age sequences using hidden markov models . In Proceedings of The International
Conference on Image Processing (ICIP), volume 3, pp. 193–197.
133
[90] Mun˜oz Salinas R., Medina-Carnicer R., Madrid-Cuevas F.J., and Carmona-
Poyato A. (2008). Depth silhouettes for gesture recognition. Pattern Recognition
Letters Journal , 29(3):pp. 319–329.
[91] Nguyen V.T., Le T.L., Tran T.T.H., Remy M., and Vincent C. (2015). A New
Hand Representation Based on Kernels for Hand Posture Recognition. In Pro-
ceedings of The 11th IEEE International Conference on Automatic Face and
Gesture Recognition (FG), volume 1, pp. 1–6.
[92] Nichols J., Myers B.A., Higgins M., Hughes J., Harris T.K., Rosenfeld R., and
Pignol M. (2002). Generating remote control interfaces for complex appliances .
In Proceedings of The 15th annual ACM symposium on User interface software
and technology (UIST), pp. 161–170.
[93] Ohn-bar E., Member S., and Trivedi M.M. (2014). Hand Gesture Recognition in
Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and
Evaluations . IEEE Transactions on Intelligent Transportation System, 15(6):pp.
2368–2377.
[94] Ong S.C.W., Ranganath S., and Venkatesh Y.V. (2006). Understanding gestures
with systematic variations in movement dynamics . Pattern Recognition Journal ,
39(9):pp. 1633–1648.
[95] Oreifej O. and Liu Z. (2013). HON4D: Histogram of oriented 4D normals for
activity recognition from depth sequences . In Proceedings of The IEEE Computer
Society Conference on Computer Vision and Pattern Recognition(CVPR), pp.
716–723.
[96] Oz C. and Leu C. (2011). American sign language word recognition with a sen-
sory glove using artificial neural networks . Engineering Applications of Artificial
Intelligence Journal (EAAI), 24(7):pp. 1204–1213.
[97] Park M., Hasan M., Kim J., and Chae O. (2012). Hand Detection and Tracking
Using Depth and Color Information.
[98] Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O.,
Blondel M., Prettenhofer P., Weiss R., Dubourg V., Vanderplas J., Passos A.,
Cournapeau D., Brucher M., Perrot M., and Duchesnay E. (2011). Scikit-learn:
Machine learning in python. Journal of Machine Learning Research, 12:pp. 2825–
2830.
[99] Pisharady P.K., Vadakkepat P., and Loh A.P. (2012). Attention Based Detection
and Recognition of Hand Postures Against Complex Backgrounds . International
Journal of Computer Vision (IJCV), 101:pp. 403–419.
134
[100] Plouffe G. and Cretu A.M. (2016). Static and dynamic hand gesture recognition in
depth data using dynamic time warping . IEEE Transactions on Instrumentation
and Measurement , 65(2):pp. 305–316.
[101] Poularakis S. and Katsavounidis I. (2014). Finger detection and hand posture
recognition based on depth information. In Proceedings of The IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4329–
4333.
[102] Prajapat S. and Upadhyay H. (2017). 
of-home-appliances-using-raspberry-pi-part-2/ .
[103] Premaratne P. and Nguyen Q. (2007). Consumer electronics control system based
on hand gesture moment invariants . IET Computer Vision, 1(1):pp. 35–41.
[104] Quinlan J.R. (1986). Induction of decision trees . Machine Learning Journal ,
1(1):pp. 81–106.
[105] Ramamoorthy A., Vaswani N., Chaudhury S., and Banerjee S. (2003). Recogni-
tion of dynamic hand gestures . Journal of Pattern Recognition, 36(9):pp. 2069 –
2081.
[106] Ramamoorthy R. and Keerthana K. (2015). Hand Gesture Based Television Con-
trol, journal= Proceedings of The National Conference on Research Advances
in Communication, Computation, Electrical Science and Structures (NCRAC-
CESS). (8):pp. 41–43.
[107] Rashid O., Al-Hamadi A., and Michaelis B. (2010). Integration of gesture and pos-
ture recognition systems for interpreting dynamic meanings using particle filter .
In Proceedings of The International Conference of Soft Computing and Pattern
Recognition, pp. 47–50.
[108] Rautaray, S. S., and Agrawal A. (2015). Vision based hand gesture recognition for
human computer interaction: A survey . Artificial Intelligence Review Journal ,
43(1):pp. 1–54.
[109] Rautaray S. and Agrawal A. (2015). Vision based hand gesture recognition for
human computer interaction: a survey . Artificial Intelligence Review Journal ,
43(1):pp. 1–54.
[110] Ravikiran J., Mahesh K., Mahishi S., Dheeraj R., Sudheender S., and Pujari
N.V. (2009). Finger Detection for Sign Language Recognition. Proceedings of The
International MultiConference of Engineers and Computer Scientists (IMECS),
I:pp. 489–493.
135
[111] Ren Z., Yuan J., and Zhang Z. (2011). Robust Hand Gesture Recognition Based on
Finger-Earth Movers Distance with a Commodity Depth Camera. In Proceedings
of The 19th ACM international conference on Multimedia (MM), pp. 1093–1096.
[112] R.Mccartney J. Yuan H.B. (2015). Gesture Recognition with the Leap Motion
Controller Gesture Recognition with the Leap Motion Controller . In Interna-
tional Conference on Image Processing, Computer Vision and Pattern Recogni-
tion (IPCV), pp. 2–9.
[113] Roweis S.T. and Saul L.K. (2000). Nonlinear Dimensionality Reduction by Locally
Linear Embedding . Science, 290:pp. 2323 – 2326.
[114] S. Escalera X. Baro J.G.M.A.B.M.M.M.R.V.P.L.H.J.E.J.S. and Guyon I. (2014).
ChaLearn Looking at People Challenge 2014: Dataset and results . In The Euro-
pean Workshops on Computer Vision (WECCV), pp. 1–8.
[115] S. Mohamed Mansoor Roomi R.J.P. and Jayalakshmi H. (2010). Hand Gesture
Recognition for Human-Computer Interaction. Journal of Computer Science,
6(9):pp. 1002–1007.
[116] Sangineto E. and Cupelli M. (2012). Real-time viewpoint-invariant hand local-
ization with cluttered backgrounds . Image Vision Computting Journal , 30(1):pp.
26–37.
[117] Sharma P.K., Sheoran R., and Kathania D.K. (2015). A Study of Hand Gesture
Technique to Communicate with a Robot . International Journal of Engineering
Research and Technology (IJERT), 4:pp. 12–13.
[118] Shen X., Hua G., Williams L., and Wu Y. (2012). Dynamic hand gesture recog-
nition: An exemplar-based approach from motion divergence fields . Image and
Vision Computing Journal , 30(3):pp. 227–235.
[119] Shimada A., Yamashita T., and i. Taniguchi R. (2013). Hand gesture based tv
control system. In Proceedings of The 19th Korea-Japan Joint Workshop on
Frontiers of Computer Vision, pp. 121–126.
[120] Shin S. and Sung W. (2016). Dynamic hand gesture recognition for wearable
devices with low complexity recurrent neural networks . The Computing Research
Repository (CoRR), abs/1608.04080:pp. 3–6.
[121] Shitole S.M., Patil S.B., and Narote S.P. (2013). Article: Dynamic hand ges-
ture recognition using pca, pruning and ann. International Journal of Computer
Applications (IJCA), 74(2):pp. 24–29.
136
[122] Sigalas M., Baltzakis H., and Trahanias P. (2010). Gesture recognition based
on arm tracking for human-robot interaction. In Proceedings of The IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS), pp. 5424–
5429.
[123] Smedt Q.D., Wannous H., and Vandeborre J.P. (2016). 
lille.fr/dhgdataset/..
[124] Smedt Q.D., Wannous H., and Vandeborre J.P. (2016). Skeleton-based dynamic
hand gesture recognition. In Proceedings of The IEEE Conference on Computer
Vision and Pattern Recognition Workshops (CVPRW), pp. 1206–1214.
[125] Sohn M.K., Lee S.H., Kim D.J., Kim B., and Kim H. (2012). A comparison of
3d hand gesture recognition using dynamic time warping . In Proceedings of The
27th Conference on Image and Vision Computing New Zealand (IVCNZ), pp.
418–422.
[126] Solanki U.V. and Desai N.H. (2011). Hand gesture based remote control for
home appliances: Handmote. In Proceedings of The IEEE World Congress on
Information and Communication Technologies (WICT), pp. 419–423.
[127] Song L. and Takatsuka M. (2005). Real-time 3d finger pointing for an aug-
mented desk . In Proceedings of The 6th Australasian Conference on User Inter-
face (AUIC), volume 40, pp. 99–108.
[128] Stauffer C. and Grimson W. (1999). Adaptive background mixture models for
real-time tracking . In Proceedings of the IEEE Computer Society Conference on
Computer Vision and Pattern Recognition (CVPR), volume 2, pp. 246 – 252.
[129] Suk H.I., Sin B.K., and Lee S.W. (2010). Hand gesture recognition based on
dynamic bayesian network framework . Pattern Recognition Journal , 43(9):pp.
3059–3072.
[130] Tenenbaum J.B., de Silva V., and Langford J.C. (2000). A global geometric frame-
work for nonlinear dimensionality reduction. Science Journal , 290(5500):pp.
2319–2323.
[131] Tran D., Bourdev L., Fergus R., Torresani L., and Paluri M. (2015). Learning
Spatiotemporal Features With 3D Convolutional Networks . In Proceedings of The
IEEE International Conference on Computer Vision (ICCV), pp. 4489–4497.
[132] Tran T.T.H. and Nguyen T.T.M. (2010). Invariant lighting hand posture classi-
fication. In Proceedings of The 2010 IEEE International Conference on Progress
in Informatics and Computing (PIC), volume 2, pp. 827–831.
137
[133] Verma R. and Dev A. (2009). Vision based hand gesture recognition using finite
state machines and fuzzy logic. In Proceedings of The International Conference
on Ultra Modern Telecommunications Workshops , pp. 1–6.
[134] Viola P. and Jones M.J. (2004). Robust real-time face detection. International
Journal of Computer Vision (IJCV), 57(2):pp. 137–154.
[135] W. Gao G. Fang D.Z. and Chen Y. (2004). Transition movement models for
large vocabulary continuous sign language recognition. In Proceedings of The 6th
IEEE International Conference Automatic Face and Gesture Recognition (FG),
pp. 553 – 558.
[136] Wang T.S., Shum H.Y., Xu Y.Q., and Zheng N.N. (2001). Unsupervised analysis
of human gestures . In Proceedings of The 2nd IEEE Pacific Rim Conference
on Multimedia Beijing Advances in Multimedia Information Processing (PCM),
volume 2195, pp. 174–181.
[137] WangGW Zhang C Z.J. (2012). An application of classifier combinationmethods
in hand gesture recognition. Mathematical Problems in Engineering Journal ,
2012:pp. 1–17.
[138] Wu D., Pigou L., Kindermans P.j., Le N., Shao L., Dambre J., and Odobez
J.m. (2016). Deep Dynamic Neural Networks for Multimodal Gesture Segmen-
tation and Recognition. IEEE Transactions on Pattern Analysis and Machine
Intelligence (TPAMI), 38(8):pp. 1–16.
[139] Xu D. (2006). A neural network approach for hand gesture recognition in virtual
reality driving training system of spg . In Proceedings of The 18th International
Conference on Pattern Recognition (ICPR), volume 3, pp. 519–522.
[140] Y. S., D. D., and R. D. (2011). Tracking Body and Hands For Gesture Recog-
nition: NATOPS Aircraft Handling Signals Database. In Proceedings of The
9th IEEE International Conference on Automatic Face and Gesture Recognition
(FG), pp. 500–506.
[141] Yang H.D. (2015). Sign Language Recognition with the Kinect Sensor Based on
Conditional Random Fields . Sensors Journal , 15(1):pp. 135–147.
[142] Yang H.D., Sclaroff S., and Lee S.W. (2009). Sign Language Spotting with a
Threshold Model Based on Conditional Random Fields . IEEE Transactions on
Pattern Analysis and Machine Intelligence (TPAMI), 31(7):pp. 1264–1277.
[143] Yang S.E., Do J.H., Jang H., Jung J.W., and Bien Z. (2006). Advanced Soft Re-
mote Control System in Human-friendliness . International Journal of Soft Com-
138
puting and Intelligent Systems, and Symposium on Advanced Intelligent Systems
(SCIS and ISIS), 2006:pp. 218–222.
[144] Yang X. and Tian Y. (2014). Super Normal Vector for Action Recognition Using
Depth Sequences . In Proceedings of The IEEE Computer Society Conference on
Computer Vision and Pattern Recognition (CVPR), pp. 804–811.
[145] Yang Z., Li Y., Chen W., and Zheng Y. (2012). Dynamic hand gesture recognition
using hidden markov models . In Proceedings of The 7th International Conference
on Computer Science Education (ICCSE), pp. 360–365.
[146] Yuan R., Cheng J., Li P., Chen G., Xie C., and Xie Q. (2010). View invariant
hand gesture recognition using 3d trajectory . In Proceedings of The 8th World
Congress on Intelligent Control and Automation, pp. 6315–6320.
[147] Zabulis X., Baltzakis H., and Argyros A. (2009). Vision-based Hand Gesture
Recognition for Human Computer Interaction. Lawrence Erlbaum Associates,
Inc. (LEA).
[148] Zaman M.F., Mossarrat S.T., Islam F., and Karmaker D. (2015). Real-time hand
detection and tracking with depth values . In Proceedings of The International
Conference on Advances in Electrical Engineering (ICAEE), pp. 129–132.
[149] Zhang X., Chen X., Wang W.h., Yang J.h., Lantz V., and Wang K.q. (2009).
Hand gesture recognition and virtual game control based on 3d accelerometer and
emg sensors . In Proceedings of The 14th International Conference on Intelligent
User Interfaces (IUI), pp. 401–406.
[150] Zhang Y., Yao Y., and Luo Y. (2015). An improved hmm/svm dynamic hand
gesture recognition algorithm. In Proceedings of The SPIE , volume 9672, pp.
96720D–96720D–7.
[151] Zou Z., Premaratne P., Monaragala R., Bandara N., and Premaratne M. (2010).
Dynamic hand gesture recognition system using moment invariants . In Proceed-
ings of The 5th International Conference on Information and Automation for
Sustainability , pp. 108–113.
139