Open Access Open Access  Restricted Access Subscription or Fee Access

Real Time Vision Based Method for Finger Counting Through Shape Analysis with Convex Hull and PCA Techniques


(*) Corresponding author


Authors' affiliations


DOI: https://doi.org/10.15866/irecos.v12i3.12278

Abstract


This paper presents a real time vision-based finger counting method combining convex-hull detection and PCA techniques. The method starts by segmenting an input image to detect the area corresponding to the observed hand. For that purpose, a skin color detection method is used to differentiate the foreground containing the hand from the image background. Lighting variation can affect the accuracy of the segmentation which has an impact on the functioning of the proposed method. In order to deal with the lighting variation problem, HLS color space is used to represent the colors. Hand contour is then calculated and fingertips are detected through the detection of the convex-hull and convexity defects of the hand shape. The use of convex-hull algorithm is simple and gives accurate results when more than one finger is observed in the input image but the accuracy decreases when it comes to deal with the case where only one finger is observed. To overcome this problem, principal component analysis technique (PCA) is used to analyze the hand shape and detect the case where only one finger is observed with better accuracy. The proposed method could be utilized in Human Computer Interaction System (HCI) where the machine reacts to each detected number. Both real and synthetic images are used to test and demonstrate the potential of our method.
Copyright © 2017 Praise Worthy Prize - All rights reserved.

Keywords


PCA; Hand Tracking; Convex-Hull

Full Text:

PDF


References


Sulfayanti, Dewiani, and A. Lawi, “A real time alphabets sign language recognition system using hands tracking,” in 2016 International Conference on Computational Intelligence and Cybernetics, 2016, pp. 69–72.
http://dx.doi.org/10.1109/cyberneticscom.2016.7892569

H. Kim, K. H. Suh, and E. C. Lee, “Multi-modal user interface combining eye tracking and hand gesture recognition,” J. Multimodal User Interfaces, pp. 1–10, 2017.
http://dx.doi.org/10.1007/s12193-017-0242-2

L. Liu, X. Li, Y. Zhao, and J. Chen, “A real-time and low-cost hand tracking system,” in 2017 IEEE International Conference on Consumer Electronics (ICCE), 2017, pp. 430–431.
http://dx.doi.org/10.1109/icce.2017.7889384

U. Soni, A. Trivedi, and N. Roberts, “Real-time hand tracking using integrated optical flow and CAMshift algorithm,” in 2016 Second International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), 2016, pp. 135–140.
http://dx.doi.org/10.1109/icrcicn.2016.7813645

E. Acay, N. Kahraman, M. Taskiran, T. Kiyan, and H. U. Yogun, “Hardware application of human-machine interface in smart air conditioners using hand tracking,” in 2016 International Symposium ELMAR, 2016, pp. 269–272.
http://dx.doi.org/10.1109/elmar.2016.7731803

G. Wu and W. Kang, “Vision-Based Fingertip Tracking Utilizing Curvature Points Clustering and Hash Model Representation,” IEEE Trans. Multimed., vol. PP, no. 99, p. 1, 2017.
http://dx.doi.org/10.1109/tmm.2017.2691538

S. Kalra, S. Jain, and A. Agarwal, “A wearable computing system for wireless communication and gesture based human computer interface,” in 2016 2nd International Conference on Next Generation Computing Technologies (NGCT), 2016, pp. 456–462.
http://dx.doi.org/10.1109/ngct.2016.7877459

M. A. Conn and S. Sharma, “Immersive Telerobotics Using the Oculus Rift and the 5DT Ultra Data Glove,” in 2016 International Conference on Collaboration Technologies and Systems (CTS), 2016, pp. 387–391.
http://dx.doi.org/10.1109/cts.2016.0075

B. Fang, F. Sun, H. Liu, and D. Guo, “A novel data glove for fingers motion capture using inertial and magnetic measurement units,” in 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2016, pp. 2099–2104.
http://dx.doi.org/10.1109/robio.2016.7866639

R. Graham and F. Francesyao, “Finding the convex hull of a simple polygon,” J. Algorithms, vol. 4, no. 4, pp. 324–331, Dec. 1983.
http://dx.doi.org/10.1016/0196-6774(83)90013-5

S. Büttner, O. Sand, and C. Röcker, “Exploring Design Opportunities for Intelligent Worker Assistance: {A} New Approach Using Projetion-Based {AR} and a Novel Hand-Tracking Algorithm,” in Ambient Intelligence - 13th European Conference, AmI 2017, Malaga, Spain, April 26-28, 2017, Proceedings, 2017, pp. 33–45.
http://dx.doi.org/10.1007/978-3-319-56997-0_3

X. Wang and S. Fu, “Aviator Hand Tracking Based on Depth Images,” in Geo-Spatial Knowledge and Intelligence: 4th International Conference on Geo-Informatics in Resource Management and Sustainable Ecosystem, GRMSE 2016, Hong Kong, China, November 18-20, 2016, Revised Selected Papers, Part I, H. Yuan, J. Geng, and F. Bian, Eds. Singapore: Springer Singapore, 2017, pp. 387–395.
http://dx.doi.org/10.1007/978-981-10-3966-9_44

A. Uribe-Quevedo, S. Ortiz, D. Rojas, and B. Kapralos, “Hand tracking as a tool to quantify carpal tunnel syndrome preventive exercises,” in 2016 7th International Conference on Information, Intelligence, Systems Applications (IISA), 2016, pp. 1–5.
http://dx.doi.org/10.1109/iisa.2016.7785396

Y. Zheng and P. Zheng, “Hand tracking based on adaptive kernel bandwidth mean shift,” in 2016 IEEE Information Technology, Networking, Electronic and Automation Control Conference, 2016, pp. 548–552.
http://dx.doi.org/10.1109/itnec.2016.7560420

K. Ito, T. Sueishi, Y. Yamakawa, and M. Ishikawa, “Tracking and recognition of a human hand in dynamic motion for Janken (rock-paper-scissors) robot,” in 2016 IEEE International Conference on Automation Science and Engineering (CASE), 2016, pp. 891–896.
http://dx.doi.org/10.1109/coase.2016.7743496

S. Sridhar, F. Mueller, M. Zollhoefer, D. Casas, A. Oulasvirta, and C. Theobalt, “Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input,” in Proceedings of European Conference on Computer Vision ({ECCV}), 2016.
http://dx.doi.org/10.1007/978-3-319-46475-6_19

L. Yang, B. Yang, H. Dong, and A. E. Saddik, “3-D Markerless Tracking of Human Gait by Geometric Trilateration of Multiple Kinects,” IEEE Syst. J., vol. PP, no. 99, pp. 1–11, 2017.
http://dx.doi.org/10.1109/jsyst.2016.2553518

O. Ben Henia and S. Bouakaz, “3D Hand Model Animation with a New Data-Driven Method,” in Proceedings of the 2011 Workshop on Digital Media and Digital Content Management, 2011, pp. 72–76.
http://dx.doi.org/10.1109/dmdcm.2011.65

M. A. Laskar, A. J. Das, A. K. Talukdar, and K. K. Sarma, “Stereo Vision-based Hand Gesture Recognition under 3D Environment,” Procedia Comput. Sci., vol. 58, pp. 194–201, 2015.
http://dx.doi.org/10.1016/j.procs.2015.08.053

P. V. V Kishore, M. V. D. Prasad, D. A. Kumar, and A. S. C. S. Sastry, “Optical Flow Hand Tracking and Active Contour Hand Shape Features for Continuous Sign Language Recognition with Artificial Neural Networks,” in 2016 IEEE 6th International Conference on Advanced Computing (IACC), 2016, pp. 346–351.
http://dx.doi.org/10.1109/iacc.2016.71

K. G. Quach, C. N. Duong, K. Luu, and T. D. Bui, “Depth-based 3D hand pose tracking,” in 2016 23rd International Conference on Pattern Recognition (ICPR), 2016, pp. 2746–2751.
http://dx.doi.org/10.1109/icpr.2016.7900051

Y. Zheng and P. Zheng, “Hand Tracking by Integrating Color and Shape Cues Based on Particle Filters,” in 2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2016, vol. 2, pp. 144–147.
http://dx.doi.org/10.1109/ihmsc.2016.10

J. Singha, V. B. Semwal, and R. H. Laskar, “An accurate hand tracking system for complex background based on modified KLT Tracker,” in 2016 IEEE Region 10 Conference (TENCON), 2016, pp. 3644–3647.
http://dx.doi.org/10.1109/tencon.2016.7848737

L. Xu, L. Zhang, K. Wang, and K. Mei, “An efficient fast hand tracking approach based on segmentation,” in 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), 2016, pp. 19–24.
http://dx.doi.org/10.1109/yac.2016.7804859

R. Y. Wang and J. Popović, “Real-time hand-tracking with a color glove,” in SIGGRAPH ’09: ACM SIGGRAPH 2009 papers, 2009, pp. 1–8.
http://dx.doi.org/10.1145/1576246.1531369

M. de La Gorce, N. Paragios, and D. J. Fleet, “Model-based hand tracking with texture, shading and self-occlusions,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
http://dx.doi.org/10.1109/cvpr.2008.4587752

A. Tkach, M. Pauly, and A. Tagliasacchi, “Sphere-meshes for Real-time Hand Modeling and Tracking,” ACM Trans. Graph., vol. 35, no. 6, p. 222:1--222:11, Nov. 2016.
http://dx.doi.org/10.1145/2980179.2980226

O. Ben Henia, M. Hariti, and S. Bouakaz, “A two-step minimization algorithm for model-based hand tracking,” in 18th International Conference (WSCG), 2010, pp. 189–196.
http://dx.doi.org/10.1109/dmdcm.2011.65

Y. Zheng and P. Zheng, “Hand Tracking Based on Improved Particle Filters with Elliptical Region Covariance Descriptors,” in 2016 3rd International Conference on Information Science and Control Engineering (ICISCE), 2016, pp. 391–396.
http://dx.doi.org/10.1109/icisce.2016.92

M. A. A. Aziz, J. Niu, X. Zhao, and X. Li, “Efficient and Robust Learning for Sustainable and Reacquisition-Enabled Hand Tracking,” IEEE Trans. Cybern., vol. 46, no. 4, pp. 945–958, Apr. 2016.
http://dx.doi.org/10.1109/tcyb.2015.2418275

R. Rosales, V. Athitsos, L. Sigal, and S. Sclaroff, “3D hand pose reconstruction using specialized mappings,” in ICCV, 2001, pp. 378–385.
http://dx.doi.org/10.1109/iccv.2001.937543

J. Raheja, A. Mishra, and A. Chaudhary, “Indian Sign Language Recognition using SVM,” Pattern Recognit. Image Anal., 2016.
http://dx.doi.org/10.1134/s1054661816020164

H.-S. Yeo, B.-G. Lee, and H. Lim, “Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware,” Multimed. Tools Appl., vol. 74, no. 8, pp. 2687–2715, 2015.
http://dx.doi.org/10.1007/s11042-013-1501-1

K. Silanon and N. Suvonvorn, “Fingertips Tracking Based Active Contour for General HCI Application,” in Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng-2013), Singapore: Springer Singapore, 2014, pp. 309–316.
http://dx.doi.org/10.1007/978-981-4585-18-7_35

A. Dhawan and V. Honrao, “Implementation of Hand Detection based Techniques for Human Computer Interaction,” Int. J. Comput. Appl., vol. 72, no. 17, pp. 6–13, Jun. 2013.
http://dx.doi.org/10.1007/3-540-46616-9_6

S. K. Dey and S. Anand, “Algorithm For Multi-Hand Finger Counting: An Easy Approach,” Adv. Vis. Comput. An Int. Journal(AVC), vol. abs/1404.2, 2014.
http://dx.doi.org/10.1007/s11263-009-0242-0

Z. Chen, J.-T. Kim, J. Liang, J. Zhang, and Y.-B. Yuan, “Real-Time Hand Gesture Recognition Using Finger Segmentation,” Sci. World J., 2014.
http://dx.doi.org/10.1155/2014/267872

Y. Zhou, G. Jiang, and Y. Lin, “A Novel Finger and Hand Pose Estimation Technique for Real-time Hand Gesture Recognition,” Pattern Recogn., vol. 49, no. C, pp. 102–114, Jan. 2016.
http://dx.doi.org/10.1016/j.patcog.2015.07.014

D.-H. Lee and S.-G. Lee, “Vision-Based Finger Action Recognition by Angle Detection and Contour Analysis,” ETRI J., 2011.
http://dx.doi.org/10.4218/etrij.11.0110.0313

S. U. Rahman, Z. Afroze, and M. Tareq, “Hand Gesture Recognition Techniques For Human Computer Interaction Using OpenCv,” Int. J. Sci. Res. Publ., 2014.
http://dx.doi.org/10.1007/978-3-319-07230-2_20

D. J. Rios-Soria, S. E. Schaeffer, and S. E. Garza-Villarreal, “Hand-gesture recognition using computer-vision techniques,” in 21st International Conference ({WSCG}), 2013.
http://dx.doi.org/10.1007/978-3-642-31149-9_14

I. T. Jolliffe, Principal Component Analysis. Springer, 2002.
http://dx.doi.org/10.1007/978-1-4757-1904-8_7

J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 8, no. 6, pp. 679–698, Jun. 1986.
http://dx.doi.org/10.1109/tpami.1986.4767851


Refbacks

  • There are currently no refbacks.


Please send any question about this web site to info@praiseworthyprize.com
Copyright © 2005-2024 Praise Worthy Prize