Content-Based Image Retrieval in Medical Domains: an Overview of Methods, Issues and Systems


(*) Corresponding author


Authors' affiliations


DOI's assignment:
the author of the article can submit here a request for assignment of a DOI number to this resource!
Cost of the service: euros 10,00 (for a DOI)

Abstract


Content-based image retrieval (CBIR) is one of the most interesting research areas of computer vision in recent years. CBIR uses feature comparison to search, browse, and retrieve images from large databases. It is considered as the main research topic in medical fields because of the daily increase in the number of recorded medical images. CBIR can help physicians diagnose various types of diseases as well as medical references, teaching purposes , training and research. This paper introduces the main concepts, stages, and methods of CBIR systems; explores the main types of features that can be extracted from images; and identifies the techniques to evaluate the systems. It also reviews and evaluates the advantages and disadvantages of several existing commercial and academic content-based medical image retrieval systems. Finally, the paper introduces suggestions for the future of medical domains to solve the discussed disadvantages. It proposes the technique of automatic medical image annotation/classification to enhance the accuracy of CBIR systems.
Copyright © 2014 Praise Worthy Prize - All rights reserved.

Keywords


Content-Based Medical Image Retrieval; Indexing; Medical Database; Medical Image Retrieval; Text-Based Image Retrieval

Full Text:

PDF


References


[1] Eakins, John P. "Automatic image content retrieval-are we getting anywhere?." In ELVIRA-PROCEEDINGS-, pp. 121-134. 1996.
[2] Bhadoria, Sonali, and C. G. Dethe. "Study of Medical Image Retrieval." In Data Storage and Data Engineering (DSDE), 2010 International Conference on, pp. 192-196. IEEE, 2010.
[3] Rodhetbhai, Wasara. "Preprocessing for content-based image retrieval." Ph.D. thesis. School of Electronics and Computer Science, University of Southampton, 2009.
[4] Kocsis, L. Costaridou, EP Efstathopoulos, D. Lymberopoulos, G. Panayiotakis, O. "A tool for designing digital test objects for module performance evaluation in medical digital imaging." Informatics for Health and Social Care, vol. 24, n. 4, pp. 291-308, 1999.
[5] Gudivada, V.N.; Raghavan, V.V., "Content based image retrieval systems," Computer , vol.28, n.9, pp.18,22, Sep 1995.
[6] Aggarwal, Preeti, H. K. Sardana, and Gagandeep Jindal. "Content based medical image retrieval: Theory, gaps and future directions." ICGST-GVIP, vol. 9, n. 2, pp.27-37, 2009.
[7] A. Blaser, Database Techniques for Pictorial Applications, Lecture Notes in Computer Science, Vol.81, Springer Verlag GmbH, 1979.
[8] Patil, Pushpa B., and Manesh B. Kokare. "Relevance Feedback in Content Based Image Retrieval: A Review." Journal of Applied Computer Science & Mathematics, n.10, 2011.
[9] Cox, Ingemar J.; Miller, M.L.; Minka, T.P.; Papathomas, T.V.; Yianilos, P.N., "The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments," Image Processing, IEEE Transactions on , vol.9, no.1, pp.20-37, Jan 2000.
[10] Tong, Simon, and Edward Chang. "Support vector machine active learning for image retrieval." In Proceedings of the ninth ACM international conference on Multimedia, pp. 107-118. ACM, 2001.
[11] MacArthur, Sean D., Carla E. Brodley, and Chi-Ren Shyu. "Relevance feedback decision trees in content-based image retrieval." In Content-based Access of Image and Video Libraries, 2000. Proceedings. IEEE Workshop on, pp. 68-72. IEEE, 2000.
[12] Smeulders, A. W M; Worring, M.; Santini, S.; Gupta, A.; Jain, R., "Content-based image retrieval at the end of the early years," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.22, no.12, pp.1349-1380, Dec 2000.
[13] Müller, Henning, Nicolas Michoux, David Bandon, and Antoine Geissbuhler. "A review of content-based image retrieval systems in medical applications—clinical benefits and future directions." International journal of medical informatics vol. 73, no. 1, pp. 1-23, 2004.
[14] Geusebroek, J.-M.; Van den Boomgaard, R.; Smeulders, A. W M; Geerts, H., "Color invariance," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.23, no.12, pp.1338-1350, Dec 2001.
[15] Squire, David McG, Wolfgang Müller, Henning Müller, and Thierry Pun. "Content-based query of image databases: inspirations from text retrieval."Pattern Recognition Lettersvol. 21, no. 13, pp. 1193-1198, 2000.
[16] Carson, Chad, Serge Belongie, Hayit Greenspan, and Jitendra Malik. "Region-based image querying." In Content-Based Access of Image and Video Libraries, 1997. Proceedings. IEEE Workshop on, pp. 42-49. IEEE, 1997.
[17] Smith, John R., and Shih-Fu Chang. "VisualSEEk: a fully automated content-based image query system." In Proceedings of the fourth ACM international conference on Multimedia, pp. 87-98. ACM, 1997.
[18] Sclaroff, S.; Taycher, L.; La Cascia, M., "ImageRover: a content-based image browser for the World Wide Web," Content-Based Access of Image and Video Libraries, 1997. Proceedings. IEEE Workshop on , vol. 2, no.9, pp. 20-20. June 1997.
[19] Niblack, Carlton W., Ron Barber, Will Equitz, Myron D. Flickner, Eduardo H. Glasman, Dragutin Petkovic, Peter Yanker, Christos Faloutsos, and Gabriel Taubin. "QBIC project: querying images by content, using color, texture, and shape." In IS&T/SPIE's Symposium on Electronic Imaging: Science and Technology, pp. 173-187. International Society for Optics and Photonics, 1993.
[20] Sural, Shamik, Gang Qian, and Sakti Pramanik. "Segmentation and histogram generation using the HSV color space for image retrieval." In Proceedings of IEEE International Conference on Image Processing, vol. 2, no. II, pp. 589-592, Sep. 2002.
[21] Milanese, Ruggero, and Michel Cherbuliez. "A rotation, translation, and scale-invariant approach to content-based image retrieval." Journal of visual communication and image representation, Vol. 10, no. 2, , pp. 186-196, 1999.
[22] Squire, David McG, Wolfgang Müller, Henning Müller, and Thierry Pun. "Content-based query of image databases: inspirations from text retrieval."Pattern Recognition Letters, vol. 21, no.13, pp. 1193-1198, 2000.
[23] Ortega, Michael, Yong Rui, Kaushik Chakrabarti, Kriengkrai Porkaew, Sharad Mehrotra, and Thomas S. Huang. "Supporting ranked boolean similarity queries in MARS." Knowledge and Data Engineering, IEEE Transactions on vol. 10, no. 6, pp. 905-925, 1998.
[24] Andaló, Fernanda A., Paulo AV Miranda, R. da S. Torres, and Alexandre X. Falcão. "Shape feature extraction and description based on tensor scale."Pattern Recognition, vol. 43, no. 1, pp. 26-36, 2010.
[25] Gnanasekar, P., A. Nagappan, S. Sharavanan, O. Saravanan, D. Vinodkumar, T. Elayabharathi, and G. Karthik. "Investigation on Feature Extraction and Classification of Medical Images." World Academy of Science, Engineering and Technology, vol. 60, pp. 327-332, 2011.
[26] Smeulders, A. W M; Worring, M.; Santini, S.; Gupta, A.; Jain, R., "Content-based image retrieval at the end of the early years," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.22, no.12, pp.1349,1380, Dec 2000.
[27] Winter, Alexandre, and Chahab Nastar. "Differential feature distribution maps for image segmentation and region queries in image databases." In Content-Based Access of Image and Video Libraries, 1999.(CBAIVL'99) Proceedings. IEEE Workshop on, pp. 13-17. IEEE, 1999.
[28] Eakins, John, Margaret Graham, and Tom Franklin. "Content-based image retrieval." In Library and Information Briefings. 1999.
[29] Kharkate, Shalini K., and Nitin J. Janwe. "Automatic Image Annotation: A Review." International Journal of Computer Science & Applications (TIJCSA), vol.1, no. 12, 2013.
[30] Manikandan, S., V. Rajamani, and N. Murugan. "Analysis of Ultra Sound Kidney Image Features for Image Retrieval by Gray Level Co-Occurrence Matrices." Lecture Notes on Software Engineering, vol.1, no. 1, 2013.
[31] Smeulders, Arnold WM, Marcel Worring, Simone Santini, Amarnath Gupta, and Ramesh Jain. "Content-based image retrieval at the end of the early years."Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 22, no. 12, pp. 1349-1380, 2000.
[32] Khodaskar, Anuja, and S. A. Ladke. "Content Based Image Retrieval with Semantic Features using Object Ontology." International Journal of Engineering, vol1, no. 4, 2012.
[33] Luccheseyz, L., and S. K. Mitray. "Color image segmentation: A state-of-the-art survey." Image Processing, Vision, and Pattern Recognition, vol. 67, no. 2, pp. 207-221, 2001.
[34] Carson, Chad, Megan Thomas, Serge Belongie, Joseph M. Hellerstein, and Jitendra Malik. "Blobworld: A system for region-based image indexing and retrieval." In Visual Information and Information Systems, pp. 509-517. Springer Berlin Heidelberg, 1999.
[35] Veltkamp, Remco C., and Michiel Hagedoorn. State of the art in shape matching. Springer London, 2001.
[36] Wang, James Ze, Jia Li, and Gio Wiederhold. "SIMPLIcity: Semantics-sensitive integrated matching for picture libraries." Pattern Analysis and Machine Intelligence, IEEE Transactions on vol. 23, no. 9, pp. 947-963, 2001.
[37] De Vries, Arjen P., Mark GLM van Doorn, Henk M. Blanken, and Peter MG Apers. "The mirror MMDBMS architecture." (1999).
[38] Müller, Henning, Paul Clough, William Hersh, Thomas Deselaers, Thomas Lehmann, and Antoine Geissbuhler. "Evaluation axes for medical image retrieval systems: the imageCLEF experience." In Proceedings of the 13th annual ACM international conference on Multimedia, pp. 1014-1022. ACM, 2005.
[39] Lee, S.-C. Liao, M. Embrechts, I-N. "Data mining techniques applied to medical information." Informatics for Health and Social Care, vol. 25, no. 2, pp. 81-102, 2000.
[40] Doi, Kunio. "Diagnostic imaging over the last 50 years: research and development in medical imaging science and technology." Physics in medicine and biology, vol. 51, no. 13, 2006.
[41] Magoulas, George D., and Andriana Prentza. "Machine learning in medical applications." In Machine Learning and its applications, pp. 300-307. Springer Berlin Heidelberg, 2001.
[42] Hwang, Kyung Hoon, Haejun Lee, and Duckjoo Choi. "Medical image retrieval: Past and present." Healthcare informatics research, vol. 18, no. 1, pp. 3-9, 2012.
[43] Orphanoudakis, Stelios C., Catherine E. Chronaki, and Despina Vamvaka. 2Cnet: Content-based similarity search in geographically distributed repositories of medical images." Computerized Medical Imaging and Graphics vol. 20, no. 4, pp. 193-207, 1996.
[44] El-Kwae, Essam A., Haifeng Xu, and Mansur R. Kabuka. "Content-based retrieval in picture archiving and communication systems." Journal of Digital Imaging, vol. 13, no. 2, pp. 70-81, 2000.
[45] Antani, Sameer, L. Rodney Long, and George R. Thoma. "A Biomedical Information System for Combined Content-Based Retrieval of Spine X-Ray Images, Associated Text Information." In ICVGIP. 2002.
[46] Le Bozec, Christel, Eric Zapletal, Marie-Christine Jaulent, Didier Heudes, and Patrice Degoulet. "Towards content-based image retrieval in a HIS-integrated PACS." In Proceedings of the AMIA Symposium, p. 477. American Medical Informatics Association, 2000.
[47] Hersh, William, Mark Mailhot, Catherine Arnott-Smith, and Henry Lowe. "Selective automated indexing of findings and diagnoses in radiology reports."Journal of biomedical informatics vol. 34, no. 4, pp. 262-273, 2001.
[48] Westerveld, Thijs. "Image Retrieval: Content versus Context." In RIAO, pp. 276-284. 2000.
[49] Tang, Lilian H., Rudolf Hanka, Horace HS Ip, and Ringo Lam. "Extraction of semantic features of histological images for content-based retrieval of images." In Medical Imaging'99, pp. 360-368. International Society for Optics and Photonics, 1999.
[50] Kwak, Dong-Min, Bum-Soo Kim, Ock-Kyung Yoon, Chul-Hyun Park, Jong-Un Won, And Kil-Houm Park. "Content-Based Ultrasound Image Retrieval Using a Coarse to Fine Approach." Annals of the New York Academy of Sciences, vol. 980, no. 1, pp. 212-224, 2002.
[51] Baroni, A. Del Bimbo, A. Evangelist, E. Vicario, M. "Symbolic representation and visual querying of left ventricular image sequences."Informatics for Health and Social Care, vol. 24, no. 1, pp. 33-52, 1999.
[52] Brodley, Carla E., A. Kak, C. Shyu, J. Dy, L. Broderick, and Alex M. Aisen. "Content-based retrieval from medical image databases: A synergy of human interaction, machine learning and computer vision." In AAAI/IAAI, pp. 760-767. 1999.
[53] Müller, H., A. Rosset, J. P. Vallée, and A. Geissbuhler. "Integrating content-based visual access methods into a medical case database." Studies in health technology and informatics vol. 95, no. 480, 2003.
[54] Liu, Chii-Tung, Pol-Lin Tai, Arlene Y-J. Chen, Chen-Hsing Peng, Tain Lee, and Jia-Shung Wang. "A content-based ct lung image retrieval system for assisting differential diagnosis images collection." In 2001 IEEE International Conference on Multimedia and Expo (ICME’01). 2001.
[55] Petrakis, Euripides GM. "Content-based retrieval of medical images."International Journal of Computer Research, vol. 11, no. 2, pp. 171-182, 2002.
[56] Sinha, Usha, and Hooshang Kangarloo. "Principal Component Analysis for Content-based Image Retrieval1." Radiographics, vol. 22, no. 5, pp. 1271-1289, 2002.
[57] Bucci, G., S. Cagnoni, and R. De Dominicis. "Integrating content-based retrieval in a medical image reference database." Computerized Medical Imaging and Graphics, vol. 20, no. 4, pp. 231-241, 1996.
[58] Petrakis, Euripides G. M., and Christos Faloutsos. "Similarity searching in medical image databases." Knowledge and Data Engineering, IEEE Transactions on, vol. 9, no. 3, pp. 435-447, 1997.
[59] Robinson, G. P., H. D. Tagare, J. S. Duncan, and C. C. Jaffe. "Medical image collection indexing: shape-based retrieval using KD-trees." Computerized Medical Imaging and Graphics, vol. 20, no. 4, pp. 209-217, 1996.
[60] Baeg, Soon, and Nasser Kehtarnavaz. "Classification of breast mass abnormalities using denseness and architectural distortion." Electronic Letters on Computer Vision and Image Analysis, vol. 1, no. 1, pp. 1-20, 2002.
[61] Berretti, Stefano, Alberto Del Bimbo, and Pietro Pala. "Content Based Retrieval Of 3d Cellular Structures." In ICME. 2001.
[62] Van Erkel, Arian R., and Peter M. Th Pattynama. "Receiver operating characteristic (ROC) analysis: basic principles and applications in radiology."European Journal of radiology, vol. 27, no. 2, pp. 88-94, 1998.
[63] Schmid-Saugeona, Philippe, Joël Guillodb, and Jean-Philippe Thirana. "Towards a computer-aided diagnosis system for pigmented skin lesions." Computerized Medical Imaging and Graphics, vol. 27, no. 1, pp. 65-78, 2003.
[64] Mojsilovic, Aleksandra, and Jose Gomes. "Semantic based categorization, browsing and retrieval in medical image databases." In Image Processing. 2002. Proceedings. 2002 International Conference on, vol. 3, pp. III-145. IEEE, 2002.
[65] Bueno, Josiane M., Fabio Chino, Agma JM Traina, and Paulo M. Azevedo-Marques. "How to add content-based image retrieval capability in a PACS." InComputer-Based Medical Systems, 2002.(CBMS 2002). Proceedings of the 15th IEEE Symposium on, pp. 321-326. IEEE, 2002.
[66] Liu, Yanxi, and Frank Dellaert. "Classification-driven medical image retrieval." InProc. of the Image Understanding Workshop. 1998.
[67] Tm, Lehmann, B. Fischer, Güld Mo, C. Thies, D. Keysers, T. Deselaers, H. Schubert, and K. Spitzer. "The IRMA reference database and its use for content-based image retrieval in medical applications." (2004).
[68] Horsch, Alexander, and Rudolf Thurmayr. "How to identify and assess tasks and challenges of medical image processing." Studies in health technology and informatics, pp. 281-285, 2003.
[69] Bouslimi, Riadh, Abir Messaoudi, and Jalel Akaichi. "USING A BAG OF WORDS FOR AUTOMATIC MEDICAL IMAGE ANNOTATION WITH A LATENT SEMANTIC." International Journal of Artificial Intelligence & Applications, vol. 4, no. 3, 2013.
[70] Wang, Xiu Ying, and Dagan Feng. "Image Registration for Biomedical Information Integration." Biomedical Diagnostics and Clinical Technologies: Applying High-performance Cluster and Grid Computing, pp. 316, 2011.
[71] Kim, Jinman, Ashnil Kumar, Tom Weidong Cai and David Dagan Feng. "Multi-Modal Content Based Image Retrieval in Healthcare: Current Applications and Future Challenges." In New Technologies for Advancing Healthcare and Clinical Practices, pp. 44-59, 2011.
[72] Yao, Jian, Zhongfei Mark Zhang, Sameer Antani, Rodney Long, and George Thoma. "Automatic medical image annotation and retrieval." Neurocomputing, vol, 71, no. 10, pp. 2012-2022, 2008.
[73] Barnard, Kobus, Pinar Duygulu, David Forsyth, Nando De Freitas, David M. Blei, and Michael I. Jordan. "Matching words and pictures." The Journal of Machine Learning Research, vol. 3, pp. 1107-1135, 2003.
[74] Duda, Richard O., and Peter E. Hart. Pattern classification and scene analysis. Vol. 3. New York: Wiley, 1973.
[75] Jebara, Tony. Machine learning: discriminative and generative. Springer, 2004.
[76] Jurie, Frédéric, and Cordelia Schmid. "Scale-invariant shape features for recognition of object categories." In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, vol. 2, pp. II-90. IEEE, 2004.
[77] Vapnik, Vladimir. The nature of statistical learning theory. springer, 2000.
[78] Liu, Bing, Wee Sun Lee, Philip S. Yu, and Xiaoli Li. "Partially supervised classification of text documents." In ICML, vol. 2, pp. 387-394. 2002.
[79] Rosset, Saharon, Ji Zhu, Hui Zou, and Trevor Hastie. "A method for inferring label sampling mechanisms in semi-supervised learning." In Advances Neural Information Processing Systems, vol. 17, 2005.
[80] Wu, Wen, and Jie Yang. "Smartlabel: An object labeling tool using iterated harmonic energy minimization." In Proceedings of the 14th annual ACM international conference on Multimedia, pp. 891-900. ACM, 2006.
[81] Yao, Jian, and Zhongfei Zhang. "Object detection in aerial imagery based on enhanced semi-supervised learning." In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, vol. 2, pp. 1012-1017, IEEE, 2005.
[82] Dietterich, Thomas G., and Ghulum Bakiri. "Solving multiclass learning problems via error-correcting output codes." arXiv preprint cs/9501101, 1995.
[83] Ghani, Rayid. "Using error-correcting codes for text classification." In ICML, pp. 303-310. 2000.
[84] Villena Román, Julio, José Carlos González Cristóbal, José Miguel Goñi Menoyo, and José Luis Martínez Fernández. "MIRACLE’s naive approach to medical images annotation." 2005.
[85] Setia, Lokesh, Alexandra Teynor, Alaa Halawani, and Hans Burkhardt. "Grayscale medical image annotation using local relational features." Pattern Recognition Letters, vol. 29, no. 15, pp. 2039-2045, 2008.
[86] Tong, Hanghang, Jingrui He, Mingjing Li, Wei-Ying Ma, Hong-Jiang Zhang, and Changshui Zhang. "Manifold-ranking-based keyword propagation for image retrieval." EURASIP Journal on Applied Signal Processing, pp. 190-190, 2006.
[87] Liu, Hui, Cai Ming Zhang, and Hua Han. "Medical image retrieval based on semi-supervised learning." Advanced Materials Research, vol. 108, pp. 201-206, 2010.
[88] Rahman, Md Mahmudur, Bipin C. Desai, and Prabir Bhattacharya. "Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion." Computerized Medical Imaging and Graphics, vol. 32, no. 2, pp. 95-108, 2008.
[89] Xu, Xiaoqian, Dah-Jye Lee, Sameer K. Antani, L. Rodney Long, and James K. Archibald. "Using relevance feedback with short-term memory for content-based spine X-ray image retrieval." Neurocomputing, vol. 72, no. 10, pp. 2259-2269, 2009.
[90] Kahn Jr, Charles E., and Cheng Thao. "GoldMiner: a radiology image search engine." American Journal of Roentgenology, vol. 188, no. 6, pp. 1475-1478, 2007.
[91] Yu, Hong, and Yong-gang Cao. "Automatically extracting information needs from ad hoc clinical questions." In AMIA annual symposium proceedings, vol. 2008, p. 96. American Medical Informatics Association, 2008.
[92] Hearst, Marti A., Anna Divoli, Harendra Guturu, Alex Ksikes, Preslav Nakov, Michael A. Wooldridge, and Jerry Ye. "BioText Search Engine: beyond abstract search." Bioinformatics, vol. 23, no. 16, pp. 2196-2197, 2007.
[93] Yottalook: http://www.yottalook.com/index_img.php.
[94] Xu, Songhua, James McCusker, and Michael Krauthammer. "Yale Image Finder (YIF): a new search engine for retrieving biomedical images." Bioinformatics, vol. 24, no. 17, pp. 1968-1970, 2008.
[95] IRMA: http://ganymed.imib.rwth-aachen.de/irma/
[96] iMedline: http://archive.nlm.nih.gov/iti/ .
[97] Deserno, Thomas M., Sameer Antani, and Rodney Long. "Ontology of gaps in content-based image retrieval." Journal of digital imaging, vol. 22, no. 2, pp. 202-215, 2009.
[98] Ghosh, Payel, Sameer Antani, L. Rodney Long, and George R. Thoma. "Review of medical image retrieval systems and future directions." In Computer-Based Medical Systems (CBMS), 2011 24th International Symposium on, pp. 1-6. IEEE, 2011.
[99] Uwimana, Epaphrodite, and Miguel E. Ruiz. "Integrating an automatic classification method into the medical image retrieval process." In AMIA Annual Symposium Proceedings, vol. 2008, p. 747. American Medical Informatics Association, 2008.
[100] Arun, K. S., and Hema P. Menon. "Content based medical image retrieval by combining rotation invariant contourlet features and Fourier descriptors."International Journal of Recent Trends in Engineering, vol. 2, no. 2, pp. 35-39, 2009.
[101] Xia, Shunren, Dingfei Ge, Weirong Mo, and Zanchao Zhang. "A content-based retrieval system for endoscopic images." In Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the, pp. 1720-1723. IEEE, 2006.
[102] Xavier, L., B. M. I. Thusnavis, and D. R. W. Newton. "Content based image retrieval using textural features based on pyramid-structure wavelet transform." In Electronics Computer Technology (ICECT), 2011 3rd International Conference on, vol. 4, pp. 79-83. IEEE, 2011.


Refbacks

  • There are currently no refbacks.


Please send any question about this web site to info@praiseworthyprize.com
Copyright © 2005-2024 Praise Worthy Prize