Open Access Open Access  Restricted Access Subscription or Fee Access

Use of Fuzzy Clustering Algorithms Ensemble for SVM Classifier Development


(*) Corresponding author


Authors' affiliations


DOI: https://doi.org/10.15866/iremos.v8i4.6825

Abstract


This paper represents an approach for object's classification based on the combined implementation of the SVM algorithm and the fuzzy clustering algorithms. It is offered to use the fuzzy clustering algorithms’ ensemble based on the clusters’ tags’ vectors’ similarity matrixes and the spectral factorization algorithm as for creation the training and test sets, used for development of SVM classifier, as for specification of the clustering results with application of the trained SVM classifier. In the first case we solve an absence problem of aprioristic information on objects’ possible class association and avoid inclusion of noise objects in the training and test sets, that will provide development of more exact SVM classifier. In the second case we use part of the clustering results received by means of the fuzzy clustering algorithms’ ensemble for training of SVM classifier, and another – for the specification of the clustering results by means of the trained SVM classifier. We use in the spectral factorization algorithm along with the k-means algorithm, which is usually applied to graph splitting into clusters, the FCM algorithm, allowing to reduce the cost of graph splitting and receive better graph splitting into clusters, that will provide higher quality of objects’ classification.
Copyright © 2015 Praise Worthy Prize - All rights reserved.

Keywords


Classification; FCM Algorithm; PCM Algorithm; PFCM Algorithm; SVM Algorithm; Training Set; Testing Set; Ensemble

Full Text:

PDF


References


Vapnik,V. Statistical Learning Theory (1998) Wiley, New York.
http://dx.doi.org/10.1007/978-1-4757-3264-1

Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S., Choosing Multiple Parameters for Support Vector Machine, (2002) Machine Learning, 46 (1–3), pp. 131–159.
http://dx.doi.org/10.1023/a:1012450327387

L. Yu, S. Wang, K.K. Lai and L. Zhou, BioInspired Credit Risk Analysis (Springer, 2008).
http://dx.doi.org/10.1007/978-3-540-77803-5

Raikwal, J.S., Saxena, K., Performance Evaluation of SVM and K-Nearest Neighbor Algorithm over Medical Data set, (2012) International Journal of Computer Applications, 50(14), pp. 35–39. http://research.ijcaonline.org/volume50/number14/pxc3881055.pdf

Joachims, T., Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, (1998) Lecture Notes in Computer Science, 1398, pp. 137–142.
http://dx.doi.org/10.1007/bfb0026683

Li, Y., Bontcheva, K., Cunningham, H., SVM Based Learning System For Information Extraction, (2005) Lecture Notes in Computer Science, 3635, pp. 319–339.
http://dx.doi.org/10.1007/11559887_19

LeCun,Y. Jackel, L.D., Bottou,L., Cortes, C. at al., Learning Algorithms for Classification: A Comparison on Handwritten Digit Recognition, (1995) Neural Networks: The Statistical Mechanics Perspective, Oh, J. H., Kwon, C. and Cho, S. (Ed.), World Scientific, pp. 261–276.

Osuna, E., Freund, R., Girosi, F. Training Support Vector Machines: An Application to Face Detection (1997) 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 130–136
http://dx.doi.org/10.1109/cvpr.1997.609310

Oren, M., Papageorgious, C., Sinha, P., Osuna, E., Poggio, T., Pedestrian Detection Using Wavelet Templates, (1997) 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 193–199.
http://dx.doi.org/10.1109/cvpr.1997.609319

Schölkopf, B., Smola, A. J., Williamson, R. C., Bartlett, P. L. New Support Vector Algorithms (2000) Neural Computation, 12(5), pp. 1207–1245.
http://dx.doi.org/10.1162/089976600300015565

Platt, J.C. Fast Training of Support Vector Machines Using Sequential Minimal Optimization, (1998) Advances in Kernel Methods – Support Vector Learning, pp. 185–208. http://research.microsoft.com:8082/pubs/68391/smo-book.pdf

Shevade, S. K., Keerthi, S. S., Bhattacharyya, C., Murthy, K. R. K., Improvements to the SMO Algorithm for SVM Regression (2000) IEEE Transactions on Neural Networks, 11(5), pp. 1188–1193.
http://dx.doi.org/10.1109/72.870050

Osuna, E., Freund, R., Girosi, F., Improved Training Algorithm for Support Vector Machines (1997) 1997 IEEE Workshop Neural Networks for Signal Processing, pp. 24–26, 1997.
http://dx.doi.org/10.1109/nnsp.1997.622408

Vishwanathan, S.V.N., Smola, A., Murty, N., SSVM: a simple SVM algorithm (2002) Proceedings of the 2002 International Joint Conference on Neural Networks, 3, pp. 2393-2398.
http://dx.doi.org/10.1109/ijcnn.2002.1007516

Shalev-Shwartz, S., Singer, Y., Srebro, N., & Cotter, A., Pegasos: Primal Estimated sub-GrAdient SOlver for SVM, (2011) Mathematical Programming, 127(1), pp. 3–30.
http://dx.doi.org/10.1007/s10107-010-0420-4

Bottou, L., Lin, C.-J., Support Vector Machine Solvers (2007) MIT Press, pp. 1–28.

Eads, D. R., Hill, D., Davis, S., Perkins, S. J., Ma, J., Porter, R. B., Theiler, J. P., Genetic algorithms and support vector machines for time series classification, (2002) Proc. SPIE 4787, Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation, 74
http://dx.doi.org/10.1117/12.453526

Lessmann, S., Stahlbock, R., Crone, S. F., Genetic algorithms for support vector machine model selection, (2006) 2006 IJCNN'06. International Joint Conference on Neural Networks, pp. 3063-3069.
http://dx.doi.org/10.1109/ijcnn.2006.1716515

Bezdek, J. C., Ehrlich, R., Full, W., FCM: The fuzzy c-means clustering algorithm (1984) Computers & Geosciences, 10(2), pp. 191-203.
http://dx.doi.org/10.1016/0098-3004(84)90020-7

Krishnapuram, R., Keller, J.M., The possibilistic c-means algorithm: insights and recommendations (1996) IEEE Transactions on Fuzzy Systems, 4(3), pp. 385-393.
http://dx.doi.org/10.1109/91.531779

Pal, N. R., Pal, K., Bezdek, J. C., A Mixed C-Means Clustering Model (1997) Sixth IEEE International Conference on Fuzzy Systems, 1, pp. 11–21.
http://dx.doi.org/10.1109/fuzzy.1997.616338

Demidova, L.A., Nesterov, N.I., Fuzzy and Possibilistic Segmentation of Earth Surface Images by Means of Intelligent Information Technologies (2014) 2014 International Conference On Computer Technologies In Physical And Engineering Applications, pp. 35–36.
http://dx.doi.org/10.1109/icctpea.2014.6893262

Strehl, A., Ghosh, J., Cluster Ensembles – a Knowledge Reuse Framework for Combining Multiple Partitions, (2003) Journal of Machine Learning Research, 3, pp. 583–617.
http://dx.doi.org/10.1162/153244303321897735

Karypis, G., Kumar, V., A fast and high quality multilevel scheme for partitioning irregular graphs (1998) SIAM Journal of Scientific Computing, 20(1), pp. 359–392, 1998.
http://dx.doi.org/10.1137/s1064827595287997

Astakhova, N.N., Demidova, L.A., Nikulchev, E.V. Forecasting method for grouped time series with the use of k-means algorithm (2015) Applied mathematical sciences, 9(97), pp. 4813–4830.
http://dx.doi.org/10.12988/ams.2015.55391

Karatzoglou, A., Meyer, D., Hornik, K. Support vector machines in R, (2005). Research Report, WU Vienna University of Economics and Business, Vienna. http://epub.wu.ac.at/id/eprint/1500

Hespanha, J.P. An Efficient MATLAB Algorithm for Graph Partitioning (2004) Technical Report, University of California. http://www.ece.ucsb.edu/~hespanha/published/tr-ell-gp.pdf

Goldberg, D.E. Korb, B., Deb K., Messy genetic algorithms: Motivation, analysis, and first results (1989) Complex Systems, 5(3), pp. 493–530. http://www.complex-systems.com/abstracts/v03_i05_a05.html

Janikow C.Z., Michalewicz Z., An Experimental Comparison of Binary and Floating Point Representations in Genetic Algorithms, (1991) Fourth International Conference on Genetic Algorithms, pp. 31–36.

Srinivas, M., Patnaik, L. M., Srinivas. M and Patnaik. L, Adaptive probabilities of crossover and mutation in genetic algorithms (1994) IEEE Transactions on System, Man and Cybernetics, 24(4), pp. 656–667.
http://dx.doi.org/10.1109/21.286385

Zhang J., Chung, H., Lo, W.L. Clustering-Based Adaptive Crossover and Mutation Probabilities for Genetic Algorithms (2007) IEEE Transactions on Evolutionary Computation, 11(3), pp. 326–335.
http://dx.doi.org/10.1109/tevc.2006.880727

Hsu, C.-W., Lin, C.-J. A comparison of methods for multi-class support vector machines (2003) IEEE Transactions on Neural Networks, 13(2), 415–425.
http://dx.doi.org/10.1109/72.991427


Refbacks

  • There are currently no refbacks.



Please send any question about this web site to info@praiseworthyprize.com
Copyright © 2005-2024 Praise Worthy Prize