Evolutionary Computation in System Identification: Review and Recommendations


(*) Corresponding author


Authors' affiliations


DOI's assignment:
the author of the article can submit here a request for assignment of a DOI number to this resource!
Cost of the service: euros 10,00 (for a DOI)

Abstract


Two of the steps in system identification are model structure selection and parameter estimation. In model structure selection, several model structures are evaluated and selected. Because the evaluation of all possible model structures during selection and estimation of the parameters requires a lot of time, a rigorous method in which these tasks can be simplified is usually preferred. This paper reviews cumulatively some of the methods that have been tried since the past 40 years. Among the methods, evolutionary computation is known to be the most recent one and hereby being reviewed in more detail, including what advantages the method contains and how it is specifically implemented. At the end of the paper, some recommendations are provided on how evolutionary computation can be utilized in a more effective way. In short, these are by modifying the search strategy and simplifying the procedure based on problem a priori knowledge
Copyright © 2014 Praise Worthy Prize - All rights reserved.

Keywords


System Identification; Model Structure Selection; Evolutionary Computation; Genetic Algorithm; Review

Full Text:

PDF


References


R. Johansson, System Modeling & Identification (Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1993)

X. Hong, R. J. Mitchell, S. Chen, C. J. Harris, K. Li and G. W. Irwin, Model Selection Approaches for Non-Linear System Identification: A Review. International Journal of Systems Science. Vol. 39, n. 10, pp. 925-946, 2008.

S. A. Billings and M. B. Fadzil, The Practical Identification of Systems with Nonlinearities, Proceeding of the 7th IFAC Symposium of Identification and System Parameter Estimation, York, United Kingdom, July, 1985, pp. 155-160.

M. Thomson, S. P. Schooling, and M. Soufian, The Practical Application of a Nonlinear Identification Methodology. Control Engineering Practice. Vol. 4, n. 3, pp. 295-306, 1996.

L. Ljung, System Identification: Theory for the User (2nd ed., Upper Saddle River, New Jersey: Prentice Hall PTR, 1999).

S. A. Billings, Identification of Nonlinear Systems. In S. A. Billings, J. O. Gray and D. H. Owens, (Eds.). Nonlinear System Design (London, United Kingdom: Peter Peregrinus Ltd., 1984, 30-45).

S. Chen and S. A. Billings, Representations of Non-Linear Systems: The NARMAX Model. International Journal of Control. Vol. 49, n. 3, pp. 1013-1032, 1989.

S. A. Billings, H. B. Jamaluddin and S. A. Chen, Comparison of the Backpropagation and Recursive Prediction Error Algorithms for Training Neural Networks. Mechanical Systems and Signal Processing. Vol. 5, n. 3, pp. 233-255, 1991.

M. Negnevitsky, Artificial Intelligence: A Guide to Intelligent System (Harlow, England: Pearson Education Limited, 2002).

S. A. Billings and G. L. Zheng, Radial Basis Function Network Configuration Using Genetic Algorithms. Neural Networks. Vol. 8, n. 6, pp. 877-890, 1995.

R. Rashid, H. Jamaluddin and N. A. Saidina Amin, Application of Radial Basis Function Network in Modeling the Tapioca Starch Hydrolisis, Proceedings of the Second International Conference on Artificial Intelligence in Engineering & Technology, Kota Kinabalu, Sabah, Malaysia: Universiti Malaysia Sabah, Aug 3-5, 2004, pp. 897-902.

S. A. Billings and H. L. Wei, A New Class of Wavelet Networks for Nonlinear System Identification. IEEE Transactions on Neural Networks. Vol. 16, n. 4, pp. 862-874. 2005.

S. A. Billings and Y. Yang, Identification of the Neighborhood and CA Rules From Spatio-Temporal CA Patterns. IEEE Transactions on Systems, Man and Cybernetics – Part B; Cybernetics. Vol. 33, n. 2, pp. 332-339, 2003.

S. A. Billings and Y. Yang, Identification of Probabilistic Cellular Automata. IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics. Vol. 33, n. 2, pp. 225-236, 2003.

Y. Yang and S. A. Billings, Neighbourhood Detection and Rule Selection from Cellular Automata Patterns. IEEE Transactions on Systems, Man and Cybernetics – Part A: Systems and Humans. Vol. 30, n. 6, pp. 840-847, 2000.

Y. Yang and S. A. Billings, Extracting Boolean Rules from CA Patterns. IEEE Transactions on Systems, Man and Cybernetics – Part B: Cybernetics. Vol. 30, n. 4, pp. 573-581, 2000.

G. C. Goodwin and R. L. Payne, Dynamic System Identification: Experiment Design and Data Analysis (New York: Academic Press, Inc., 1977).

T. Söderström and P. Stoica, System Identification (London: Prentice Hall International (UK) Ltd., 1989).

S. M. Veres, Structure Selection of Stochastic Dynamic Systems: The Information Criterion Approach (New York: Gordon and Breach Science Publishers, 1991).

N. R. Draper and H. Smith, Applied Regression Analysis. (3rd ed., New York: John Wiley and Sons, Inc., 1998).

M. Korenberg, S. A. Billings, Y. P. Liu and P. J. McIlroy, Orthogonal Parameter Estimation Algorithm for Non-linear Stochastic Systems. International Journal of Control. Vol. 48, n. 1, pp. 193-210, 1988.

E. M. A. M. Mendes and S. A. Billings, An Alternative Solution to the Model Structure Selection Problem. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans. Vol. 31, n. 6, pp. 597-608, 2001.

H. L. Wei, S. A. Billings and J. Liu, Term and Variable Selection for Non-Linear System Identification. International Journal of Control. Vol. 77, n. 1, pp. 86-110, 2004.

I. J. Leontaritis and S. A. Billings, Model Selection and Validation Methods for Non-linear Systems. International Journal of Control. Vol. 45, n. 1, pp. 311-341, 1987.

P. Stoica, P. Eykhoff, P. Janssen and T. Söderström, Model-Structure Selection by Cross-Validation. International Journal of Control. Vol. 43, n. 6, pp. 1841-1878, 1986.

T. G. Yen, C. C. Kang and W. J. Wang, A Genetic Based Fuzzy-Neural Networks Design for System Identification, IEEE International Conference on Systems, Man and Cybernetics, Vol. 1, Big Island of Hawaii, Oct 10-12, 2005, pp. 672-678.

Y. Mitsukura, M. Fukumi, N. Akamatsu and T. Yamamoto, A System Identification Method Using a Hybrid-Type Genetic Algorithm, Proceedings of the 41st SICE Annual Conference (SICE 2002), Vol. 3, Osaka, Japan, Aug 5-7, 2002, pp. 1598-1602.

A. Sakaguchi and T. Yamamoto, A Study on System Identification Using GA and GMDH Network, The 29th Annual Conference of the IEEE Industrial Electronics Society (IECON’03), Vol. 3, Roanoke, Virginia, Nov 2-6, 2003, pp. 2387-2392.

G. C. Luh and G. Rizzoni, Nonlinear System Identification Using Genetic Algorithms with Application to Feedforward Control Design, Proceedings of the American Control Conference. Vol. 4, Philadelphia, Pennsylvania, June 24-26, 1998, pp. 2371-2375.

G. C. Luh and C. Y. Wu, Nonlinear System Identification Using Genetic Algorithms. Proceedings of the Institution of Mechanical Engineers - Part I: Journal of Systems and Control Engineering. Vol. 213, n. 2, pp. 105-118, 1999.

R. Ahmad, H. Jamaluddin and M. A. Hussain, Selection of a Model Structure in System Identification Using Memetic Algorithm, Proceedings of the Second International Conference on Artificial Intelligence in Engineering & Technology, Kota Kinabalu, Sabah, Malaysia: Universiti Malaysia Sabah, Aug 3-5, 2004, pp. 714-720.

R. Ahmad, H. Jamaluddin and M. A. Hussain, Model Structure Selection for a Discrete-Time Non-Linear System Using Genetic Algorithm. Proceedings of the Institution of Mechanical Engineers - Part I: Journal of Systems and Control Engineering. Vol. 218, n. 2, pp. 85-98. 2004.

H. Jamaluddin, M. F. Abd Samad, R. Ahmad and M. S. Yaacob, Optimum Grouping in a Modified Genetic Algorithm for Discrete-Time, Non-Linear System Identification. Proceeding of Institution of Mechanical Engineers Part I: Journal of Systems and Control Engineering.Vol. 221, pp. 975-989, 2007.

M. F. Abd Samad, H. Jamaluddin, R. Ahmad, M.S. Yaacob and A. K. M. Azad, Deterministic Mutation-Based Algorithm for Model Structure Selection in Discrete-Time System Identification. International Journal of Intelligent Control and Systems. Vol. 16, n. 3, pp. 182-190, 2011.

X. Yao, Evolutionary Computation, In R. Sarker, , M. Mohammadian, and X. Yao, (Eds.). Evolutionary Optimization (Boston: Kluwer Academic Publishers., 2002, 27-56).

R. Sarker, M. Mohammadian and X. Yao, (Eds.) Evolutionary Optimization (Boston: Kluwer Academic Publishers, 2002).

T. Bäck, Introduction to Evolutionary Algorithms, In T. Bäck, D. B. Fogel and Z. Michalewicz, (Eds.) Evolutionary Computation 1: Basic Algorithms and Operators (Bristol: Institute of Physics Publishing, 2000, 59-63).

D. B. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence (Hoboken, New Jersey: John Wiley & Sons, Inc., 2006).

K. A. De Jong, Evolutionary Computation: A Unified Approach (Cambridge, Massachusetts: The MIT Press, 2006).

D. B. Fogel, Introduction to Evolutionary Computation, In T. Bäck, D. B. Fogel and Z. Michalewicz (Eds.) Evolutionary Computation 1: Basic Algorithms and Operators (Bristol: Institute of Physics Publishing. 2000, 1-3)

J. H. Holland, Genetic Algorithms and the Optimal Allocation of Trials. SIAM Journal of Computing. Vol. 2, n. 2, pp. 88-105, 1973.

J. H. Holland. Adaptation in Natural and Artificial Systems (MIT Press ed., Massachusetts Institute of Technology, 1992).

D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning (Reading, Massachusetts: Addison-Wesley Publishing Company, Inc., 1989).

J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (Cambridge, Massachusetts: The MIT Press, 1992).

J. R. Koza, Simultaneous Discovery of Detectors and a Way of Using the Detectors via Genetic Programming, IEEE International Conference on Neural Networks.Vol. 3, San Francisco, California, Mar 28-Apr 1, 1993, pp. 1794-1801.

T. Bäck, U. Hammel and H.-P. Schwefel, Evolutionary Computation: Comments on the History and Current State, IEEE Transactions on Evolutionary Computation. Vol. 1, n. 1, pp. 3-17, 1997.

T. Bäck, D. B. Fogel and Z. Michalewicz, (Eds.) Evolutionary Computation 1: Basic Algorithms and Operators (Bristol: Institute of Physics Publishing, 2000).

D. Beasley, Possible Applications of Evolutionary Computation. In T. Bäck, D. B. Fogel and Z. Michalewicz (Eds.) Evolutionary Computation 1: Basic Algorithms and Operators (Bristol: Institute of Physics Publishing, 2000, 4-19)

M. Gen and R. Cheng, Genetic Algorithms and Engineering Design (New York: John Wiley & Sons, Inc., 1997).

A. P. Alves da Silva and P. J. Abrão, Applications of Evolutionary Computation in Electric Power Systems, Proceedings of the 2002 Congress on Evolutionary Computation (CEC’02), Vol. 2, Honolulu, Hawaii, USA, May 12-17, 2002, pp. 1057-1062.

D. S. Weile and E. Michielssen, Genetic Algorithm Optimization Applied to Electromagnetics: A Review. IEEE Transactions on Antennas and Propagation. Vol. 45, n. 3, pp. 343-353, 1997.

C. Dimopoulos and A. M. S. Zalzala, Recent Developments in Evolutionary Computation for Manufacturing Optimization: Problems, Solutions and Comparisons. IEEE Transactions on Evolutionary Computation. Vol. 4, n. 2, pp. 93-113, 2000.

H. Aytug, M. Khouja, and F. E. Vergara, Use of Genetic Algorithms to Solve Production and Operations Management Problems: A Review. International Journal of Production Research. Vol. 41, n. 17, pp. 3955-4009. 2003.

P. J. Fleming and R. C. Purshouse, Evolutionary Algorithms in Control Systems Engineering: A Survey. Control Engineering Practice. Vol. 10, n. 11, pp. 1223-1241. 2002.

T. Bäck, Adaptive Business Intelligence Based on Evolution Strategies: Some Application Examples of Self-Adaptive Software. Information Sciences. Vol. 148, n. 1-4, pp. 113-121. 2002.

M. S. Hossain and A. El-shafie, Intelligent Systems in Optimizing Reservoir Operation Policy: A Review. Water Resources Management. Vol. 27, n. 9, pp. 3387-3407, 2013.

N. Chaiyaratana and A. M. S. Zalzala, Recent Developments in Evolutionary and Genetic Algorithms: Theory and Applications. Second IEE/IEEE International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications (GALESIA), Conference Publication No. 446, Glasgow, UK, Sept 2-4, 1997, pp. 270-277.

Al Gizi, A.J.H., Mustafa, M.W., Review of intelligent control system, (2013) International Review of Automatic Control (IREACO), 6 (5), pp. 565-588.

T. Kumon, M. Iwasaki, T.Suzuki, T. Hashiyama, N. Matsui and S. Okuma, Nonlinear System Identification Using Genetic Algorithms, 26th Annual Conference of the IEEE Industrial Electronics Society (IECON 2000), Vol. 4, Nagoya, Japan, Oct 22-28, 2000, pp. 2485-2491.

R. Ahmad, H. Jamaluddin and M. A. Hussain, Multivariable System Identification for Dynamic Discrete-Time Nonlinear System Using Genetic Algorithm, IEEE International Conference on Systems, Man and Cybernetics, Vol. 5. Hammamet, Tunisia, Oct 6-9, 2002, 6 pages.

Z. Zibo and F. Naghdy, Application of Genetic Algorithms to System Identification, Proceedings of the 1995 International Conference on Evolutionary Computing, Vol. 2, Perth, Western Australia, Nov 29-Dec 1, 1995, pp. 777-782.

V. Duong and A. R. Stubberud, System Identification by Genetic Algorithm, IEEE Aerospace Conference Proceedings, Vol. 5, Big Sky, Montana, Mar 9-16, 2002, pp. 2331-2337.

K. C. Tan, Y. Li, D. J. Murray-Smith and K. C. Sharman, System Identification and Linearisation Using Genetic Algorithms with Simulated Annealing, First IEE/IEEE International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications (GALESIA), Conference Publication No. 414, Sheffield, UK, Sept 12-14, 1995, pp. 164-169.

A. B. Tcholakian, A. Martins, R. C. S. Pacheco and R. M. Barcia, Fuzzy System Identification Through Hybrid Genetic Algorithm, 1997 Annual Meeting of the North American Fuzzy Information Processing Society (NAFIPS’97), Syracuse, New York, Sept 21-24, 1997, pp. 428-433.

K. Kristinsson and G. A. Dumont, System Identification and Control Using Genetic Algorithms. IEEE Transactions on Systems, Man and Cybernetics. Vol. 22, n. 5, pp. 1033-1046, 1992.

T. Hatanaka, K. Uosaki and M. Koga, Evolutionary Computation Approach to Block Oriented Nonlinear Model Identification, 5th Asian Control Conference. Vol. 1, Melbourne, Australia, July 20-23, 2004, pp. 90-96.

X. Tan and H. Yang, The Optimization of Nonlinear Systems Identification Based on Genetic Algorithms, International Conference on Computational Intelligence and Security, Vol. 1, Guangzhou, China, Nov 3-6, 2006, pp. 266-269.

J.-G. Juang, Application of Genetic Algorithm and Recurrent Network to Nonlinear System Identification, Proceedings of 2003 IEEE Conference on Control Applications, Vol. 1, Istanbul, Turkey, June 23-25, 2003, pp. 129-134.

H. M. Abbas and M. M. Bayoumi, Volterra-System Identification Using Adaptive Real-Coded Genetic Algorithm. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans. Vol. 36, n. 4, pp. 671-684, 2006.

M. Iwasaki, M. Miwa and N. Matsui, GA-Based Evolutionary Identification Algorithm for Unknown Structured Mechatronic Systems. IEEE Transactions on Industrial Electronics. Vol. 52, n. 1, pp. 300-305, 2005.

M. Rocha, P. Sousa, P. Cortez and M. Rio, Quality of Service Constrained Routing Optimization using Evolutionary Computation. Applied Soft Computing, Vol. 11, n. 1, pp. 356–364, 2011.

O. Valenzuela, B. Delgado-Marquez and M. Pasadas, Evolutionary Computation for Optimal Knots Allocation in Smoothing Splines, Applied Mathematical Modelling. Vol. 37, n. 8, pp. 5851–5863, 2013.

J. F. Izquierdo and J. Rubio, Antenna Modeling by Elementary Sources Based on Spherical Waves Translation and Evolutionary Computation. IEEE Antennas and Wireless Propagation Letters. Vol. 10, pp. 923 – 926, 2011.

D. Marco, D. Cairns and C. Shankland, Optimisation of Process Algebra Models using Evolutionary Computation, IEEE Congress on Evolutionary Computation (CEC), New Orleans, Louisiana, June 5-8, 2011, pp. 1296 – 1301.

V. K. Dertimanis, D. V. Koulocheris and C. N. Spentzas, Design of a Hybrid Algorithm for ARMA Parameter Estimation. 5th GRACM International Congress on Computational Mechanics, Vol. 1, Limassol, Cyprus, June 29-July 1, 2005, pp. 395-402.

C. Unsihuay-Vila, A. C. Zambroni de Souza, J. W. Marangon-Lima and P. P. Balestrassi, Electricity Demand and Spot Price Forecasting using Evolutionary Computation Combined with Chaotic Nonlinear Dynamic Model. International Journal of Electrical Power & Energy Systems. Vol. 32, n. 2, pp. 108–116, 2010.

C. Sales, R. M. Rodrigues, F. Lindqvist, J. Costa, A. Klautau, K. Ericson, J. Rius i Riu and P. O. Borjesson, Line Topology Identification using Multiobjective Evolutionary Computation. IEEE Transactions on Instrumentation and Measurement. Vol. 59, n. 3, pp. 715-729, 2010.

D. Koulocheris, V. Dertimanis and H. Vrazopoulos, Evolutionary Parametric Identification of Dynamic Systems. Forschung im Ingenieurwesen. Vol. 68, n. 4, pp. 173-181, 2004.

D. S. Pereira and J. O. P. Pinto, Genetic Algorithm Based System Identification and PID Tuning for Optimum Adaptive Control, Proceedings of the 2005 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Monterey, California, July 24-28, 2005, pp. 801-806.

M. J. Perry, G. C. Koh and Y. S. Choo, Modified Genetic Algorithm Approach to Structural Identification, The 3rd International Conference on Structural Stability and Dynamics, Kissimmee, Florida, June 19-22, 2005, 6 pages.

K. Asan Mohideen, G. Saravanakumar, K. Valarmathi, D. Devaraj and T.K. Radhakrishnan, Real-Coded Genetic Algorithm for System Identification and Tuning of a Modified Model Reference Adaptive Controller for a Hybrid Tank System. Applied Mathematical Modelling. Vol. 37, n. 6, pp. 3829–3847, 2013.

A. Kusiak, H. Zheng, and Z. Song, Power optimization of wind turbines with data mining and evolutionary computation. Renewable Energy. Vol. 35, n. 3, pp. 695–702, 2010.

S. Salcedo-Sanz, D. Gallo-Marazuela, A. Pastor-Sánchez, L. Carro-Calvo, A. Portilla-Figueras and L. Prieto, Evolutionary computation approaches for real offshore wind farm layout: A case study in northern Europe. Expert Systems with Applications. Vol. 40, n. 16, pp. 6292–6297, 2013.

A. H. Wright, Genetic Algorithms for Real Parameter Optimization. In G. J. E. Rawlins, (Ed.) Foundations of Genetic Algorithms (San Mateo, California: Morgan Kaufmann Publishers, Inc., 1991, 205-220)

T. Bäck and D. B. Fogel, Glossary, In T. Bäck, D. B. Fogel and Z. Michalewicz, (Eds.). Evolutionary Computation 1: Basic Algorithms and Operators (Bristol: Institute of Physics Publishing, 2000, xxi-xxxvii)

Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs (3rd, revised and extended ed., Berlin: Springer-Verlag, 1996).

A. Czarn, C. MacNish, K. Vijayan and B. Turlach, Statistical Exploratory Analysis of Genetic Algorithms: The Importance of Interaction, Proceedings of the 2004 IEEE Congress on Evolutionary Computation (CEC’04), Vol. 2, Portland, Oregon, June 19-23, 2004, pp. 2288-2295.

A. M. S. Zalzala and P. J. Fleming (Eds.) Genetic Algorithms in Engineering Systems (London, United Kingdom: The Institution of Electrical Engineers, 1997).

T. A. Johansen, Identification of Non-Linear Systems using Empirical Data and Prior Knowledge – An Optimization Approach. Automatica. Vol. 32, n. 3, pp. 337-356, 1996.

Y.-X. Li and M.Gen, Nonlinear Mixed Integer Programming Problem Using Genetic Algorithm and Penalty Function, IEEE International Conference on Systems, Man and Cybernetics, Vol. 4, Beijing, China, Oct 14-17, 1996, pp. 2677-2682.

T. Yamada and C. R. Reeves, Permutation Flowshop Scheduling by Genetic Local Search. Second IEE/IEEE International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications (GALESIA), Conference Publication No. 446, Glasgow, UK, Sept 2-4, 1997, pp. 270-277.

A. Kanarachos, D. Koulocheris and H. Vrazopoulos, Evolutionary Algorithms with Deterministic Mutation Operators Used for the Optimization of the Trajectory of a Four-Bar Mechanism. Mathematics and Computers in Simulation. Vol. 63, n. 6, pp. 483-492, 2003.

Y. Liang and K. S. Leung, Two-Way Mutation Evolution Strategies, Proceedings of the 2002 IEEE Congress on Evolutionary Computation (CEC’02), Vol. 1, Honolulu, Hawaii May 12-17, 2002, pp. 789-794.

J. J. Grefenstette, Optimization of Control Parameters for Genetic Algorithms. IEEE Transactions on Systems, Man and Cybernetics. Vol. 16, n. 1, pp. 122-128, 1986.

H. Mühlenbein, L. Zinchenko, V. Kureichik and T. Mahnig, Effective Mutation Rate for Probabilistic Evolutionary Design of Analogue Electrical Circuits. Applied Soft Computing. Vol. 7, n. 3, pp. 1012-1018, 2007.

K. A. De Jong, An Analysis of the Behavior of a Class of Genetic Adaptive Systems. Ph.D. dissertation, The University of Michigan, 1975.

L. B. Booker, D. B. Fogel, D. Whitley, P. J. Angeline and A. E. Eiben, Recombination. In T. Bäck, D. B. Fogel and Z. Michalewicz, (Eds.) Evolutionary Computation 1: Basic Algorithms and Operators (Bristol: Institute of Physics Publishing, 2000, 256-307)

T. Murata and H. Ishibuchi, Positive and Negative Combination Effects of Crossover and Mutation Operators in Sequencing Problems, Proceedings of the 1996 IEEE International Conference on Evolutionary Computation, Nagoya, Japan, May 20-22, 1996, pp. 170-175.

A. E. Eiben, R. Hinterding and Z. Michalewicz, Parameter Control, In T, Bäck, D. B. Fogel, and Z. Michalewicz (Eds.) Evolutionary Computation 2: Advanced Algorithms and Operators (Bristol: Institute of Physics Publishing, 2000, 170-187).


Refbacks

  • There are currently no refbacks.


Please send any question about this web site to info@praiseworthyprize.com
Copyright © 2005-2024 Praise Worthy Prize