Method for Detecting and Eliminating Data Time Series Outlier in High-Speed Process Data Sensors

Fariza Tebueva; Vladimir Kopytov; Viacheslav Petrenko; Pavel Kharechkin; Alesia Sidorchuk

doi:10.15866/irecap.v7i7.13453

Method for Detecting and Eliminating Data Time Series Outlier in High-Speed Process Data Sensors

Fariza Tebueva^(1*), Vladimir Kopytov⁽²⁾, Viacheslav Petrenko⁽³⁾, Pavel Kharechkin⁽⁴⁾, Alesia Sidorchuk⁽⁵⁾

^(*) Corresponding author

Authors' affiliations

DOI: https://doi.org/10.15866/irecap.v7i7.13453

Abstract

The objective is to identify and eliminate measurement errors in data time series of data sensors of high-speed processes. Fractal analysis is used to identify and eliminate outliers in data time series of data sensors of high-speed processes. The proposed method provides an accurate detection of an outlier and an acceptable error in a series characteristic modification when the outlier is replaced with a predicted value. As a result, the time series becomes persistent again. The originality of the method consists in using such a hidden regularity of time series as a correlation of values to detect measurement errors. The importance of the study is determined by the immediacy of the problem of detecting and eliminating outliers in data time series of data sensors with stringent requirements imposed upon to reduce decision-making time in the event of critical situations.
Copyright © 2017 Praise Worthy Prize - All rights reserved.

Keywords

High-Speed Processes; Measurement Errors; Outliers; Persistence

Full Text:

PDF

References

L.W.P. Biscainho, Random Signals and Stochastic Processes, Academic Press Library in Signal Processing, Volume. 1. Signal Processing Theory and Machine Learning. Edited by Diniz, P. S.R.; Suykens, J.A.K.; Chellappa R.; Theodoridis, S. Chapter 4, 2014, pp.113-168.
http://dx.doi.org/10.1016/b978-0-12-396502-8.00004-8

M.R. Maleki, A.Amiri,; Castagliola, P. Measurement errors in statistical process monitoring: A literature review. Computers & Industrial Engineering, 103, (2017), pp. 316-329.
http://dx.doi.org/10.1016/j.cie.2016.10.026

Siegel E. Practical Business Statistics. Academic Press; 6 edition. 2011.

Efstratiadis, A., Dialynas, Y.G., Kozanis, S., Koutsoyiannis, D. A multivariate stochastic model for the generation of synthetic time series at multiple time scales reproducing long-term persistence. Environmental Modelling & Software, 62(2014), pp. 139-152.
http://dx.doi.org/10.1016/j.envsoft.2014.08.017

Dave D., Varma T. A Review of various statistical methods for Outlier Detection. International Journal of Computer Science & Engineering Technology, 5, 02(2014), pp. 137–140.

URL: http://www.ijcset.com/docs/IJCSET14-05-02-064.pdf. (12/02/2016)

de Oliveira, E.C., de Faro Orlando, A., dos Santos Ferreira, A.L., de Oliveira Chaves, C.E. Comparison of different approaches for detection and treatment of outliers in meter proving factors determination. Flow Measurement and Instrumentation, 48(2016), pp. 29-35.
http://dx.doi.org/10.1016/j.flowmeasinst.2016.02.002

Zhu, Y., Ting, K.M., Carman, M.J. Density-ratio based clustering for discovering clusters with varying densities. Pattern Recognition, 60(2016), pp. 983-997.
http://dx.doi.org/10.1016/j.patcog.2016.07.007

Qi, M., Fu, Z., Chen, F. Outliers detection method of multiple measuring points of parameters in power plant units. Applied Thermal Engineering, 85(2015), pp. 297-303.
http://dx.doi.org/10.1016/j.applthermaleng.2015.04.008

Yuen, K.-V.; Ortiz, G.A. Outlier detection and robust regression for correlated data. Computer Methods in Applied Mechanics and Engineering, 313(2017), pp. 632-646.
http://dx.doi.org/10.1016/j.cma.2016.10.004

Sangeux, M., Polak, J. A simple method to choose the most representative stride and detect outliers. Gait & Posture, 41, 2(2015), pp. 726-730.
http://dx.doi.org/10.1016/j.gaitpost.2014.12.004

Zhang, L., Wang, D., Gao, R., Li, P., Zhang, W., Mao, J., Yu, L., Ding, X., Zhang, Q. Improvement on enhanced Monte-Carlo outlier detection method. Chemometrics and Intelligent Laboratory Systems, 151(2016), pp. 89-94.
http://dx.doi.org/10.1016/j.chemolab.2015.12.006

Aljoumani, B., Sànchez-Espigares, J.A., Cañameras, N., Josa, R., Monserrat, J. Time series outlier and intervention analysis: Irrigation management influences on soil water content in silty loam soil. Agricultural Water Management, 111(2012), pp. 105-114.
http://dx.doi.org/10.1016/j.agwat.2012.05.008

Guo, J., Huang, W., Williams, B.M. Real time traffic flow outlier detection using short-term traffic conditional variance prediction. Transportation Research Part C: Emerging Technologies, 50(2015), pp. 160-172.
http://dx.doi.org/10.1016/j.trc.2014.07.005

Marczak, M., Proietti, T. Outlier detection in structural time series models: The indicator saturation approach. International Journal of Forecasting, 32, 1(2016), pp. 180-202.
http://dx.doi.org/10.1016/j.ijforecast.2015.04.005

Tang, B., He, H. A Local Density-Based Approach for Outlier Detection. Neurocomputing, 241(2017), 171-180.
http://dx.doi.org/10.1016/j.neucom.2017.02.039

Huang, J., Zhu, Q., Yang, L., Cheng, D.D., Wu, Q. A novel outlier cluster detection algorithm without top-n parameter. Knowledge-Based Systems, 121(2017), pp. 32-40.
http://dx.doi.org/10.1016/j.knosys.2017.01.013

Ha, J., Seok, S., Lee, J.-S. A precise ranking method for outlier detection. Information Sciences, 324, 10(2015), pp. 88-107.
http://dx.doi.org/10.1016/j.ins.2015.06.030

Bhattacharya, G., Ghosh, K., Chowdhury, A.S. Outlier detection using neighborhood rank difference. Pattern Recognition Letters, 60-61(2015), pp. 24-31.
http://dx.doi.org/10.1016/j.patrec.2015.04.004

Pasillas-Díaz, J.R., Ratté, S. An Unsupervised Approach for Combining Scores of Outlier Detection Techniques, Based on Similarity Measures. Electronic Notes in Theoretical Computer Science. 329(2016), pp. 61-77.
http://dx.doi.org/10.1016/j.entcs.2016.12.005

Gunupudi, R.K., Nimmala, M., Gugulothu, N., Gali, S.R CLAPP: A self-constructing feature clustering approach for anomaly detection. Future Generation Computer Systems, In Press, Corrected Proof, Available online 4 January 2017.
http://dx.doi.org/10.1016/j.future.2016.12.040

Huang, J., Zhu, Q., Yang, L., Feng, J. A non-parameter outlier detection algorithm based on Natural Neighbor. Knowledge-Based Systems, 92(2016), pp. 71-77.
http://dx.doi.org/10.1016/j.knosys.2015.10.014

Maciá-Pérez, F., Berna-Martinez, J.V., Oliva, A.F., Ortega, M.A.A. Algorithm for the detection of outliers based on the theory of rough sets. Decision Support Systems, 75(2015), pp. 63-75.
http://dx.doi.org/10.1016/j.dss.2015.05.002

Chiang, A., David, E., Lee, Y.-J., Leshem, G., Yeh, Y.-R. A study on anomaly detection ensembles. Journal of Applied Logic, 21(2017), pp. 1-13.
http://dx.doi.org/10.1016/j.jal.2016.12.002

Gil, P., Martins H., Januário, F. Detection and accommodation of outliers in Wireless Sensor Networks within a multi-agent framework. Applied Soft Computing, 42(2016), pp. 204–214.
http://dx.doi.org/10.1016/j.asoc.2015.12.042

Saka M.H. Performance evaluation of outlier detection methods in GNSS vector networks using 1D and 3D component analysis. Measurement, 82(2016), pp. 145-150.
http://dx.doi.org/10.1016/j.measurement.2015.12.029

Zhang, L., Lin, J., Karim, R. An angle-based subspace anomaly detection approach to high-dimensional data: With an application to industrial fault detection. Reliability Engineering & System Safety, 142(2015), pp. 482-497.
http://dx.doi.org/10.1016/j.ress.2015.05.025

GOST R ISO 5479-2002. Statistical methods. Test for departure of the probability distribution from the normal distribution. 2002.

Peters, E.E. Chaos and Order in the Capital Markets: A New View of Cycles, Prices, and Market Volatility. (WILEY FINANCE). Moscow: Mir, 2000.

Shelukhin, O.I. Multifractals. Infocommunication applications. Moscow: Goryachaya Liniya – Telekom, 2011.

Vaclavovic, P., Dousa, J. Backward smoothing for precise GNSS applications. Advances in Space Research. 56, 8, (2015), pp. 1627-1634.
http://dx.doi.org/10.1016/j.asr.2015.07.020

Deb, C., Zhang, F., Yang, J., Lee, S.E., Shah, K.W. A review on time series forecasting techniques for building energy consumption. Renewable and Sustainable Energy Reviews, 74(2017), pp. 902-924.
http://dx.doi.org/10.1016/j.rser.2017.02.085

Pylkin, A.N., Demidova, L.A., Skvortsov, S.V., Skvortsova, T.S. Hybrid models for forecasting short time series. Moscow: Horline-Telecom; 2012.

Gupta, P., Batra, S.S., Jayadeva. Sparse short-term time series forecasting models via minimum model complexity. Neurocomputing, 243(2017), pp. 1-11.
http://dx.doi.org/10.1016/j.neucom.2017.02.002

Sadaei, H.J., Guimarães, F.G., da Silva, C.J., Lee, M.H., Eslami, T. Short-term load forecasting method based on fuzzy time series, seasonality and long memory process. International Journal of Approximate Reasoning, 83(2017), pp. 196-217.
http://dx.doi.org/10.1016/j.ijar.2017.01.006

Refbacks

There are currently no refbacks.

Username
Password
Remember me