An ensemble method for data stream classification in the presence of concept drift

ABBASZADEH Omid; AMIRI Ali; KHANTEYMOORI Ali Reza

doi:10.1631/FITEE.1400398

Your Location：

Home >

Browse articles >

An ensemble method for data stream classification in the presence of concept drift

Computer & Automation | Updated：2022-06-06

- An ensemble method for data stream classification in the presence of concept drift
- 一种概念漂移情况下数据流分类的整体方法
- Frontiers of Information Technology & Electronic Engineering Vol. 16, Issue 12, Pages: 1059-1068(2015)
- Affiliations：
  
  Department of Computer Engineering, University of Zanjan, Zanjan 45371-38791, Iran
- Author bio：
  
  [ "Omid ABBASZADEH, E-mail: o.abbaszadeh@znu.ac.ir" ]
  Ali AMIRI, E-mail: a_amiri@znu.ac.ir
  [ "Ali Reza KHANTEYMOORI, E-mail: khanteymoori@znu.ac.ir" ]
- Funds：
- DOI：10.1631/FITEE.1400398
  CLC： TP391
- Received：19 November 2014，
  
  Revised：2015-11-11，
  
  Published：2015-12
- Accepted：
Scan QR Code
ABBASZADEH Omid, AMIRI Ali, KHANTEYMOORI Ali Reza. An ensemble method for data stream classification in the presence of concept drift[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(12): 1059-1068.
DOI：

ABBASZADEH Omid, AMIRI Ali, KHANTEYMOORI Ali Reza. An ensemble method for data stream classification in the presence of concept drift[J]. Frontiers of Information Technology & Electronic Engineering, 2015, 16(12): 1059-1068. DOI： 10.1631/FITEE.1400398.

摘要

目的

数据流（data stream）管理和处理是计算机科学领域的热点问题。本文提及的“数据流”指连续且快速生成的数据包。数据流的专有特性有数据量极大、生成率高、处理时间有限和数据概念漂移（concept drift）等。这些特性将数据流区别于其他标准数据形式。数据流的一个重要问题即为输入数据分类。本文提出一种新型的整体分类器（ensemble classifier）。

创新点

在数据流分类器的基础上，提出一种包含概念漂移检测、基分类器移除和动态加权机制的方法。

方法

（1）针对不同数据输入条件，对基分类器使用两种加权函数；（2）利用Kappa系数确定概念漂移，提升算法精度；（3）基于基分类器的质量，移除不同数目的基分类器；（4）在决策阶段对基分类器应用加权机制，提升算法对漂移的适应性，提高分类器效率。

结论

在标准数据集上测试，本文方法较现有整体分类器和单分类器可获得更高的精度；在某些情况下可节省运行时间和内存用量。

Abstract

One recent area of interest in %different fields of computer science is data stream management and processing. By `data stream'

we refer to continuous and rapidly generated packages of data. Specific features of data streams are immense volume

high production rate

limited data processing time

and data concept drift; these features differentiate the data stream from standard types of data. An issue for the data stream is classification of input data. A novel ensemble classifier is proposed in this paper. The classifier uses base classifiers of two weighting functions under different data input conditions. In addition

a new method is used to determine drift

which emphasizes the precision of the algorithm. Another characteristic of the proposed method is removal of different numbers of the base classifiers based on their quality. Implementation of a weighting mechanism to the base classifiers at the decision-making stage is another advantage of the algorithm. This facilitates adaptability when drifts take place

which leads to classifiers with higher efficiency. Furthermore

the proposed method is tested on a set of standard data and the results confirm higher accuracy compared to available ensemble classifiers and single classifiers. In addition

in some cases the proposed classifier is faster and needs less storage space.

关键词

Keywords

references

M. Baena-García , J. del Campo-Ávila , R. Fidalgo , 等 . Early drift detection method . Early drift detection method. ECML PKDD. , 2006 .

A. Bifet . Adaptive learning and mining for data streams and frequent patterns . ACM SIGKDD Explor. Newsl. , 2009 . 11 ( 1 ): 55 - 56 . DOI: 10.1145/1656274.1656287 http://doi.org/10.1145/1656274.1656287 .

A. Bifet , G. Holmes , R. Kirkby , 等 . MOA: massive online analysis . J. Mach. Learn. Res. , 2010 . 11 1601 - 1604 . .

D. Brzezinski , J. Stefanowski . Reacting to different types of concept drift: the accuracy updated ensemble algorithm . IEEE Trans. Neur. Netw. Learn. Syst. , 2014 . 25 ( 1 ): 81 - 94 . DOI: 10.1109/TNNLS.2013.2251352 http://doi.org/10.1109/TNNLS.2013.2251352 .

J. Gama . Knowledge Discovery from Data Streams . Chapman & Hall/CRC, London. , 2010 .

J. Gama , P. Medas , G. Castillo , 等 . Learning with drift detection . Brazilian Symp. on Artificial Intelligence , 2004 . p.286 - 295 . DOI: 10.1007/978-3-540-28645-5_29 http://doi.org/10.1007/978-3-540-28645-5_29 .

G. Hulten , L. Spencer , P. Domingos . Mining timechanging data streams . Proc. 7th ACM SIGKDD Int. Conf. on Knowledge Discovery Data Mining , 2001 . p.97 - 106 . DOI: 10.1145/502512.502529 http://doi.org/10.1145/502512.502529 .

T. Jiang , Y.C. Feng , B. Zhang , 等 . Monitoring correlative financial data streams by local pattern similarity . J. Zhejiang Univ.-Sci. A , 2009 . 10 ( 7 ): 937 - 951 . DOI: 10.1631/jzus.A0820445 http://doi.org/10.1631/jzus.A0820445 .

J.Z. Kolter , M.A. Maloof . Dynamic weighted majority: an ensemble method for drifting concepts . J. Mach. Learn. Res. , 2007 . 8 2755 - 2790 . .

L.I. Kuncheva . Combining Pattern Classifiers: Methods and Algorithms . John Wiley & Sons, Hoboken. , 2004 .

L.L. Minku , X. Yao . DDD: a new ensemble approach for dealing with concept drift . IEEE Trans. Knowl. Data Eng. , 2012 . 24 ( 4 ): 619 - 633 . DOI: 10.1109/TKDE.2011.58 http://doi.org/10.1109/TKDE.2011.58 .

N.C. Oza . Online bagging and boosting . IEEE Int. Conf. on System and Man Cybernetics , 2005 . p.2340 - 2345 . DOI: 10.1109/ICSMC.2005.1571498 http://doi.org/10.1109/ICSMC.2005.1571498 .

S. Ruping . Incremental learning with support vector machines . IEEE 13th Int. Conf. on Data Mining , 2001 . p.641 - 642 . DOI: 10.1109/ICDM.2001.989589 http://doi.org/10.1109/ICDM.2001.989589 .

J. Sim , C.C. Wright . The kappa statistic in reliability studies: use, interpretation, and sample size requirements . Phys. Ther. , 2005 . 85 ( 3 ): 257 - 268 . .

W.N. Street , Y.S. Kim . A streaming ensemble algorithm (SEA) for large-scale classification . Proc. 7th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining , 2001 . p.377 - 382 . DOI: 10.1145/502512.502568 http://doi.org/10.1145/502512.502568 .

A. Tsymbal , M. Pechenizkiy , P. Cunningham , 等 . Dynamic integration of classifiers for handling concept drift . Inform. Fus. , 2008 . 9 ( 1 ): 56 - 68 . DOI: 10.1016/j.inffus.2006.11.002 http://doi.org/10.1016/j.inffus.2006.11.002 .

H. Wang , W. Fan , P.S. Yu , 等 . Mining concept-drifting data streams using ensemble classifiers . Proc. 9th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining , 2003 . p.226 - 235 . DOI: 10.1145/956750.956778 http://doi.org/10.1145/956750.956778 DOI: 10.1145/956750.956778 http://doi.org/10.1145/956750.956778 .

W.H. Xu , Z. Qin , Y. Chang . Clustering feature decision trees for semi-supervised classification from high-speed data streams . J. Zhejiang Univ.-Sci. C (Comput. & Electron.) , 2011 . 12 ( 8 ): 615 - 628 . DOI: 10.1631/jzus.C1000330 http://doi.org/10.1631/jzus.C1000330 DOI: 10.1631/jzus.C1000330 http://doi.org/10.1631/jzus.C1000330 .

P. Zhang , X. Zhu , Y. Shi . Categorizing and mining concept drifting data streams . Proc. 14th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining , 2008 . p.812 - 820 . DOI: 10.1145/1401890.1401987 http://doi.org/10.1145/1401890.1401987 .

P. Zhang , C. Zhou , P. Wang , 等 . E-tree: an efficient indexing structure for ensemble models on data streams . IEEE Trans. Knowl. Data Eng. , 2015 . 27 ( 2 ): 461 - 474 . DOI: 10.1109/TKDE.2014.2298018 http://doi.org/10.1109/TKDE.2014.2298018 .

X. Zhu , P. Zhang , X. Lin , 等 . Active learning from stream data using optimal weight classifier ensemble . IEEE Trans. Syst. Man Cybern. B , 2010 . 40 ( 6 ): 1607 - 1621 . DOI: 10.1109/TSMCB.2010.2042445 http://doi.org/10.1109/TSMCB.2010.2042445 .

I. Žfliobaite . Learning under Concept Drift: an Overview . Technical Report. Vilnius University, Lithuania. , 2009 .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

FAAD: an unsupervised fast and accurate anomaly detection method for a multi-dimensional sequence over data stream

An efficient online histogram publication method for data streams with local differential privacy

Related Author

Bin LI

Yi-jie WANG

Dong-sheng YANG

Yong-mou LI

Xing-kong MA

Tao TAO

Funan ZHANG

Xiujun WANG

Related Institution

Science and Technology on Parallel and Distributed Processing Laboratory, College of Computer, National University of Defense Technology

Block Chain Research Institute of LianLian Pay

School of Computer Science and Technology, Anhui University of Technology

Anhui Engineering Research Center for Intelligent Applications and Security of Industrial Internet

Engineering Research Institute, Anhui University of Technology

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰