Skip to main navigation Skip to search Skip to main content

Classification and novel class detection in concept-drifting data streams under time constraints

  • Mohammad Masud
  • , Jing Gao
  • , Latifur Khan
  • , Jiawei Han
  • , Bhavani M. Thuraisingham
  • University of Texas at Dallas
  • University of Illinois at Urbana-Champaign

Research output: Contribution to journalArticlepeer-review

362 Scopus citations

Abstract

Most existing data stream classification techniques ignore one important aspect of stream data: arrival of a novel class. We address this issue and propose a data stream classification technique that integrates a novel class detection mechanism into traditional classifiers, enabling automatic detection of novel classes before the true labels of the novel class instances arrive. Novel class detection problem becomes more challenging in the presence of concept-drift, when the underlying data distributions evolve in streams. In order to determine whether an instance belongs to a novel class, the classification model sometimes needs to wait for more test instances to discover similarities among those instances. A maximum allowable wait time Tc is imposed as a time constraint to classify a test instance. Furthermore, most existing stream classification approaches assume that the true label of a data point can be accessed immediately after the data point is classified. In reality, a time delay Tl is involved in obtaining the true label of a data point since manual labeling is time consuming. We show how to make fast and correct classification decisions under these constraints and apply them to real benchmark data. Comparison with state-of-the-art stream classification techniques prove the superiority of our approach.

Original languageEnglish
Article number5453372
Pages (from-to)859-874
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume23
Issue number6
DOIs
StatePublished - 2011

Keywords

  • concept-drift
  • Data streams
  • ensemble classification
  • K-means clustering
  • k-nearest neighbor classification
  • novel class
  • silhouette coefficient.

Fingerprint

Dive into the research topics of 'Classification and novel class detection in concept-drifting data streams under time constraints'. Together they form a unique fingerprint.

Cite this