Improving the Prediction Cost of Drift Handling Algorithms by Abstaining

Abstract : The problem considered in this paper is regression with a constraint on the precision of each prediction in the framework of data streams subject to concept drifts (when the hidden distribution which generates the observations can change over time). Concept drifts can diminish the reliability of the predictions over time and it might not be possible to output a prediction which satisfies the constraints on the precision. In this case, we claim that if the costs associated with a good and with a bad prediction are known beforehand, the overall prediction cost can be improved by allowing the regressor to abstain. To this end, we propose a generic method, compatible with any regressor, which uses an ensemble of reliability estimators to estimate whether the constraints on the precision of a given prediction can be met or not. In the later case, the regressor is allowed to abstain. Empirical results on 30 datasets including different types of drifts back our claim.
Document type :
Conference papers
Complete list of metadatas

Cited literature [22 references]  Display  Hide  Download

https://hal.sorbonne-universite.fr/hal-01429068
Contributor : Pierre-Xavier Loeffel <>
Submitted on : Friday, January 6, 2017 - 6:03:52 PM
Last modification on : Wednesday, May 15, 2019 - 3:47:46 AM
Long-term archiving on : Friday, April 7, 2017 - 5:19:31 PM

File

PID4471505.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01429068, version 1

Citation

Pierre-Xavier Loeffel, Vincent Lemaire, Christophe Marsala, Marcin Detyniecki. Improving the Prediction Cost of Drift Handling Algorithms by Abstaining. IEEE International Conference on Data Mining (ICDM 2016), Dec 2016, Barcelone, Spain. ⟨hal-01429068⟩

Share

Metrics

Record views

600

Files downloads

250