1. ¹Department of Mathematics, Imperial College, London, UK
2. ²Institute for Mathematical Sciences, Imperial College, London, UK

Correspondence: DJ, Hand, Department of Mathematics, South Kensington Campus, Imperial College, London SW7 2AZ, UK. E-mail: d.j.hand@imperial.ac.uk

Received November 2006; Accepted February 2007; Published online 9 May 2007.

Abstract

In predictive data mining, algorithms will be both optimized and compared using a measure of predictive performance. Different measures will yield different results, and it follows that it is crucial to match the measure to the true objectives. In this paper, we explore the desirable characteristics of measures for constructing and evaluating tools for mining plastic card data to detect fraud. We define two measures, one based on minimizing the overall cost to the card company, and the other based on minimizing the amount of fraud given the maximum number of investigations the card company can afford to make. We also describe a plot, analogous to the standard ROC, for displaying the performance trace of an algorithm as the relative costs of the two different kinds of misclassification—classing a fraudulent transaction as legitimate or vice versa—are varied.