On the validity of virtual subjects for multibiometric systems evaluation

Abstract: In this work, we address the question of the validity of using virtual subjects for multibiometric systems evaluation. Starting from a recent work by Poh and al. on fusion of several face and speech verification systems, we show that it is not sufficient in order to draw a general conclusion, to use a single performance measure and a protocol performing only one split of the database in evaluation and test subsets. We made experiments with the same scores but with an open-set protocol and a 2-fold Cross Validation procedure which show different behaviors between fusion results, on virtual and real subjects databases, in function of the considered experts combinations.