Main menu
Home
Background
Call for participation
Databases
Reference systems
Scenarios
Access Control Scenario :
Mobile Scenario :
Final workshop
Participants
Search
Contact us
Reference Systems

In the framework of identity verification, it has been noticed that it is very difficult to compare two different methods from two different articles in the literature, even though they deal with the very same task. It raises a real problem when one wants to know if a new original method performs better than the current state of the art, for example. This can be explained by the fact that a lot of researchers have recorded their own test database and are the only one performing experiments on it, which are subsequently impossible to reproduce.

Reference systems bring an easy yet efficient answer to this problem. Since they are open-source and freely available for everybody, when publishing results on a specific database according to a specific protocol, experiments using the reference system can be added as a way of calibrating the difficulty of this particular task.

Furthermore, reference systems are made of replaceable modules what is also of great interest. Indeed, researchers often work on a specific part of the system and do not have the time nor the interest in building a complete system. In this way, a researcher could show the improvement of his new features extraction algorithm (for example) simply by replacing the corresponding module in the reference system and without having to bother about the pattern recognition algorithm.

To sum up, the main properties of a reference system are:

  • efficiency and robustness.
  • easy implementation: in general, a reference system is composed of four modules (pre-processing, features extraction, model building and matching).
  • absolute reproducibility: a reference system is accompanied with one publicly available database and evaluation protocols.
  • development as open-source software.

 

[ download ]