FpVTE is designed to assess the capability of fingerprint systems to meet requirements for both large-scale and small-scale real world applications. FpVTE 2003 consists of multiple tests performed with combinations of fingers (e.g., single fingers, two index fingers, four to ten fingers) and different types and qualities of operational fingerprints (e.g., flat livescan images from visa applicants, multi-finger slap livescan images from present-day booking or background check systems, or rolled and flat inked fingerprints from legacy criminal databases).
FpVTE 2003 will serve as part of NIST's statutory mandate under section 403c of the USA PATRIOT Act to certify those biometric technologies that may be used in U.S. VISIT (U.S. Visitor and Immigrant Status Indication Technology, formerly known as the US entry-exit system). In addition, FpVTE results may form the basis for the design and acquisition of other large-scale fingerprint identification systems.
FpVTE 2003 analysis and methodologies are built on the multi-agency Face Recognition Vendor Test (FRVT) 2002. See http://www.frvt.org
FpVTE 2003 evaluations will be conducted at the NIST facilities at Gaithersburg, MD, no earlier than September 29 and no later than December 31, 2003.
Test results will be made publicly available in a NIST report after the conclusion of the test.
Interested parties should send contact information (name, email, telephone, address) to FpVTE@nist.gov to receive email updates on the status of FpVTE 2003.
The deadline for registration for Participants was 12 August 2003.
In addition, companies, research organizations, and universities that have developed mature prototype or research fingerprint matching, verification, or identification systems are invited to participate. Makers of fingerprint screening or filtering systems are also encouraged to participate.
FpVTE is not evaluating image acquisition devices (fingerprint scanners).
FpVTE will be composed of three separate tests, the Small-Scale Test (SST), Medium-Scale Test (MST), and the Large-Scale Test (LST).
The SST and MST will evaluate matching accuracy using individual fingerprint images. This contrasts with the LST, which will evaluate matching accuracy using sets of fingerprint images.
In general, the SST and MST will be of interest to makers of single-stage matchers, verification or small-scale identification systems, or new algorithms. Makers of fingerprint screening or filtering systems are also encouraged to participate in the MST. In general, the LST will be of interest to AFIS vendors.
The tests are designed so that the SST is a subset of the MST. LST participants are encouraged to participate in the MST.
If space is available, FpVTE personnel may allow a participant to enter more than one system in the evaluation. A second system might be submitted to demonstrate capabilities that can be achieved only through distinct parameter settings, an alternate system configuration, or different algorithms.
Click here to jump to the Test Plan for more information about SST, MST, or LST.
Tentative Participants will be required to complete a System Throughput Questionnaire to qualify for the tests. This information will be used to determine the final test design. The size and structure of the test will be designed to optimize among competing analysis objectives, available resources, and the desire to include all qualified participants.
The System Throughput Questionnaire is located in the Participant Area, which is only available to registered Participants and Tentative Participants.
The results from each test are participant-defined measures of similarity, which for most systems means matcher scores. Note that the systems do not return Match and Non-match determinations. FpVTE analysis is based on distributions of similarity scores for mate vs. non-mate comparisons. Which fingerprints are mated is, of course, not revealed.
For the SST and MST tests, matchers compare individual fingerprint images, and return a similarity score for each comparison. When comparing one dataset of fingerprint images against another, a fully-populated matrix of similarity scores is generated. This fully-populated similarity matrix is what will be used to evaluate SST and MST Participants.
For the LST, matchers compare sets of fingerprint images (e.g. a 2-image, 4-image, or 10-image electronic "card" of fingerprint images). Since an AFIS generally filters out many non-mates, similarity scores may not be available for every comparison. Participants nevertheless will be expected to generate a fully-populated matrix of scores, but may, for instance, fill the majority of the matrix with a default value indicating no similarity. Participants are advised that the choice of true scores vs. default values will affect scoring, and are encouraged to fill the matrix as completely as possible with true scores.
Click here to jump to the Test Plan for more information about similarity scores and matrices.
Evaluations will be based on ROC (Receiver Operator Characteristic) or, equivalently, DET (Detection Error Trade-off) analysis based on similarity scores. Rank-based analysis will also be performed.
For SST, evaluations will focus on the relationship between TAR (True Accept Rate: TAR = 1 - FRR (False Reject Rate)) and FAR (False Accept Rate), when FAR is between 10-2 and 10-5.
For MST, evaluations will focus on the relationship between TAR and FAR when FAR is between 10-2 and 10-7. In addition, evaluations will focus on the relationship between TAR and FAR when TAR is between 98% and 99.99%.
For LST, the relationship between TAR and FAR will be analyzed, focusing solely on very low FAR values (down to about 10-8).
Since some algorithms do not operate equally well at a broad range of operating points, participants may submit two algorithms or tunings of an algorithm for the SST or MST, one optimized for low FAR and another for high TAR.
Individual flat (plain) fingerprints were collected using a variety of single-finger livescan devices.
Slap fingerprints were collected using multi-finger livescan devices, as well as inked fingerprint cards scanned using flatbed scanners.
Rolled fingerprints were collected using rolled livescan devices, as well as inked fingerprint cards scanned using flatbed scanners.
The fingerprints were collected by the following devices. The listing of makes and models does not imply a recommendation by NIST or FpVTE personnel, but simply recognizes the actual devices used by the variety of Federal agencies that contributed data to FpVTE.
Single-finger livescan devices:
The source (type of scanner) of each fingerprint will not be provided in the tests.
All images are WSQ compressed.
Note: The FBI’s Electronic Fingerprint Transmission Specification (EFTS) is based on ANSI/NIST. Fingerprint files that are EFTS compliant are necessarily ANSI/NIST compliant.
FpVTE 2003 Evaluation Datasets and data derived from the datasets shall not be retained in any way or form whatsoever by Participants after completion of the Evaluation.
Systems being evaluated in FpVTE 2003 shall not be accessible from outside the room in which the evaluation is being conducted. No system entered in the test can have or use any wireless networking equipment, modems, or access to the Internet.
FpVTE 2003 Sample Datasets and Software will be supplied to Participants to assist in preparing for FpVTE 2003. The FpVTE 2003 Sample Datasets are representative of the FpVTE 2003 Evaluation Datasets in format. Image quality, collection device and other characteristics may vary between the Sample and Evaluation Datasets.
After completion of the evaluations, the Government will combine all results into a Final Report. The FpVTE 2003 Final Report is expected to be released in early 2004. The FpVTE 2003 Final Report will contain, at a minimum, descriptive information concerning FpVTE 2003, descriptions of each experiment, evaluation results, and each Participant’s five-page system description document.
Participants will get an advance copy of the draft final report shortly before release and will be allowed to submit a position paper that will be included in the FpVTE 2003 Final Report when released.
The Final Report will be placed on the FpVTE 2003 website (http://FpVTE.nist.gov) when it has been completed and approved for public release. We will announce the release on the Biometrics Consortium listserv.