Home

Public Notice

Patient Matching Algorithm Challenge Winners

Best “F-score” (a measure of accuracy that factors in both precision and recall):

  • First Place ($25,000): Vynca
  • Second Place ($20,000): PICSURE
  • Third Place ($15,000): Information Softworks

Best First Run ($5,000): Information Softworks

Best Recall ($5,000): PICSURE

Best Precision ($5,000): Ocuvera

Each winner employed widely different methods. PICSURE used an algorithm based on the Fellegi-Sunter (1969) method for probabilistic record matching and performed a significant amount of manual review. Vynca used a stacked model that combined the predictions of eight different models. They reported that they manually reviewed less than .001 percent of the records. Although Information Softworks also used a Fellegi-Sunter-based enterprise master patient index (EMPI) system with some additional tuning, they also reported extremely limited manual review.

 

Patient Matching Challenge

Patient Matching Algorithm Challenge

$75,000 in prizes

The goal of the Patient Matching Algorithm Challenge is to bring about greater transparency and data on the performance of existing patient matching algorithms, spur the adoption of performance metrics for patient data matching algorithm vendors, and positively impact other aspects of patient matching such as deduplication and linking to clinical data. Participants will be provided a data set and will have their answers evaluated and scored against a master key. Up to 6 cash prizes will be awarded with a total purse of up to $75,000.00.

 

Important Note

The Patient Matching Challenge opened on 6/12/17 at noon EST.

 

Challenge Timeline

  • Announcement of Challenge: April 28, 2017
  • Registration Period Begins May 10, 2017
  • Submission Period Begins: Upon availability of test data
  • Submission Period Ends: October 12, 2017 (at 9 am)
  • Winners Notified: 1 week from the end of submission period
  • Winners Announced: 1 week from winner notification date

 

Webinars

On May 10, May 17, and May 24 online webinars were held. The recorded webinar and PowerPoint slides can be found below.

Date: May 10, 2017

Recorded Webinar Link
PowerPoint Slides

Date: May 17, 2017

Recorded Webinar Link
PowerPoint Slides

Date: May 24, 2017

Recorded Webinar Link
PowerPoint Slides

 

Prizes

Highest F-Score

  • First Place: $25,000
  • Second Place: $20,000
  • Third Place $15,000

Best in Category Supplemental Prizes (1 prize for each category at $5,000):

  • Best precision (with Recall >= .9)
  • Best recall (with Precision >= .9)
  • Best first F-Score run

Total Prize Purse: Up to $75,000

View Challenge Details