Challenge Leaderboard

Public Notice

Patient Matching Algorithm Challenge Winners

Best “F-score” (a measure of accuracy that factors in both precision and recall):

  • First Place ($25,000): Vynca
  • Second Place ($20,000): PICSURE
  • Third Place ($15,000): Information Softworks

Best First Run ($5,000): Information Softworks

Best Recall ($5,000): PICSURE

Best Precision ($5,000): Ocuvera

Each winner employed widely different methods. PICSURE used an algorithm based on the Fellegi-Sunter (1969) method for probabilistic record matching and performed a significant amount of manual review. Vynca used a stacked model that combined the predictions of eight different models. They reported that they manually reviewed less than .001 percent of the records. Although Information Softworks also used a Fellegi-Sunter-based enterprise master patient index (EMPI) system with some additional tuning, they also reported extremely limited manual review.

 

The Patient Matching Algorithm is now closed. The Patient Matching Algorithm Team will be contacting the winners individually. Thank you for your interest.

Leaderboard FScore

Place Team Fscore Version
1 PICSURE 0.97503429806 2
2 Vynca 0.975028236477 2
3 Information Softworks 0.974632204132 2
4 Shaken Not Stirred 0.974465523923 2
5 Matchmaker 0.974403492237 2
6 NCI AdvanceMed 0.974207202143 2
7 Ocuvera 0.972565940106 2
8 IBM Watson Health Cleveland 0.971877098028 2
9 Team F 0.969001071794 2
10 Mitch McBride 0.966267284171 2
11 Shindo 0.96542210778 2
12 SpectraMedix1 0.963362159001 2
13 MatchIT 0.961060313732 2
14 EMPIRollers 0.960007301571 2
15 BAH_PRIV 0.958882543085 2
16 Game Set Match 0.958823922933 2
17 wh0tever 0.957788556755 2
18 MPI20 0.9572497865 2
19 PM123 0.956011337887 2
20 Match Guru 0.954914730764 2

Top 5 - Leaderboard FirstRun

Place Team First Fscore Version
1 Information Softworks 0.973135465492 2
2 Vynca 0.971278461756 2
3 Ocuvera 0.963202520192 2
4 NCI AdvanceMed 0.960046381488 2
5 IBM Watson Health Cleveland 0.957284451675 2

Top 5 - Leaderboard Recall

Place Team Recall Version
1 PICSURE 0.972347027831 2
2 Vynca 0.966512590102 2
3 IBM Watson Health Cleveland 0.956314445486 2
4 Shaken Not Stirred 0.955134628438 2
5 Information Softworks 0.953453793193 2