Spike Detection: Inter-reader agreement and a statistical Turing test on a large data set

Publication date: Available online 14 November 2016
Source:Clinical Neurophysiology
Author(s): Mark L. Scheuer, Anto Bagic, Scott B. Wilson
ObjectiveCompare the spike detection performance of three skilled humans and three computer algorithms.Methods40 prolonged EEGs, 35 containing reported spikes, were evaluated. Spikes and sharp waves were marked by the humans and algorithms. Pairwise sensitivity and false positive rates were calculated for each human-human and algorithm-human pair. Differences in human pairwise performance were calculated and compared to the range of algorithm versus human performance differences as a type of statistical Turing test.Results5,474 individual spike events were marked by the humans. Mean, pairwise human sensitivities and false positive rates were 40.0%, 42.1%, and 51.5%, and 0.80, 0.97, and 1.99/min. Only the Persyst 13 (P13) algorithm was comparable to humans – 43.9% and 1.65/min. Evaluation of pairwise differences in sensitivity and false positive rate demonstrated that P13 met statistical noninferiority criteria compared to the humans.ConclusionHumans had only a fair level of agreement in spike marking. The P13 algorithm was statistically noninferior to the humans.SignificanceThis was the first time that a spike detection algorithm and humans performed similarly. The performance comparison methodology utilized here is generally applicable to problems in which skilled human performance is the desired standard and no external gold standard exists.



from Physiology via xlomafota13 on Inoreader http://ift.tt/2g9qcK5
via IFTTT