HAIDi blog

Publishing our work.

blog
2020-02-18

Humans vs. Machines: Accuracy of HAI Detection

When a computer makes an error, people tend to judge it harshly. No matter the complexity of the task, we usually start complaining about the software program or stop trusting the algorithm at all. But do we assess our mistakes in the same way? Can we even realize how many mistakes we make?

Let’s look at a complex task of identification of healthcare-associated infections (HAIs) during the point prevalence study conducted in 2017 (see the complete report by Suetens et al.). Each patient staying at that time in the hospital was assessed against the HAIs definitions proposed by the European Centre for Disease Prevention and Control (ECDC). This assessment was done by medical experts, i.e., members of infection prevention and control teams or hospital hygiene departments.

In some countries, national validation teams carried the validation study of the reported HAIs independently from the primary surveyors (i.e., acute care hospitals) to check the results. They re-examined 12,228 patient charts in total. The authors of the paper considered the validation teams’ results as a gold standard and they used it to assess the quality of the original survey results.

On average, experts found less than 70% of all HAIs during the point prevalence study. Besides that, 20% of reported HAIs were not identified correctly.

Suetens et al. state that the mean recall (sensitivity) of HAI detection was 69.4%. This means, that less than two-thirds of the HAI patients were discovered and reported by the reviewed hospitals. 30.6% of HAI patients were not reported at all! The mean precision of the original results was 79.7%, i.e., only 79.7% of the originally reported HAI patients truly had an HAI and the rest was reported falsely.

The poor validation results show, that even the highly qualified experts sometimes err. Therefore, we might be a little more cautious and tolerant next time we assess the performance of computers and automated algorithms. However, you may still be wondering who would win: humans or the machines? In our experience, nothing can beat human-machine cooperation.

latest blog posts

HAIDi Tackles…

The cases of antibiotic resistant bacteria continue to grow worldwide. While antibiotic resistance can occur naturally, misuse of antibiotics has accelerated the process. As more and more bacteria…
latest blog posts

6 Reasons Why Keywords…

Listening to the following words may become a little tiresome: "I've heard you have a solution which can read unstructured texts. Can you tell me which keywords do you use?" It was a reason why we…
latest blog posts

Highlights from recent…

Results of an European survey on healthcare-associated infections (HAIs) has been published recently. In 2016 and 2017, the European Centre for Disease Prevention and Control (ECDC) conducted two…