Skip to main content

Table 4 Diagnostic performance of the different included models in the systematic review of prediction models to diagnose COVID-19 from the start of the epidemic to March 2021

From: A systematic review of prediction models to diagnose COVID-19 in adults admitted to healthcare centers

Reference Classification measure Discrimination measure
Aldobyany [12] Score 4 sensitivity (Se), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) were 65.9, 49.1, 28.8, and 82.1%, respectively.
Score 5 sensitivity, specificity, positive predictive value, and negative predictive value are 64, 55.7, 31.1, and 83.2%, respectively.
The receiver operating characteristics ROC of COVID-19 respiratory triage score was above the line of no predictive value with an area under the ROC curve (AUROC) value of 0.60 (95% CI: 0.57–0.64).
Bar [13] Sensitivity, 97% (83–100%); specificity, 62% (50–74%); PPV, 54% (41–98%); and NPV, 98% (88–99%) AUROC for final logistic model: 0.82 (0.75–0.90).
Callejon-Leblic [17] Sensitivity = 72 (69–75) %
Specificity = 84 (82–87) %
PPV = 83 (80–85) %
NPV = 74 (71–77) %
AUROC = 0.78 (0.72–0.83)
Fink [18] Using a cut-off threshold of 2 for the risk score, the diagnostic prediction model has a sensitivity of 78.1% and specificity of 86.8%. At COVID-19 prevalence of 85%, the diagnostic prediction model has a PPV of 95.1% and NPV of 36.0%. At COVID-19 prevalence of 10%, the PPV falls to 28.1% and NPV rises to 96.5%. AUROC 0.8535 (95% CI (0.8121–0.8950). The optimism-corrected AUROC was 0.8465 (95% CI 0.7814–0.9038). The model performed comparably well for patients aged less than 80 years (AUROC 0.8736, 95% CI 0.8291–0.9181) and greater than 80 years (AUROC 0.8364, 95% CI 0.7492–0.9236).
Gupta-Wright [19] Score threshold (< 4)
Sensitivity = 26.6%
Specificity = 96.6%
PPV = 89%
NPV = 56%
Score threshold (> 9)
Sensitivity = 37%
Specificity = 96.1%
PPV = 90.1%
NPV = 61.2%
AUROC = 0.83 (0.82–0.85) for the model
AUROC = 0.83 (0.81–0.84) for the score
AUROC = 0.82 (0.80–0.84) for the bootstrapping
Huang [20] A cut-off value of 20: specificity: 86.6%; sensitivity: 81.3% A cutoff value of 20: AUROC was 0.921 (95%CI: 0.896–0.945, P < .01)
Kurstjens [21] According to cutoffs value:
2, Se: 98% (0.96–0.99), Sp: 42%(0.35–0.49), True/False Negative: 83/7
3, Se: 98% (0.95–0.99), Sp: 53%(0.46–0.60), True/False Negative: 105/10
4, Se: 96% (0.94–0.98), Sp: 63%(0.56–0.70), True/False Negative: 125/15
5, Se: 94% (0.91–0.96), Sp: 72%(0.66–0.78), True/False Negative: 144/25
9, Se: 78% (0.73–0.82), Sp: 89%(0.84–0.93), True/False Negative: 305/22
10, Se: 68% (0.63–0.72), Sp: 92%(0.87–0.95), True/False Negative: 267/17
11, Se: 56% (0.51–0.61), Sp: 95%(0.90–0.97), True/False Negative: 219/11
12, Se: 45% (0.40–0.50), Sp: 97%(0.94–0.99), True/False Negative: 177/6
Model population: AUROC: 0.94 (95% CI 0.91–0.96)
Validation population: 0.91 (95% CI 0.89–0.94)
McDonald [22] Logistic regression: Se:97 (83–100) %, Sp: 69(62–75) %, PPV: 29 (20–36) %, NPV: 99(96–100) %
Random forest: Se: 97 (83–100), Sp: 50 (43–57) %, PPV: 20 (14–28) %, NVP: 99 (95–100) %
XGBoost: Se: 97 (83–100) %, Sp: 54 (47–61) %, PPV: 22 (15–30) %
Logistic regression: AUROC = 0.89(0.84–0.94)
Random forest: AUROC = 0.86 (0.79–0.92)
XGBoost: AUROC =0.85 (0.79–0.91)
Nakakubo [23] Not reported Not reported
Plante [24] Score cutoff 1: Se: 95.9%, Sp: 41.7%, likelihood ratio: 0.099
Score cutoff 2: Se: 92.6%, Sp: 60.0%, likelihood ratio: 0.124
Score cutoff 5: Se: 85.5%, Sp: 78.5%, likelihood ratio: 0.185
Score cutoff 10: Se: 79.4%, Sp: 87.6%, likelihood ratio: 0.235
AUROC
Training: 0.91 (0.90–0.92)
External validation: 0.91 (0.90–0.92)
Sensitivity analysis 0.89 (0.88–0.90)
Sung [16] Development cohort (Risk score > =3)
Sensitivity = 85.1%
Specificity = 75%
PPV = 71.8%
NPV = 87%
Validation cohort
Sensitivity = 79.6%
Specificity = 70.9%
PPV = 60.9%
NPV = 85.9%
Development cohort
Model 1: AUROC = 0.87 (0.83–0.92), Model 2: AUROC = 0.87 (0.83–0.92), Model 3: AUROC = 0.87 (0.82–0.92)
Validation cohort
Model 1: AUROC = 0.85 (0.78–0.92), Model 2: AUROC = 0.83 (0.76–0.90), Model 3: AUROC = 0.85 (0.78–0.92)
Tordjman [14] PARIS score:
0: sensitivity = 100%, specificity = 0%
1: sensitivity = 100%, specificity = 28%
2: sensitivity = 99%, specificity = 53%
3: sensitivity = 92%, specificity = 72%
4: sensitivity =79%, specificity = 90%
5: sensitivity = 38%, specificity = 99%
Validation cohort: AUROC = 0.889, for score ≥ 4 points
Derivation cohort: AUROC = 0·921; STD = 0·027; CI = [0·867–0·974]
Vieceli [15] Score: 96% of sensitivity, 73.5% of specificity Before bootstrapping: AUROC of 0.847
(95% CI 0.77–0.92)
After bootstrapping: AUROC of 0.827 (95%
CI 0.75–0.90)