Methodological and Statistical Issues in Diagnostic Research

Diagnostic researches are among interesting field of clinical researches. However, methodological and statistical issues in such researches are not being considered appropriately. Diagnostic value should be considered as diagnostic accuracy (validity) and diagnostic precision (reliability or agreement). In case of binary variable, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), likelihood ratio positive (LR+), likelihood ratio negative (LR-) as well as odds ratio (ratio of true to false results) are the most appropriate estimates to evaluate validity of a test compared to a gold standard. Therefore, it is better to report all these validity estimates together. Otherwise, final interpretation will be confusing. Moreover, it is important to know that for clinical purposes, reporting diagnostic added value should be considered using receiver operating characteristic (ROC) curve because all the above mentioned validity estimates can be acceptable while diagnostic added value may be clinically negligible. Regarding quantitative variables, Interclass correlation coefficient (Pearson r or spearman rho) can be considered as an appropriate statistical test to assess validity [1-6].

Reliability (precision or agreement) as a different methodological issue of the diagnostic value should also be assessed using appropriate estimate. For qualitative variables, weighted kappa should be used with caution. Two important weaknesses of Cohen’s kappa to assess agreement of a qualitative variable are as follows. First, it depends on the prevalence in each category, which means it can be possible to have different k values having the same percentage for both concordant and discordant cells. Table 1 shows that in both (a) and (b) situations, the prevalence of concordant cells are 90% and of discordant cells, 10%; however, we get different kappa values (0.44 as moderate and 0.80 as very good, respectively). Kappa value also depends on the number of categories. In such a situation, a weighted kappa is a preferable test, giving an unbiased result. Finally, the P value or 95% CI is not reported for a weighted kappa in reliability analysis, because statistically significant does not necessarily means clinically important. Regarding quantitative variables, Intraclass correlation coefficient (ICCC) agreement single measure and Bland Altman plot can be considered as appropriate tests to assess reliability [7-11].

Table 1: Comparison of Two Observers’ Diagnoses with Different Prevalence in the Two Categories^a`. Author’s own hypothetical data to demonstrate the limitation of the kappa value to assess agreement.

In this editorial, I discuss the methodological and statistical issues in diagnostic researches. Therefore, conclusion of a diagnostic research should be supported by the above mentioned statistical and methodological issues. Otherwise, misdiagnosis and mismanagement of the patients cannot be avoided.