Original Articles

Guidelines of the minimum sample size requirements for Kappa agreement test


Abstract


Background and aims:

    To estimate sample size for Cohen’s kappa agreement test can be challenging especially when it is expected that the true marginal rating frequencies are not the same. This study aims to present the tables that display a minimum sample size determination for an agreement test when certain assumptions are hold.

Method:

    We adopted the sample size formula provided by Flack and colleagues (1988) to calculate the minimum sample sizes required using PASS software. The power is pre-specified to be at least 80% and the alpha to be less than 0.05. The effect sizes were derived from several pre-specified estimates such as the pattern of the true marginal rating frequencies and the difference between the two kappa coefficients in the hypothesis testing.

Results:

    When the true marginal rating frequencies are the same, the minimum sample size determination can range from 2 to 698 depending on the actual value of the effect size. When the true marginal rating frequencies are not the same, then the majority of the minimum sample size required for this condition is more than double than that required sample size when the true marginal rating frequencies are the same.

Conclusion:

  Concerning that the sample size formula could produce a very extreme small sample size, therefore, the determination of K1 and K2 should be based on reasonable estimates. We recommend for all sample size determinations for Cohen’s kappa agreement test, the true marginal rating frequencies can be assumed the same. Otherwise, it will be necessary to multiple the estimated minimum sample size by two to accommodate if the true marginal rating frequencies is not the same.




DOI: http://dx.doi.org/10.2427/12267

References



Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.


Copyright (c) 2013 Epidemiology, Biostatistics and Public Health

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

EBPH Epidemiology, Biostatistics and Public Health | ISSN 2282-0930

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.