Date of this Version

3-1-2017

Document Type

Journal Article

Publication Details

Accepted version

Albarqouni, L. N., López-López, J. A., & Higgins, J. P. T. (2017). Indirect evidence of reporting biases was found in a survey of medical research studies. Journal of Clinical Epidemiology, 83, 57-64. doi:10.1016/j.jclinepi.2016.11.013

Access the journal

Copyright © 2017 Elsevier Inc

Distribution License

Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

ISSN

1878-5921

Abstract

OBJECTIVES:

To explore indirect evidence of reporting biases by examining the distribution of P-values reported in published medical articles and to compare P-values distributions across different contexts.

STUDY DESIGN AND SETTING:

We selected a random sample (N = 1,500) of articles published in PubMed in March 2014. We extracted information on study type, design, medical discipline, and P-values for the first reported outcome and primary outcome (if specified) from each article. We plotted the P-values transformed to the z-score scale and used caliper tests to investigate threshold effects.

RESULTS:

Out of the 1,500 randomly selected records, 758 (50.5%) were included. We retrieved or calculated 758 P-values for first reported outcomes and 389 for primary outcomes (specified in only 51% of included studies). The first reported and the primary outcome differed in 28% (110/389) of the included studies. The distributions of P-values for first reported outcomes and primary outcomes showed a notable discontinuity at the common thresholds of statistical significance (P-value = 0.05 and P-value = 0.01). We also found marked discontinuities in the distributions of z-scores across various medical disciplines, study designs, and types.

CONCLUSION:

Reporting biases are still common in medical research. We discuss their implications, strategies to detect them, and recommended practices to avoid them.

Available for download on Thursday, January 11, 2018

Share

COinS
 

This document has been peer reviewed.