Date of this Version


Document Type

Journal Article

Publication Details

Accepted version

Albarqouni, L. N., López-López, J. A., & Higgins, J. P. T. (2017). Indirect evidence of reporting biases was found in a survey of medical research studies. Journal of Clinical Epidemiology, 83, 57-64. doi:10.1016/j.jclinepi.2016.11.013

Access the journal

Copyright © 2017 Elsevier Inc

Distribution License

Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.





To explore indirect evidence of reporting biases by examining the distribution of P-values reported in published medical articles and to compare P-values distributions across different contexts.


We selected a random sample (N = 1,500) of articles published in PubMed in March 2014. We extracted information on study type, design, medical discipline, and P-values for the first reported outcome and primary outcome (if specified) from each article. We plotted the P-values transformed to the z-score scale and used caliper tests to investigate threshold effects.


Out of the 1,500 randomly selected records, 758 (50.5%) were included. We retrieved or calculated 758 P-values for first reported outcomes and 389 for primary outcomes (specified in only 51% of included studies). The first reported and the primary outcome differed in 28% (110/389) of the included studies. The distributions of P-values for first reported outcomes and primary outcomes showed a notable discontinuity at the common thresholds of statistical significance (P-value = 0.05 and P-value = 0.01). We also found marked discontinuities in the distributions of z-scores across various medical disciplines, study designs, and types.


Reporting biases are still common in medical research. We discuss their implications, strategies to detect them, and recommended practices to avoid them.



This document has been peer reviewed.


To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.