URLs in this document have been updated. Links enclosed in {curly brackets} have been changed. If a replacement link was located, the new URL was added and the link is active; if a new site could not be identified, the broken link was removed.

Open Access Citation Advantage: An Annotated Bibliography

A. Ben Wagner
Sciences Librarian
Science & Engineering Library
University at Buffalo
Buffalo, New York
abwagner@buffalo.edu

Introduction

This annotated bibliography lists studies and review articles that examine whether open access (OA) articles receive more citations than equivalent subscription; i.e., toll access (TA) articles. The bibliography is divided into three sections:

Review articles [5 reviews]
Studies showing an open access citation advantage (OACA) [39 articles]
Studies showing either no OACA effect or ascribing OACA to factors unrelated to OA publication [7 articles]

Scope and Methods

Scholarly material from the first known report of an open access citation advantage in 2001 by S. Lawrence up through mid-2009 has been included. In an attempt to be both current and comprehensive, this bibliography contains both peer-review articles, web reports, and other working documents and data analysis. These distinctions are made in the form of the citation and the text of the annotation. Obviously, peer-reviewed studies should be given the most weight. Editorials and letters to the editor generally have not been included.

The following databases were searched: Google Scholar, SciFinder (web version) including MEDLINE, Web of Science, Library Literature & Information Science FT, and Library, Information Science & Technology Abstracts with Full Text. Any article with the keywords 'open access' AND 'citation*' were retrieved, where '*' was the appropriate truncation symbol for a given database. The search results were cross-checked against an extensive, more general bibliography maintained on the web by S. Hitchcock and listed in Section A below.

The annotations focus on findings directly related to open access citation advantage (OACA) rather than attempting to itemize every finding in the study. The annotations should not be used as a substitute for the methodological details and full set of findings found in the full text of each article.

The reported OACA, expressed in percentage terms, is placed at the beginning of each annotation bolded and in brackets. Hence, [OACA=100%] indicates that the study found that the set of OA articles analyzed were cited twice as much (100%) more than the corresponding set of TA articles.

Three studies in the last section of the bibliography indicate OA articles are not cited more that TA articles and are labeled [OACA=No Effect], though one study analyzed the data only at a journal title level. The other four studies in the last section report a positive correlation between OA articles and number of citations, but dispute a cause and effect relationship, assigning the citation advantage to other factors. These studies are labeled [OACA=No causation].

What Is Open Access Publishing?

Articles may be open access by virtue of publication in an OA journal, deposition in an OA repository, or self-archived on a personal or departmental web page. Publishing in OA journals is referred to as Gold OA whereas repositories and self-archiving on transitory web sites are designated as Green OA. This terminology is used in a number of the studies listed below.

Unfortunately, OA proponents typically do not make a distinction between mounting items to a formal, managed repository versus mounting to a personal or departmental web site, referring to both processes as self-archiving/green OA. It is true that researchers usually initiate making their work open access. Hence, the use of 'self' in the term 'self-archiving' is appropriate. However, it is a disservice to archives and archivists to use the term 'archiving' when referring to placing documents on a transient personal or departmental web site. These locations are anything but an archive.

The Central Question: Correlation or Causation

It is interesting to note that no study has ever claimed that OA articles were cited less than TA articles. The research question still being debated is whether other factors explain the widely observed OACA rather than the mere fact an article is open access. In other words: Is the OACA merely a positive correlation between OA publication and number of citations, or is it, at least to some degree, a causation factor?

Other suggested causative factors include:

Number of co-authors leading to a higher chance that at least one author posts an OA version
Author self-selection of higher quality articles for OA (Quality or Selection bias)
Earlier dissemination via preprints/OA repositories (Early access bias)

Though this question is not settled, the bibliography cites a number of studies designed to test the hypothesis of confounding extraneous causes. It is clear that open access articles are downloaded far more than toll access articles. Studies indicate this download advantage is easily 100% over toll access articles. It seems unlikely such a large download advantage would not to some degree eventually influence the number of citations.

Studies typically show a 25-250% OACA or more. The higher end of that range might prove illusionary. However, even if the true OACA turns out to be only 10-15%, this would still be a major incentive for scholars to choose an open access publishing option. Note that many studies showed an OA advantage merely by reserving the right to mount their preprint or postprint to a repository or web site. Publication in an open access journal (Gold OA) apparently is not required to get a significant OA citation advantage.

This bibliography should provide information professionals with a significant selling point to convince their scholars to favor open access publishing. There is a slow, but growing trend in academia and among grant agencies to move away from citation metrics that report only gross averages such as the Thomson Scientific Journal Impact Factor towards metrics that report citations to an individual's work such as the h-index. The h-index proposed in 2005 by Jorge Hirsch, a physicist, reports the h number of articles cited at least h times. It is described in a news item in Physics World on August 16, 2005.

Section A: Reviews

Craig, I. D., Plume, A. M., McVeigh, M. E., Pringle, J., & Amin, M. 2007. Do open access articles have greater citation impact? A critical review of the literature. [Review]. Journal of Informetrics, 1(3), 239-248.: This methodological review of 26 papers is highly critical of studies showing any open access vs. toll access citation advantage. The authors conclude that only one study [Moed 2007] properly used a fixed time window and corrected for quality/selection and early view bias effects. When data is adjusted for these biases, the remaining open access citation advantage was reduced to 7%. This bibliography lists various responses to these criticisms. Note that all five authors work for major commercial publishers. This type of debate is exactly how scholarly research moves forward.
Harnad, S., & et. al. 2004. Gold and the green roads to open access. [Review]. Nature Web Focus. [Online]. Available: http://www.nature.com/nature/focus/accessdebate/21.html [Accessed: August 10, 2009].: An preliminary review of the major studies in this field. Updated by the 2008 Serials Review article, "The Access/impact problem and the green and the gold roads to Open Access: an update" also listed in this bibliography.
Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Gingras, Y., et al. 2008. The Access/impact problem and the green and gold roads to open access: an update [Review]. Serials Review, 34(1), 36-40.: This excellent review of the major studies in this field discusses citation impact of both the gold road (publish in an open access journal) and the green road (reserving right to place manuscript or article in an open access repository). This paper updates the 2004 Nature Web Focus document, "The Green and the gold roads to open access" which is also listed in this bibliography. The authors advocate that all research employers and funders mandate deposit of scholarly output in open access archives.
Hitchcock, S. Effect of open access and downloads (hits) on citation impact: a bibliography of studies (Open Citation Project) [Review]. Open Citation Project. [Online]. Available: http://opcit.eprints.org/oacitation-biblio.html [Accessed: August 12, 2009].: This is a frequently updated and extensively annotated chronological bibliography on the relationship between open access, downloads, and impact, including citation impact. This web resource has sections for latest additions, studies with original data, web tools for measuring impact, comparative reviews, and background articles. Hence, the scope is broader than my effort, though it was invaluable is cross-checking my independent literature searches.
Turk, N. 2008. Citation impact of open access journals [Review]. New Library World, 109(1/2), 65-74.: A very good, current review of the research in this area, focusing on library and information science journals (37 references). However, it does include some studies that do not directly compare OA to subscription-based articles, and the author draws only very general conclusions.

Section B: Studies Showing an Open Access Citation Advantage (OACA)

Antelman, K. 2004. Do open-access articles have a greater research impact? College & Research Libraries, 65(5), 372-382.: [OACA= 45-91%] Web of Science citation rates are enhanced for OA articles over non-OA articles by 91% for mathematics, 51% for electrical and electronic engineering, 86% for political science) and 45% for philosophy.
Bernius, S., & Hanauske, M. 2009. Open access to scientific literature: increasing citations as an incentive for authors to make their publications freely accessible. Paper presented at the 42nd Annual Hawaii International Conference on System Sciences, HICSS, January 5, 2009 - January 9, 2009. [Online]. Available: {http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4755635&tag=1} [Accessed: September 25, 2009].: [OACA= Plus] This paper focuses on developing a simulation model of the scientific citation network using vertices. The simulation results support the empirical data regarding the open access citation advantage. They indicate that if two authors produce articles of the same quality, the authors using OA is likely to receive more citations.
Brody, T., Harnad, S., & Carr, L. 2006. Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology, 57(8), 1060-1072.: The study analyzes how early, short-term web usage impact (downloads of preprints) predicts medium-term citation impact. Significant correlations between citation counts and downloading counts of arXiv preprints were shown in physics (0.462), mathematics (0.347), astrophysics (0.477) and condensed matter (0.330). Though it does not directly speak to open access citation advantage of published articles, it does indicate that higher downloads generally result in higher citation counts.
Brody, T. 2004. Citation analysis in the open access world. Southampton, UK: University of Southampton, School of Electronics and Computer Science. [Online]. Available: {http://eprints.ecs.soton.ac.uk/10000/} [Accessed: August 12, 2009].: [OACA= 80-200%] Consistent citation advantage shown for OA articles over Non-OA articles published in the same journal and year ranging from 80% to 200% across 12 years of articles in physics and mathematics based on preprints appearing in arXiv.org. A rise in downloads is shown to lead to a later rise in citations. No record could be found of the supposed forthcoming publication of this study in Interactive Media International, perhaps a defunct newsletter.
Brody, T. D. 2007. Evaluating research impact through open access to scholarly communications [PhD. Thesis, Philosophy]. University of Southampton, School of Electronics and Computer Science. [Online]. Available: {http://eprints.soton.ac.uk/263313/} [Accessed: August 10, 2009].: [OACA= 50-250%] Articles self-archived by authors receive between 50-250% more citations. Rapid pre-printing on the Web has dramatically reduced the peak citation rate from over a year to virtually instantaneously. In addition to discussing citation impact, the thesis describes a new web metric, download impact.
Cheng, W., & Ren, S. 2008. Evolution of open access publishing in Chinese scientific journals. Learned Publishing, 21, 140-152.: [OACA= 26-57% at journal title level] This study of 1,608 journals covered by the Chinese Science & Technology Journal Citation Reports found citation indicators (impact factor and immediacy index) of OA journals and delayed OA journals were found to be higher than those of non-OA journals in five areas (medicine, biology, agriculture, chemistry, and university journals). Calculations using the data provided indicate that OA journals had on average a 57% citation advantage over non-OA journals and that delayed OA journals had on average a 26% citation advantage over non-OA journals. No data was provided at the article level which makes it difficult to compare this study with most of the other research in this field.
Clauson, K. A., Veronin, M. A., Khanfar, N. M., & Lou, J. Q. 2008. Open access publishing for pharmacy-focused journals. American Journal of Health-System Pharmacy, 65(16), 1539-1544.: [OACA= Plus at journal level] A study of 317 pharmacy journals, of which only a small number were at least partially open access, indicated that journals publishing OA articles may be more likely to be cited than traditional journals. Note this comparison was done at the journal level, not the article level. However, it provides a small measure of support to the OA citation advantage argument.
Davis, P. M., & Fromerth, M. J. 2007. Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? Scientometrics, 71(2), 203-215.: [OACA= 35%] An analysis of 2,765 articles published in four math journals for 1997 to 2005 shows a 35% open access citation advantage for articles deposited in arXiv. Of the three non-exclusive postulates for explaining citation advantage, there was no empirical support for Early View, some inferential support for a quality differential caused by more highly-citable articles being deposited in arXiv, and little support for a universal Open Access explanation. The study also noted that arXiv preprints had a pronounced negative impact on downloads of the corresponding articles from the journal publisher's web site with 23% fewer downloads.
Davis, P. M. 2009. Author-choice open-access publishing in the biological and medical literature: a citation analysis. Journal of the American Society for Information Science and Technology, 60(1), 3-8.: [OACA= 17%] A study of 11 biological and medical journals from 2003-2007 that publish both open access and subscription access articles (author-choice model). The study indicates an average citation advantage for open access articles of 17%, a smaller effect than many other studies in this bibliography have reported. Given the focus on citation counts in research impact evaluations, this smaller effect would likely still be considered very significant to individual researchers. The authors conclude there is also strong evidence that the open-access advantage is declining by about 7% per year, from 32% in 2004 to 11% in 2007, though this is based on comparing actual vs. predicted citations via a complex model.
Davis, P. M., & Antelman, K. 2006. Do open-access articles really have a greater research impact? and Response to Philip Davis [by Antelman]. College & Research Libraries, 67(2), 105-105.: Two letters to the editor. Davis raises concerns about confounding factors like article duplication & self-promotion. The question of whether Adelman's data shows a correlation vs. causation of open access and citedness. Antelman's response to Davis' letter clarifies that the data show only a correlation and that confounding factors need to be more carefully explored, though some of those concerns appear to be overstated.
Evans, J. A., & Reimer, J. 2009. Open access and global participation in science. Science, 323(5917), 1025-1025.: [OACA= 8%] Although showing a more modest effect for open access advantage (~8% on average), this study clearly demonstrates that open access articles are much more likely to be cited in poorer countries. Hence, open access has great value in developing world participation in global science. This study has been reported by some as disputing the citation advantage of open access articles. In fact, depending on the discipline, the study shows citation advantages of up to 25% for open access. It also provides quantitative data supporting the widely held assumption that OA benefits scientists in poorer countries and those not associated with large institutions.
Eysenbach, G. 2006. Citation advantage of open access articles. PLoS Biology, 4(5), 692-698.: [OACA= 100%] Study of open access vs. non-open access articles in the same journal, Proceedings of the National Academy of Science. Open access articles twice as likely to be cited and were more heavily cited than non-OA articles. Though the research methodology itself eliminates some confounding factors such as journal impact factors, regression models adjusted for other factors such as number of authors, discipline, country, and author's lifetime publication count. Articles published immediately as open access on a journal web site have higher impact than self-archived or otherwise OA articles. The authors found strong evidence that, even in a journal that is widely available in research libraries, OA articles are more immediately recognized and cited by peers than non-OA articles published in the same journal.
Gargouri, Y., Harnad, S., & Hajjem, C. 2009. Impact Of open-access self-archiving mandate on citation advantage. [Online]. Available: http://users.ecs.soton.ac.uk/harnad/Temp/selfarchdraft1.doc [Accessed: August 12, 2009].: [OACA= Plus] Compared citation counts for open access articles where the author's institution mandated open access deposit vs. those that did not have such a mandate. If there is any self-selection (quality) bias whereby authors make their best work open access, it would be expected that citations to the set of non-mandated articles (voluntarily made open access) would be higher on average than the mandated set where authors are required to make all their work open access. The data showed no self-selection effect. An open access citation advantage was shown for both sets. A stepwise logistic regression showed that citation count is positively correlated with journal impact factor, date of publication (age), the number of references cited by the article/review articles, and the number of co-authors. Articles by U.S. based first authors and those from institutions with open access mandates also show a higher citation count than those that are not. The study concludes that the OA advantage is a statistically significant, independent positive increment. Data is based on 27,197 articles published between 2002-2008 in 1,984 journals.
Gentil-Beccot, A., Mele, S., & Brooks, T. C. 2009. Citing and reading behaviours in high-energy physics: how a community stopped worrying about journals and learned to love repositories [arXiv"cs/0906.5418]. [Online]. Available: http://arxiv.org/abs/0906.5418 [Accessed: September 10, 2009].: [OACA= 400%] Compares three sets of 286,180 high energy physics articles and preprints, excluding conference papers, which appeared from 1991 to 2007: 1) arXiv preprints never published; 2) published articles never posted to arXiv, and 3) published articles previously posted to arXiv. For year 2008 citation counts, Set 3 showed a citation advantage factor of 5; i.e. cited 5 times more than articles/preprints in sets 1 & 2. The study also shows that citations to arXiv preprints begins well before formal publication occurs, achieving 20% of their total number of citations that will collect at the end of the following 2 years. Non-arXiv articles on average never able to overcome this 20% head start. See also Stevan Harnad's "Commentary on: 'Citing and reading behaviours in high-energy physics" (2009).
Greyson, D., Morgan, S., Hanley, G., & Wahyuni, D. 2009. Open access archiving and article citations within health services and policy research. Journal of the Canadian Health Libraries Association (JCHLA), 30(2), 51-58.: [OACA=60%] 1,923 articles from four health care journals were analyzed via a two-stage approach that adjusted for potential data biases or confounders (number of authors, time since publication, journal, and article subject). The adjusted data still showed that OA archived articles were 60% more likely to be cited at least once and, once cited, were cited 29% more than non-OA articles.
Hajjem, C. 2007. Chawki Hajjem intérêts de recherché. [Online]. Available: http://www.crsc.uqam.ca/lab/chawki/ch.htm [Accessed: August 5, 2009].: [OACA= Plus] Reports the research interests and activities of Chawki Hajjem, a doctoral candidate at the University of Quebec at Montreal. Links are provided to several unpublished studies rich with data a) showing OA advantage in biology and social sciences and b) refuting claims that OA advantage is due only or largely to a quality/impact self-selection bias. Mr. Hajjem has co-authored published papers with Stevan Harnad, also of the University of Quebec and Tim Brody from the University of Southampton, UK listed elsewhere in this bibliography. The web page unfortunately was last updated in May 2007.
Hajjem, C., Gingras, Y., Brody, T., Carr, L., & Harnad, S. 2005. Open access to research increases citation impact. [Online]. Available: http://eprints.ecs.soton.ac.uk/11687/ [Accessed: October 3, 2006].: [OACA= 25-250%] Open access articles had a citation impact advantage varying from 25% to over 250% in four disciplines and 28 subspecialties. The mean advantage was 96% and the median, 73%. The study was based on about one million Web of Science articles in about 1,000 journals from 1992-2003
Hajjen, C., Harnad, S., & Gingras, Y. 2005. Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. Bulletin of the Technical Committee on Data Engineering (IEEE Computer Society), 28(4), 39-46.: [OACA= 25-250%] Studying 1.3 million articles across 12 years (1992-2003) in 10 disciplines, the OA articles received 25%-250% more citations that non-OA articles. Concludes that although causality (open access causes higher citation counts) can not be determined from this data, data from physics which with a very high percentage of open access articles makes it unlikely that the OA citation advantage is merely or mostly a self-selection (quality) bias.
Harnad, S. 2006. OA impact advantage = EA + (AA) + (QB) + QA + (CA) + UA. [Online]. Available: http://eprints.ecs.soton.ac.uk/12085/ [Accessed: July 30, 2009].: Defines six different possible categories of open access citation advantages: Early advantage, arXiv advantage, quality bias, quality advantage, competitive advantage, and usage advantage. Notes OA articles are downloaded and read three times as much as non-OA articles. There is also a sizeable correlation between early download counts and later citation counts. No original data is presented. The purpose of this brief paper is to present a framework to guide future research to isolate the exact causes of the widely reported citation advantage for open access articles and whether it is biased by confounding factors.
Harnad, S. 2007. Citation advantage for OA self-archiving is independent of journal impact factor, article age, and number of co-authors [arXiv:cs/0701136]. [Online]. Available: http://openaccess.eprints.org/index.php?/archives/2007/01/17.html [Accessed: August 10, 2009].: [OACA= Plus] The title pretty much says it all. This study is in response to Eysenbach's "Citation Advantage of Open Access Articles" published in PLoS Biology in 2006 which suggested that OA citation advantage might be an artifact of confounding factors such as article age, number of authors, subject matter, country, etc. Chawki Hajjem ran a multiple regression analysis that simultaneously tested article age, journal impact factor, and number of authors against OA self-archiving for over 440,000 articles in 576 biomedical journals across 11 years. The analysis shows that each of the four factors contributes an independent, statistically significant increment to the citation counts.
Harnad, S. 2006. The Self-archiving impact advantage: quality advantage or quality bias? [Online]. Available: http://eprints.ecs.soton.ac.uk/13193/1/kurtz-moed.html [Accessed: August 13, 2009].: A rejoinder to the studies by Kurtz and by Moed concluding that open access citation advantage is the result of confounding factors, especially Early Access (EA) and Quality Bias (QB). Harnad argues that findings pointing to QB could just as easily be interpreted as a Quality Advantage (QA); i.e., a tendency for higher quality articles to benefit more than lower quality articles from being self-archived. An outline of future research to resolve these questions is described.
Harnad, S. 2008. Confirmation bias and the open access advantage: some methodological suggestions for the Davis citation study [arXiv:cs/0808.3296]. [Online]. Available: http://arxiv.org/abs/0808.3296 [Accessed: July 7, 2009].: A very detailed review of the Davis 2008 study questioning the methods and some of the conclusions.
Harnad, S. 2009. Commentary on "Citing and reading behaviours in high-energy physics" (Gentil-Beccot et al 2009 arXiv:cs/0906.5418) [arXiv:cs/0906.5418]. [Online]. Available: http://arxiv.org/abs/0907.3154 [Accessed: August 20, 2009].: Comments on the conclusions of Gentil-Beccot et. al. (2009) preprint, especially that there is no advantage to publishing green OA (preprint/manuscript deposition by author) vs. gold OA (formal publication in an open access journal). Harnad notes that journals are still necessary for peer-review/certification, but it is not surprising that there is no different between citation impact of green vs. gold OA. The mechanism by which an article is open access is much less important that the fact that it is freely available to all readers.
Harnad, S., & Brody, T. 2004. Comparing the impact of open access (OA) vs. non-OA articles in the same journals. D-Lib Magazine, 10(6). [Online]. Available: http://www.dlib.org/dlib/june04/harnad/06harnad.html [Accessed: October 30, 2009].: [OACA= 150-480%] Report preliminary results of their analysis of Web of Science 1992-2001 citation data for physics showing an OA citation advantage of 2.5 to 5.8 ratio over non-OA, based on author-initiated archiving in an open access repository. However, the article title is slightly misleading as there is no head-to-head comparison of OA and non-OA articles appearing in the same, individual journal title. The introduction provides a good overview of studies in this area.
Harnad, S., & Hajjem, C. 2007. The Open access citation advantage: quality advantage or quality bias? [arXiv:cs/0701137]. [Online]. Available: http://arxiv.org/abs/cs/0701137/ [Accessed: August 5, 2009].: [OACA= Plus] Are self-archived, i.e., open access, articles more likely to be cited because authors self-select only their better articles ("Quality Bias," QB) or because articles that are self-archived are more likely to be cited ("Quality Advantage": QA)? The probable answer is both. One of Harnad's students has now analyzed over 100,000 articles from multiple fields, comparing self-selected self-archiving with mandated (by institutions) self-archiving to estimate the contributions of QB and QA to the Open Access Advantage. Both factors contribute, and the contribution of QA is greater. The paper also comments on the studies that have shown no OA advantage or ascribe it to variables other than simply the article being open access.
Henneken, E. A., Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Thompson, D., et al. 2006. Effect of E-printing on citation rates in astronomy and physics [arXiv cs.DL-0604061]. Journal of Electronic Publishing, 9(2). [Online]. Available: http://dx.doi.org/10.3998/3336451.0009.202 [Accessed: October 30, 2009].: [OACA= 100%] Using the NASA-Smithsonian Astrophysics Data System citation data, they confirm the findings from other studies that the average citation rate is about twice as high for articles that appear as OA preprints than those that do not.
Lawrence, S. 2001. Free online availability substantially increases a paper's impact. Nature, 411(6837), 521-521.: [OACA= 157%] Appears to be the earliest study of open access citation advantage. Lawrence studied 119,924 computer science articles and found a 157% increase (~2.5 times more likely) in the mean number of citations of OA articles over non-OA.
Lin, S. K. 2007. Non-open access and its adverse impact on Molecules. Molecules, 12(7), 1436-1437.: [OACA= 100-220%] During 2005-06, the two journals, Molecules and International Journal of Molecular Sciences (IJMS), published both open access and non-open access articles. Open access articles were cited on average twice as many times in Molecules and 3.2 as many times in IJMS as non-open access articles. The impact factor of the journals was also shown to be negatively influenced by the percentage of non-open access articles. As a result, these two journals have gone 100% open access.
Lin, S. K. 2009. Full open access journals have increased impact factors. Molecules, 14(6), 2254-2255.: [OACA= Plus] In an updated editorial to their 2007 article, the editors of four journals published by MDLPI provide charts showing increasing impact factors from 2005-2008 for each journal after going fully open access in 2007. Since journal impact factors depend directly on the average number of citations per article, this provides further support to the open access citation advantage.
Metcalfe, T. S. 2005. Rise and citation impact of astro-ph in major journals [arXiv:astro-ph-0503519]. Bulletin of the American Astronomical Society, 37, 555-557.: [OACA= 100%] A study of articles from 13 major astronomy journals indicates that papers that are posted to arXiv astro-ph are cited about twice as often as papers that are not posted.
Metcalfe, T. S. 2006. Citation impact of digital preprint archives for solar physics papers [arXiv astro-ph-0607079]. Solar Physics, 239, 549-553.: [OACA= 100%] Solar physics papers that were posted to one of two digital preprint archives are typically cited twice as often as papers that are not posted. The arXiv.org/astro-ph and the Montana State University Solar Physics E-Print Archive were independently compared. Two sets of documents were studied: articles published in Solar Physics and conference papers published in a 2003 International Astronomical Union proceedings volume. The study suffers somewhat from the small number of open access papers (33) vs. the number of toll-access papers (308), due to the slow adoption by the solar physics community of digital preprint archives. Still the reported open access citation advantage is statistically significant.
Norris, M., Oppenheim, C., & Rowland, F. 2008. The Citation advantage of open-access articles. Journal of the American Society for Information Science and Technology, 59(12), 1963-1972.: [OACA= 57%] A study of four subject areas (ecology, applied mathematics, sociology, and economics) showed an average of 57% more citations for journal articles that have an open access version vs. those that are exclusively toll access (subscription-based). Interestingly, sociology had the highest citation advantage, but the lowest number of OA articles while ecology had the highest citation counts for OA articles, but the smallest citation advantage. Tests of correlation between OA status and a number of variables such as number of authors was generally found to be weak or inconsistent.
Norris, M., Oppenheim, C., & Rowland, F. 2008. Open access citation rates and developing countries. Paper presented at the ELPUB 2008: 12th International Conference on Electronic Publishing. [Online]. Available: http://elpub.scix.net/cgi-bin/works/Show?_id=335_elpub2008 [Accessed: September 29, 2009].: One proposed cause for OA citation advantage is that authors from developing countries with fewer subscription resources disproportionally cite open access articles. Using mathematics articles, the study shows that the majority of citations were done by Americans to American authors. The overwhelming majority of articles are both authored and cited from high income countries. However, the small number of citations from authors in developing countries does show a higher proportion of citations to OA articles than high income countries. However, the authors note the evidence for this conclusion is mixed and may well point toward a complex picture of citation behavior.
Piwowar, H. A., Day, R. S., & Fridsma, D. B. 2007. Sharing detailed research data is associated with increased citation rate. PLoS ONE, 2(3), e308.: [OACA= 69% for OA data sets] This is the first study to show a correlation between open access data sets and citation impact for 85 cancer microarray clinical trial publications. Open access data resulted in a 69% increase in citations.
Sahu, D. K., Gogtay, N. J., & Bavdekar, S. B. 2005. Effect of open access on citation rates for a small biomedical journal. Paper presented at the Fifth International Congress on Peer Review and Biomedical Publication. [Online]. Available: http://openmed.nic.in/1174/ [Accessed: September 15, 2009].: [OACA= 365%] The Journal of Postgraduate Medicine showed a remarkable 365% increase in citations per article after switching to open access in 2001.
Schwarz, G. J., & Kennicutt, R. C., Jr. . 2004. Demographic and citation trends in astrophysical journal papers and preprints [arXiv:astro-ph/0411275]. Bulletin of the American Astronomical Society, 36(5), 1654-1663.: [OACA= 100%] An analysis of data from the Astrophysical Data System, the American Astronomical Society, and the astro-ph section of arXiv.org shows that, on average, Astrophysical Journal and conference papers posted on astro-ph are cited twice as frequently as documents not so posted. The paper also makes a number of interesting observations about preprint posting habits, information discovery by researchers, and author demographics.
Sotudeh, H., & Horri, A. 2007. The Citation performance of open access journals: A Disciplinary investigation of citation distribution models. [Feature]. Journal of the American Society for Information Science and Technology, 58(13), 2145-2156.: [OACA= ??] This paper uses fairly high powered statistical analysis to show on a general, global basis that the citation performance of the open access journal science system is similar to that of the entire science system. Almost half the fields studied exceed or equal their expected citation rates compared to non-OA journals. This is a difficult article to interpret. Even the conclusions are not stated as simply as one would wish. Any study based on calculated expected citation rates involves using a complex model, which by definition involves a number of assumptions. The authors do state that citations to OA articles increase at a faster rate relative to the increase in the publication of the OA articles themselves.
Zhang, Y. 2006. The Effect of open access on citation impact: a comparison study based on web citation analysis. [Feature]. Libri, 56(3), 145-156.: [OACA= 100%] This study compares web citations from peer-reviewed articles and informal web sources to an open access communications journal and a subscription communications journal. Both titles are peer-reviewed journals published quarterly with similar scope, number of articles, and journal impact factors. The OA articles received twice as many web citations than the subscription articles. Even when limited to citations from peer-reviewed articles, OA articles still received a significantly higher rate of citations. The open access articles also received a greater percentage of web citations from developing countries.

Section C: Studies Showing Either No OACA Effect or Ascribing OACA to Factors Unrelated to OA Publication

Davis, P. M., Lewenstein, B. V., Simon, D. H., Booth, J. G., & Connolly, M. J. L. 2008. Open access publishing, article downloads, and citations: randomised controlled trial. British Medical Journal, 337(7665), a568.: [OACA= No effect] Articles assigned to open access were associated with 89% more full-text downloads and 23% more unique visitors than subscription access articles in the first six months after publication. Open access articles were no more likely to be cited than subscription access articles in the first year after publication. Many reviewers believe the one year citation window is far too short. It also is counterintuitive that 89% more full-text downloads would not eventually result in at least some measurable citation advantage. The study examined 1,619 articles from 11 journals published by the American Physiological Society.
Gaule, P., & Maystre, N. 2008. Getting cited: does open access help? (No. CEMI-Working Paper-2008-007): Ecole Polytechnique Federale de Lausanne. [Online]. Available: { http://dx.doi.org/10.1016/j.respol.2011.05.025} [Accessed: March 8, 2010].: [OACA= No causation] Based on a sample of 4,388 biology papers published from May 2004 to March 2006 in a single journal, Proceedings of the National Academy of Sciences (PNAS). Using an instrumental variable approach, the study found no significant effect of open access. Authors believe self-selection of higher quality articles into open access explains at least part of the observed open access citation advantage. The authors seem to presuppose that open access is relatively more attractive to authors of higher quality papers, though the basis for opinion is less than clear.
Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Henneken, E., et al. 2005. Effect of use and access on citations. Information Processing & Management, 41(6), 1395-1402.: [OACA= No causation] One of the earliest papers attempting to sort out the three non-exclusive postulates: open access (OA), early view (EA), and self-selection/quality (SB). A more technical paper discussing individual factors and causes of why Internet-posted articles are more heavily cited than those that are not. Uses data from arXiv and the NASA Astrophysics Data System. This paper requires a fairly high degree of comfort with statistical methods, but the author concludes that there is a strong EA and SB effect and no indication of an OA effect; i.e., no open access citation advantage in astrophysics.
Kurtz, M. J., & Henneken, E. A. 2007. Open access does not increase citations for research articles from The Astrophysical Journal (arXiv:0709.0896v1 [cs.DL]). [Online]. Available: http://arxiv.org/abs/0709.0896 [Accessed: August 15, 2009].: [OACA= No causation] As the title notes, this article studies a single journal and claims that the OA citation advantage is due entirely to the nature and timing of the deposited papers. The authors conclude categorically that there is no advantage independent of the Early Access and Quality Bias
Lansingh, V. C., & Carter, M. J. 2009. Does open access in ophthalmology affect how articles are subsequently cited in research? Ophthalmology, 116(8), 1425-1431.: [OACA= No effect] Based on 480 articles in ophthalmology, a univariate general linear model analysis showed that OA was not a significant factor that explained citation data. Author number, country, subject area, language, and funding were the statistically significant variables. Hence, no open access advantage was demonstrated for this field.
Moed, H. F. 2007. The Effect of "open access" on citation impact: an analysis of ArXiv's condensed matter section. [Feature]. Journal of the American Society for Information Science and Technology, 58(13), 2047-2054.: [OACA= No causation] The study was designed to measure a quality bias (hypothesis: better articles more likely to be open access) and early view bias (preprints often available for open access articles). Citation impact differentials of individual authors and corrections for co-authorship were calculated in order to measure quality bias. The author concludes strong evidence for both biases and no sign of a general open access advantage. As such, it puts this article at direct odds with a vast majority of the other studies cited here. Is it possible that these attempts to adjust for biases introduced their own bias? Further research will ultimately decide that question. The study focused on an individual sub-discipline and a single repository where a vast majority of articles are published as preprints regardless of nature of their final publication.
Testa, J., & McVeigh, M. E. 2004. Impact of open access journals: a citation study from Thomson ISI. [Online]. Available: {http://thomsonscientific.jp/event/oal/impact-oa-journals.pdf} [Accessed: August 16, 2009].: [OACA= No effect at journal level] Notes that Web of Science in 2004 covers nearly 500 open access peer-reviewed journals. Found there was "no discernable difference in terms of citation impact of frequency with which the journal is cited" between the OA journals and non-OA journals they covered. This study speaks directly to the charge that OA journals are uniformly lower quality/impact compared to subscription journals. It does not speak directly to open access citation advantage or lack thereof at an article level.

Contents

Previous	Contents		Next
Issues in Science and Technology Librarianship		Winter 2010
DOI: 10.5062/F4Q81B0W