Previous Contents Next
Issues in Science and Technology Librarianship
Winter 2016
DOI:10.5062/F4XD0ZP5

Short Communications

Project Bibliographies: Tracking the Expansion of Knowledge Using JPL Project Publications

Ann Coppin
Jet Propulsion Laboratory (Retired)
California Institute of Technology
Pasadena, California
coppin@pacbell.net

Abstract

The Jet Propulsion Laboratory (JPL) Library defines a project bibliography as a bibliography of publicly available publications relating to a specific JPL instrument or mission. These bibliographies may be used to share information between distant project team members, as part of the required Education and Public Outreach effort, or as part of required reports to management or sponsors. While some bibliographies may be done by the mission's team members, the JPL Library staff has been involved in a number of project bibliographies over the years. The JPL Library currently has 10 ongoing project bibliographies. Basic requirements and search terms are developed by discussions with the project representative. For most of our project bibliographies this means looking for the project/mission name in the title, abstract, or keyword fields in commercial databases. We provide regular updates to the project. Metrics compiled from the comprehensive bibliographies have been particularly useful to projects. Both metrics and the publication lists support the contribution by JPL missions to scientific knowledge.

Introduction

The Jet Propulsion Laboratory (JPL) is a not-for-profit, federally funded research and development center (FFRDC) operated by The California Institute of Technology for the National Aeronautics and Space Administration (NASA) under a prime contract. JPL's primary mission is focused on the development and operation of robotic spacecraft to explore the Earth, our solar system, and the universe. JPL also conducts a wide range of research to advance the capabilities of future missions. JPL has always had a library. It can be categorized as a special, technical library. Despite all the changes to the field of special librarianship, there continue to be opportunities for librarians to provide ongoing bibliographies to support their parent organizations.

My experience with project bibliographies started with one after another literature search request which became ongoing bibliographies. My bibliography construction "process" started with an individual literature search using terms for the subject requested. Over time, "standardization" of the process occurred, partly as a result of similar requirements, and partly as a result of the bibliographic management tool being used.

The bibliometrics I provide first started with a request by one project for the number of publications per year. When I showed other people the chart for the first request, the other projects requested charts too. Through mymembership in the Special Libraries Association (SLA) Physics, Astronomy, and Mathematics Division I became aware of what astronomy librarians were doing with bibliographies and bibliometrics. Observatory libraries have documented their efforts (Lagerstrom et al. 2012; Accomazzi et al. 2012) and participated in the development of the International Astronomical Union's 2013 "Best Practices for Creating a Telescope Bibliography." Bibliometrics can support the impact of instruments, or data can be compiled from bibliographies (Savaglio and Grothkopf 2013; Rots et al. 2012), or from specific bibliometric studies of publications (Major 2011; Taskin and Aydinoglu 2015). While astronomy authors may use a common format for articles and a relatively limited set of journals, the article formats by authors in other areas may not be as standardized. For instance, articles using data from an instrument flying on a satellite or spacecraft may appear in a wide variety of publications, and the mention of the instrument is not standardized to one specific section of the article.

JPL Bibliographies

The JPL Library defines a project bibliography as a bibliography of publicly available publications relating to a specific JPL instrument or mission. There are different initial objectives for these project bibliographies. When a JPL project is in the operational phase, the teams frequently want to track publications using the science generated by the instrument/mission. A publication list may be a part of the required Education and Public Outreach Plan for the project. Publication lists are also useful in sharing information between geographically dispersed science team members, and in showing how others are using the data from the mission. Metrics may be compiled from the bibliography to support the benefits of the mission/instrument. These metrics may be used in regular reports to NASA Headquarters, or in Senior Review requests for additional funding. In the Senior Review, publication information helps support recent scientific advances of the mission, accomplishments with respect to mission objectives, or Education and Public Outreach programs (Seal and Manor-Chapman 2012). Peer reviewed article counts are considered an indication of the scientific merit of a flight project. When a project closes, the team may want a comprehensive bibliography as part of the final documentation.

The JPL Library's involvement in project bibliographies has evolved over the years. In 2003 a JPL reference librarian was asked to compile a bibliography for the Galileo Mission final report. In 2004, shortly after I joined the JPL Library, I was asked to find all published articles with the term "AIRS" or "Atmospheric Infrared Sounder" in the databases to which the JPL Library subscribed. Then a year later I was asked to update the list. The AIRS Project participants thought they needed only to search Compendex and download results into an Excel file to maintain accurate publication information for AIRS papers since 2000. They also wanted a publication list for their public web site. The project member doing the Compendex search found it time intensive to get the documents into Excel. Discussions with the AIRS project members about how to find and track AIRS articles resulted in my providing regular updates from all databases and instructions for them on using EndNote, which is used to compile the bibliography.

I received a request from a JPL researcher for a bibliography for the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) project as part of the support for the ASTER Senior Review he was preparing. Shortly after that was compiled, an ASTER science team member asked me for information about searching databases for mission publications. This discussion evolved into another ongoing bibliography.

The Cassini Mission team felt they were not getting published article information from science team members in a timely manner and also wanted to know about non-team member publications. They inquired of a contact they had in the JPL Library about what could be done and were referred to me. The result for me was another ongoing project bibliography. Additional project bibliographies started in similar ways or occasionally because someone heard about what I was doing for a specific project.

A good project bibliography at a minimum will include published scientific papers using data derived from the mission. These papers will be authored by both the science team members or by other scientists from around the world. Depending upon the requirement of the mission, the bibliography may be limited to journal publications or include conference publications, reports, etc. A comprehensive project bibliography is not limited to well-known journals from major publishers, such as those indexed by Web of Science, but includes articles from more specialized or less well-known journals, such as those included in the GeoRef database. The bibliography should be done in a way that is consistent and reproducible. Using the same search str ategy in the same databases over time means that someone else can find the same publications. Also, it means that within the bibliography comparisons can be made over time.The approach used in searching for mission papers can have a great impact on the number of papers identified. The compilation of a project bibliography may be done by the mission's science team or other support personnel. Self-reporting of publications authored by team members is a frequent starting point for compiling project bibliographies. Comments about this methodology from team personnel who have been responsible for tracking self-reported articles include:

Also observed for team-reported articles have been the submission of draft articles with titles different from the final publication and a lack of final publication information being provided when publication occurs. When science teams are larger and spread around the world, self-reporting may become insufficient.

Another way to compile a bibliography is assigning it to a project member. Beyond the self-reported articles, this person may just search one database, such as Compendex or Web of Science, with they are familiar. Or they may search only the Internet in addition to the self-reported articles. Common concerns are a lack of knowledge about how to most effectively search for publications, the time required for this effort, and differing criteria used for including publications into the bibliography. Also, the result may not be easily reproducible. At least one science team that does the compilation itself searches the Web of Science with a broad, general term and inspects the resulting list to remove papers that are not obviously using the mission's data. The text of the remaining papers are then individually examined for discussions and figures that use mission data. This time consuming approach results in more articles selected than if the search is done by specific keywords. One perceived advantage of this approach is the science team member has great familiarity of the spacecraft and its instruments, and they can easily determine relevancy.

Another option is for the mission to request the organization's library staff to compile the bibliography. This taps librarians' search expertise, guided by the project's subject matter expertise, and can typically yield the most thorough and representative results. The JPL Library staff has been involved with 13 project bibliographies in the past twelve years, of which ten are currently ongoing. At various times four of the library's reference librarians have been involved with one or more bibliographies. One long-standing bibliography is now being updated by the third librarian to be involved. Benefits provided by the library's compilation of the project bibliographies include:

When a request is made by a representative of an instrument/mission team I have a detailed discussion with them of what is wanted. This discussion usually partly involves my educating the scientist about literature searching of commercial databases. One key point I make is that I am searching for terms in the title, abstract, and keyword fields of records. The other part of the discussion is their explaining their mission needs for bibliographic information. I explain that I do not charge the projects for my time or for the use of databases subscribed to by the JPL Library; Web of Science, Compendex, and GeoRef are the core databases. Other databases, such as Inspec, have been used when available through the JPL Library. Occasionally a customer has requested a search in a database accessible through another commercial source. I explain to the customer that anything that the JPL Library pays for the use of the database is charged back to the customer. Usually it is sufficient that the mission or instrument is mentioned in the title, abstract, or assigned key words of publications. The most common additional request by projects is to identify those journal articles which are peer reviewed.

Occasionally teams have requested identification of specific acknowledgements within a publication (Coppin 2013). For example, the Cassini-Huygens team wanted the published articles using the data returned by the mission/instrument even when only mentioned in the publication. This really requires access to the full text of possible articles to be sure ithe mention of Cassini/Huygens data is therein. This sort of citation request limits the scope of the bibliography to those products to which we subscribe in full text and it is very time consuming. So I have not usually recommended this level of effort to requestors of bibliographies since then.

Two missions have supplied lists of team members and requested publications be tagged for authorship by a team member. ASTER periodically requests that non-Library resources be used to identify Japanese publications using ASTER data. Originally in the case of ASTER, a budget account was provided to pay for downloading articles found in the Japanese Sci-Tech database available from Dialog in 2007 (Coppin 2015). Several projects also supplied me with funds to continue GeoRef databases searches during the period when the JPL Library did not subscribe to that database.

I establish the ultimate search strategies. Basic search terms are usually the name of the project and its acronym. Depending upon the complexity of the request, I may make iterative consultations with the team contact about the pertinence of the search results. The team contact may supply key concepts which I use. An example is the expansion of the terminology for the Ocean Surface Topography (OST) Project Bibliography. Since the Topex/Poseidon satellite launched in 1992 there are now large data sets available from several partner agencies that provide data from NASA and other altimeter missions. Articles in this area have evolved from mentioning the instrument name to either additionally mentioning the names of the data providers or only mentioning the data provider. So these partner agency names – AVISO, SSALTO, DUACS - are now included as search terms.

The Cassini project request for all articles using data from the mission means including the names of Saturn's 53 named moons in a search. The search strategy is a challenge since Saturn's names come from mythological Greek or Roman giants or titans, or more recently from other mythologies. Particularly for the Greek or Roman mythological names, they have often been used for other purposes that fall within the same broad scientific or technical categories. Some examples are:

Depending upon when the request is made, the initial search results for a project bibliography may retrieve hundreds or thousands of publications. Depending upon the request and the number of records to download, I may review the initial results for pertinence and select records to be downloaded. Or, all results may be downloaded and the project contact helps review the publications for pertinence. For example, one project contact identified certain more popular science journals to be excluded at all times. I save the effective search strategy and use it to establish alerts for new publications. The alert updates are easy to review for false drops, duplication, or for publications that are questionable for meeting requestor criteria.

Bibliographic databases often appear to use automatic indexing processes that pick up mission names based upon references at the end of a paper. However, this automatic indexing also creates "false drops" when the references are only used in the introduction section of the article as background. The paper is actually about another topic rather than the background mentioned in the introduction. For a mission such as Cassini with multiple instruments, abstracts may mention the instrument rather than the mission name. Instruments and their names may be identical on different missions. For instance, the MER rovers and the Mars Science Laboratory mission's Curiosity rover each have an Alpha Particle X-ray Spectrometer (APXS). Depending upon the project bibliography's criteria, if the published title and abstract seem to be really different from the mission, I may take a deeper look at the pertinence of the article. In those cases I will either find in the full text of the article where the mission name appears, or I will check what references are listed.

Search results are downloaded into a bibliographic citation manager. I started with EndNote. Then I switched to RefWorks, and last year I converted back to EndNote. The choice of bibliographic management tool has been a combination of functionality, personal choice, and funding for particular tools. It has been important to create bibliographies that can easily migrate from tool to tool. JPL's missions are often decades long, and tools come and go.

Only the requested tagging is done for the initial download of search results. Peer review status is assigned according to the refereed status listed in the 2012, 2009, and earlier the 2007 editions of Ulrich's Periodicals Directory. If a journal is not listed in the Ulrich's available in the JPL Library, the publisher's web site is visited and a yes or no determination made according to the information provided about the publication process. Since we now subscribe to the ProQuest Natural Sciences collection I also check there to see if there is refereed status available. Different databases may label publications differently or choose a series title over a journal tile for the main source reference. Particularly for some conference publication series, some databases have labeled it a journal reference type while others label it as a conference proceeding. Standardization of these different database practices is only done on individual records when I process records from scheduled alerts. When an American Geophysical Union, Geological Society of America, or Lunar and Planetary Science Institute meeting abstract appears in an alert, it is assigned Abstract as a Reference Type.

Currently I use a different EndNote Library for each project bibliography. Groups are used to separate articles-in-press from published publications, publications identified by the mission as not wanted, for questionable publications needing input from the mission contact, and for the accumulation of wanted publications. I have assigned specific fields in EndNote for coding/tagging purposes. Global editing is used as much as possible for the required tagging/editing. Since for JPL purposes the DOI link needs to be active, I use the ability in EndNote to add http://dx.doi.org/ before the unique part of the DOI downloaded from databases. One practice that has proved useful is adding the database name and date downloaded into the Database Name field. Most databases do not download their name into that field. With that information I can answer questions about when a publication was first known or for where the information came from. If it is necessary to copy records from EndNote to RefWorks or vice versa, the information in the Database Name field is retained while the date of transfer becomes the record's date of input into the bibliographic manager. Another common addition I make is adding the code for the update period to the records downloaded from alert updates. This is useful when compiling updates and knowing when I sent a record to the project.

Output for project bibliographies varies according to the specific project. Basic output is a bibliography or update to the bibliography with just the basic citation information and active DOIs. Because of licensing constraints anything else, such as the abstract, is not shared outside JPL. Some projects want XML records with all a record's fields. These projects have set up programs to load these records into publication databases accessible by the public. While words in Abstract and Keyword fields may be searched, the public only sees the basic citation and any DOI. Other projects want Excel files either with all fields or with basic citation information, abstracts, and keywords. They will retain these files or review them for selecting records to add to public bibliographies. When my project bibliography file contains all the records for the entire period covered by the project request, I can use it to compile publication metrics.

A popular product is a chart showing publications-by-year. Depending upon the project's request, the chart may show just journal and peer reviewed journal articles by year. Or, the chart may show total publications, journal articles, and conference publications by year. Also I can use these comprehensive files to answer specific requests for different listings. For instance, one special request was for a bibliography of non-team member publications to show the use of the mission data beyond the science team. Another example was simply providing the project scientist a current table showing the number of journal articles per year so he could respond to a reporter's request. Special requests have included searching Web of Science for the top publications citing the mission data. The most cited article for the Cassini Mission was unexpected. There were 708 total citations to the article about a test of general relativity using the Cassini spacecraft in August 2015 and only 493 citations to the next article which is about Titan's atmosphere.

I maintain a document describing the process I follow for each project bibliography. This document has two main parts -- the first section has some general comments about processes, such as exporting to Excel from Endnote, and the second section lists the individual bibliographies. For each bibliography I list the project contact, when the mission/instrument was launched/started, the period each bibliography covers, databases generally used, and comments such as for ASTER: "Spaceborne may also be spelled as two words, space borne." Then I go into detail on the processing of downloaded records including the special coding required, frequency of updates, and what to do with "Articles in Press" when downloaded from Compendex. When there are several products required for an update, I list that as a checklist. I have a separate document with all my search strategies even though these strategies have been saved in the various databases used. Both documents are used when cross training another JPL Librarian as backup for these ongoing bibliographies.

Conclusions

Project bibliographies relating to a specific instrument or mission provide a number of benefits. The project can include publications as part of the institution's (JPL or a partner) education and public outreach effort; demonstrate the value of the science effort by providing metrics derived from the bibliography; show the use of the science/data by non-team members; spot trends and unanticipated uses of the data; and use publication lists and metrics in reports and funding requests. Metrics can show the growth in the use of the data over time when derived from a comprehensive compilation of publications. A trend noticed by OST in publications over the past ten years is an assumption of the use of satellite data in research without necessarily specifying a particular mission/instrument in the title, abstract, or author assigned keywords. ASTER publications also support the trend toward mentioning remote sensing or satellite information without specifying a mission name in the title or abstract.

While project bibliographies can be done by project team members, there are benefits to having JPL librarians compiling the bibliographies. We understand the resources available and can effectively search the various databases. The searches are consistent and the results are reproducible. We save the project team members' time at no additional cost to the project. Use of a bibliographic manager to collect the search results over time provides a data file from which metrics can be compiled. The bibliographic manager also provides different export options to meet the needs of the project. An additional benefit to the library is that team members become aware of our services and then come to us for requests unrelated to the project. These additional requests have varied from simple literature searches to competitive intelligence requests.

Acknowledgements

The author gratefully acknowledges the assistance of JPL Library's supervisor, Robert Powers, Robert.A.Powers@jpl.nasa.gov, in reading the manuscript and providing valuable comments.

The research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. Copyright 2016 California Institute of Technology. Government sponsorship acknowledged.

References

Accomazzi, A., Henneken, E., Erdmann, C., & Rots, A. 2012. Telescope bibliographies: an essential component of archival data management and operations. In: Observatory Operations: Strategies, Processes, and Systems IV. Vol. 8448. Amsterdam, Netherlands: SPIE. p. 84480K-10, doi: http://dx.doi.org/10.1117/12.927262.

Best Practices for Creating a Telescope Bibliography. [Internet]. [updated 2013 Dec 6]. International Astronomical Union Working Group Libraries. Available from http://iau-commission5.wikispaces.com/WG+Libraries+-+Best+Practices+for+Creating+a+Telescope+Bibliography

Coppin, A. 2015. Snapshot: Finding Japanese Scientific & Technical Information, Issues in Science and Technolology Librarianship 81, doi: http://dx.doi.org/10.5062/F40C4SS7.

Coppin, A. 2013. Finding Science and Engineering Specific Data Set Usage or Funding Acknowledgements. Issues in Science and Technolology Librarianship 73, doi: http://dx.doi.org/10.5062/F4CV4FP0

Lagerstrom, J., Winkelman, S., Grothkopf, U. & Bishop, M. 2012. Observatory bibliographies: Current practices. In: Observatory Operations: Strategies, Processes, and Systems IV. Vol. 8448. Amsterdam, Netherlands: SPIE. p. 84481X-8, doi: http://dx.doi.org/10.1117/12.925482.

Major, G.R. 2011. Impact of NASA EOS Instrument Data on the Scientific Literature: 10 Years of Published Research Results from Terra, Aqua, and Aura. Issues Sci. Technol. Librariansh 67, doi: http://dx.doi.org/10.5062/f4cc0xmj.

Rots, A.H., Winkelman, S.L. & Becker, G.E. 2012. Meaningful metrics for observatory publication statistics. In: Observatory Operations: Strategies, Processes, and Systems IV. Vol. 8448. Amsterdam, Netherlands: SPIE. p. 84480J-9, doi: http://dx.doi.org/10.1117/12.927134.

Savaglio, S. & Grothkopf, U. 2013. Swift Publication Statistics: A Comparison With Other Major Observatories. Publications of the Astronmical Society of the Pacific 125:287-294, doi: http://dx.doi.org/10.1086/670027.

Seal, D.A. & Manor-Chapman, E.A.. 2012. The keys to successful extended missions. In 12th International Conference on Space Operations, SpaceOps 2012, June 11, 2012 - June 15, 2012, Stockholm, Sweden, American Institute of Aeronautics and Astronautics Inc., doi: http://dx.doi.org/10.2514/6.2012-1295834.

Taşkın, Z. & Aydinoglu, A. 2015. Collaborative interdisciplinary astrobiology research: a bibliometric study of the NASA Astrobiology Institute. Scientometrics 103:1003–1022, doi: http://dx.doi.org/10.1007/s11192-015-1576-8.

Previous Contents Next
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.W3C 4.0 Checked!