Database Reviews

Ulrich's Serials Analysis System

Chew Chiat Naun
Assistant Serials Librarian
University of Illinois at Urbana-Champaign
naunc@uiuc.edu

Michael Norman
Head of Serials Cataloging
University of Illinois at Urbana-Champaign
manorman@uiuc.edu

General Description

Ulrich's Periodicals Directory is a resource well known to serials librarians in both its print and CD-ROM formats. Ulrich's is a database of serial titles containing not only detailed bibliographic information such as ISSNs, publishers, frequency and available formats, but also business information such as prices and contact details, and additional evaluative data such as a description of the journal and its subject coverage and an indication of whether it is refereed. The database is indexed by a variety of criteria, most notably a subject classification.

The web version, Ulrichsweb, was launched in 1999 and enhances this already useful content with such value-added features as links to tables of contents, reviews, ISI impact factor ratings and alerts. It is also, as one would expect, more current than its print and CD-ROM predecessors.

Ulrich's latest offering, the Serials Analysis System, introduces a further value-added feature to the basic Ulrich database. The Serials Analysis System allows a library to upload a list of its periodical titles (or, more strictly, ISSNs) and then to break it down and analyze it according to Ulrich's subject categories and business and evaluative data. By doing so the system leverages the existing database to create a powerful tool for assessing a library's periodical collection.

Libraries can evaluate their serials collections and gain detailed information about how those same serials compare to defined industry standards. For the first time, this new system allows a library the ability to compare its entire serials holdings against 1) Ulrich's complete database of serial titles or 2) a selective list of collection titles determined by such criteria as whether a journal is refereed, is available electronically, has an ISI impact factor (a measure based on the number of times a journal is cited within the articles populating the leading journals in the various subject disciplines, especially the sciences), or is reviewed in Katz's Magazines for Libraries. The data that are generated, which can be downloaded into a spreadsheet for easier manipulation, gives percentages of matches the library's serials collection has with the serials contained within the Serials Analysis System and also with these four important standards. Equally important, this new component can produce lists of serial titles that the library does not own. Both of these lists provide librarians the ability to determine easily titles that can be added or deselected in order to develop collections that reflect more of the core titles determined from these industry standards.

A Background to Collection Analysis

Collection analysis tools have been around for quite a few years but have gained more prominence with the introduction of computer statistical and bibliometric programs which help minimize time and effort in collection analysis. For most of the twentieth century, manual comparisons to print lists of the best or core (according to certain established criteria) monographs and serials constituted the norm. It took a lot of time and effort to compare what the library had with what other libraries also owned, or with evaluative lists of the best or core monographs or serials. It was difficult to try to compare library collections quantitatively against national standards or against the collections of other prominent libraries. But in 1989, the collaboration between the AMIGOS Bibliographic Council and OCLC to form a Joint Collection Analysis Systems provided for the first time the ability for libraries to examine sections of their collections systematically to determine strengths and weaknesses. This CD-ROM service compared a library's collection in a particular subject area to that of a peer institution. The service generated statistical data that provided lists of titles for retrospective collection development in a targeted subject area. The most obvious limitation of this service was its focus on only a portion of the library's collection, i.e., monographs, and excluded serials, government documents, and dissertations.

An important development that helped produce better analysis tools was the creation of the Conspectus methodology for collection assessment. In the 1980s the Research Libraries Group developed the Conspectus concept and centered the methodology around 24 subject divisions, 500 subject categories and 4,000 descriptors. The Conspectus includes a range of Dewey Decimal or Library of Congress classification schemes and provides a standardized format, methodology, and codes or indicators to characterize the nature and scope of library collections. Like the AMIGOS/OCLC Collection Analysis System, the Conspectus compares a library's collection in a particular subject area with the holdings of other libraries, but it also adds to its criteria standard bibliographies, indexes, lists of serials in a given subject area, and citation reports. For the past twenty years many libraries have used the data created from these comparisons to determine quality of their collections, decide collecting priorities, help with accreditation reviews, make decisions on weeding and remote storage, and provide information about improving cooperative collection development between consortia. The Conspectus is still in use today. It looks at subject collections as a whole, incorporating monographs, serials, microforms and, recently, electronic resources.

In 1997, the AMIGOS/OCLC CD-ROM product was discontinued. However, within a few years, OCLC brought out a suite of services that could help libraries in the analysis of their collections. The umbrella product, called Automated Collection Assessment and Analysis System (ACAS), allows a library to provide a list of titles or have OCLC run reports of the library's holdings in WorldCat to analyze their collection from four different aspects including 1) age and content; 2) group analysis with overlap and uniqueness; 3) library collection comparisons (gap analyses); and 4) recommended list comparisons (with Books for College Libraries, Choice's Outstanding Academic Titles, and Booklist). Within the ACAS umbrella, the Interactive Collection Analysis System (iCAS) component brings in the latest technology to help provide qualitative and quantitative measures to describe the current strength of the library's resources for all the different formats. Using platforms such as Microsoft Access or SQL databases, one can look at the age and content of a certain subject area of their collections.

While these collection analysis tools have provided valuable and helpful information to evaluate a library's overall collection, they have not been designed specifically for analyzing periodical collections. These evaluative tools look at the collection as a whole or subsets based on subject area. Monographs and other formats, including serials, are viewed in combination to produce lists of strengths and weaknesses within the collection.

Ulrich's Serials Analysis System is the first evaluative service to provide collection analysis criteria specifically for serials. Serials have their own factors and criteria when trying to determine quality of one's collection including, as we have mentioned, the ones available in Ulrich's: whether a journal is refereed, is published electronically, has an ISI impact factor, or is reviewed in Katz's Magazines for Libraries. So the arrival of Ulrich's new product is potentially of great interest to libraries, especially in light of Ulrich's authoritative status in the field of serials. The product clearly has a niche, and it is likely to be all the more attractive in the present environment, where libraries are closely scrutinizing their serials expenditures. How well, then, does it do the job?

Producing the Reports

The Serials Analysis System is quite easy to set up. The initial setup can be done by any librarian who has a basic knowledge of how to copy files, provided that there is a current and complete list of the library's periodical titles. Getting such a list is, of course, the difficult part, and depends on how the library manages its periodicals data. A library that has its print and electronic collections fully cataloged should have little difficulty producing such a list. This review was based on a file of ISSNs drawn from a partial listing of the University of Illinois at Urbana-Champaign's periodicals collection. This list includes some 23,000 currently active titles and a similar number of inactive ones, but largely omits electronic holdings.

To set up the system for use, the librarian prepares a text file of ISSNs (one ISSN to a line) or, alternatively, a text file of ISSNs and associated titles. This file is then uploaded into the system through a simple browse and click. If titles are included in the uploaded file, then it is these supplied titles rather than the titles as given in Ulrich's that appear in the reports. The system will report any unmatched ISSNs and will reject duplicate ISSNs. The system appears to consider only active titles, which means that inactive titles will be reported as non-matches and will therefore be excluded from the analysis. For most libraries the exclusion of inactive titles will not matter, although it is possible to conceive of situations where the additional analysis could have been useful.

If necessary, a library can upload more than one list of ISSNs representing distinct sub-collections that can then be analyzed independently. There is also a field made available for the library to upload its own usage statistics for each title so that these data can be included in the reports generated by the Serials Analysis System.

Non-matches are output as a separate report accessible from the main result page. Reasons given for non-matches include "non-active," "ISSN non-match," and "Title history," among others. Some of these terms are self-explanatory, others less so. We were unable to find definitions of these terms in the help pages.

The accuracy of the analyses produced depends on the accuracy of the matching. Satisfactory matching depends on the data being complete and current on both sides. One important area where Ulrich's data fall short is in the apparent failure to include electronic ISSNs, at least for the titles that we viewed. For example, the American Journal of Bioethics is listed under its print ISSN, 1526-5161, but not under its electronic ISSN, 1536-0075. This kind of omission matters less in the context of a manual lookup, where if a user fails to find a journal by its ISSN she can simply try an alternative search by title, or research the print ISSN. But for automated matching the consequences are more serious. If a library has an electronic-only subscription and is relying on data from an aggregator or another source that does not supply the equivalent print ISSN, the match will fail and the title will not show up in the Ulrich's reports. The potential for skewing the results of such reports is obvious. The point bears repeating: libraries which are considering a subscription to this service should evaluate their own sources of data before proceeding.

The Serials Analysis System reports are located in a password-protected area of the Ulrichsweb site and are intended for use by librarians only.

Functionality and Usability

The Serials Analysis System outputs its results as a set of tables giving the number of titles in the library's collection in each of the 150 or so broad subject categories used in the Ulrich's database.

The number of titles is given both as a raw total and as a percentage of the total number of titles that Ulrich lists under each of the subject categories. The library can choose to compare its collection against either the complete Ulrich list of periodicals (known as "Ulrich's Universe") or a selected list of academic and scholarly titles ("Ulrich's Core"). The results can be further broken down by subcategories under each subject heading. For example, Chemistry breaks down into such subcategories as Analytical Chemistry, Crystallography, and Organic Chemistry, among others.

What might also have been useful is a further column giving holdings in each subject category as a percentage of a library's total periodical collection. But it would be easy enough for a librarian to add this column after exporting the data to a spreadsheet (see below).

There are additional options for filtering these results in order to obtain more detailed analyses. The results can be limited to refereed journals, electronic journals, journals with an ISI impact factor, journals reviewed in Katz's Magazines for Libraries, and journals published in a particular country or language. The user can move from these statistical summaries to further tables giving a detailed title-by-title listing of the journals in each of the subject categories or subcategories.

Each title is listed with its ISSN, publisher, price and an indication of whether it is refereed. There are additional columns providing links, where available, to the electronic version, the ISI impact factor, and reviews in Katz's. This page is also where the library's usage statistics appear if they have been loaded into the system. Clicking on the ISSN will take the user to the full Ulrich's record for that title.

All of the fields in the title listing are optional; the library can choose which ones to include in its reports by making a selection on a "Default Report Display Values" screen.

A full listing of the library's uploaded titles is also available, although the usefulness of this option is much reduced by the fact that the results cannot be downloaded, and by the absence of any easy way to navigate the list. More useful perhaps is the column in each report showing which titles the library does not hold in a given subject category.

Finally, the system offers an analogous set of breakdowns not by library, but by major publishers. This option enables the user to assess the strength of each publisher in specific subject areas, as well as to measure each publisher against the other criteria supplied by Ulrich's: refereed status and the rest.

With the exception already noted, any of these tables can be downloaded into a spreadsheet. Given the purpose of this service as a collection analysis tool this is a very useful feature. We did notice, however, that the summary statistics page can be downloaded directly into Excel while other pages come down as pipe-delimited text files. What the downloaded spreadsheets do not include is the live links to additional data. Something that Ulrich's might want to consider is to enhance the system to allow its customers to save selected pages to an area within their Ulrichsweb account, thus preserving the links. A further possible enhancement would be to provide an option to save the reports as live queries, ensuring that the reports are always current. From the vendor's point of view, such enhancements would also give customers an incentive to continue subscribing to the product once they had conducted their initial analysis. It is at present possible to create library-defined lists of titles, but these lists need to be compiled one title at a time, and they do not reproduce the evaluation-oriented format used by the reports.

Another benefit of allowing users to save specific pages is that it would make it easier to find specific result sets. A library using this product on an ongoing basis will probably have reason to access the same report on several occasions. At present that process that is somewhat cumbersome because the system requires the user to start again each time from the opening screen. In most other respects, navigating the reports is fairly straightforward. There are more ways of getting to some pages than appears to be strictly necessary, but that is better than having too few.

Navigating large sets of results is quite easy although not perhaps as intuitive as it might be. The default sort is by ISSN, and there is an option to re-sort by any of the displayed fields, including title. On the default sort page there is a search box allowing the user to jump to a specific ISSN. What is not immediately obvious is that this turns into a title search box when you re-sort by title. The actual re-sorting process is satisfyingly quick.

It has to be said that Ulrich's has not done its users any great favors with its help and instructional material. There is a "Quick Guide" which serves well enough for a novice user to find her way around the system, but we suspect that many users would also like to have a detailed manual available for downloading. The help pages are rather sparse in content and they are not context-sensitive. Often the ALT text supplies the best clue about what to do.

Subject Analysis

Ulrich's subject classification is much less detailed than the Conspectus, but it is doubtful whether that is really a disadvantage. The Conspectus was designed essentially for monograph collections, which demand finer-grained analysis for the simple reason that monographs are generally more specific in their subject-matter than periodicals are. For most purposes Ulrich's classification should be perfectly adequate for periodicals.

A more serious drawback has to do with the way the system makes use of the subject classifications. As we have seen, the system provides a count of a library's holdings under various subject classes and compares this information against various other indicators. The problem is that the system counts each title in only one subject classification. If a journal receives more than one subject classification the rest are simply ignored for the purposes of compiling the reports. For example, Physics and Chemistry of Liquids is indexed in Ulrich's under both "Physics-Mechanics" and "Chemistry-Physical Chemistry". But the Serials Analysis System reports it only under "Physics-Mechanics". That is to say, you will not find this title either listed or counted under "Chemistry-Physical Chemistry". Since there are many journals which straddle subject classifications, the analyses provided by the system must necessarily be incomplete, not to say misleading. The problem is particularly acute with journals that are inherently interdisciplinary in nature, such as ACM Transactions on Mathematical Software, which is reported under "Computers-Software" but not, surprisingly enough, under "Mathematics-Computer Applications". With the growing trend towards interdisciplinary journal publications, as well as interdisciplinary collections, this limitation of the system is likely to become more noticeable over time.

In our view the system's inability to provide full listings by subject compromises its usefulness for assessing periodical collections. It may be argued that each title only needs to be reported under its primary subject, but in the examples we have cited there are multiple subjects which are clearly of equal or at least comparable importance. Ulrich's says that titles are counted under only one subject "to ensure that reports reflect accurate counts and that the same title is not counted under multiple subjects within a collection" (Ulrich's 2003). Our feeling is that it is essential for a service of this kind to be able to provide comprehensive listings by subject, and that there should at least be an option to obtain these results.

Evaluation

Ulrich's Serials Analysis System is an ingenious and potentially very powerful tool for analyzing periodical collections. It is for the most part easy to use and offers a useful range of criteria and additional information for libraries to assess their collections. It can be extremely useful, for example, to pull up a library's list of refereed journals within a given subject area. It's not easy to think where else a library could get a comparable compilation of data. Moreover, the availability of downloads into a spreadsheet format makes it easy to re-use the data elsewhere.

A resource of this kind can only be as good as the information that goes into it. Fortunately Ulrich's is a reputable reference source of long standing, and the existing features that are leveraged in this offering, such as the subject classification and the selected listing known as Ulrich's Core, appear to be well regarded. Of course, the reliability of the service also depends on the data that a library feeds into it, and for some libraries the usefulness of the system will be limited by how current and complete their own journal lists (and particularly their lists of ISSNs) are.

In its present form the product does have a number of limitations which cannot entirely be overlooked. We have noted our reservations about the apparent omission of a key piece of identifying data, the e-ISSN, and about what we think is a basic flaw in the way the subject analyses are produced. We hope that these drawbacks will be addressed in future releases of the system.

During the review period (July 2003) we encountered some performance problems in generating the reports. On several occasions the process stopped before the report was produced and we received a "connection with server was reset" message. Occasionally the response time would slow to a standstill and could be restored only by closing all browser windows and logging in again. We assume that Ulrich's will address these problems as the system comes into widespread use. In other respects we encountered no serious difficulties using the product, although the help pages could be improved.

Pricing of this product is based on the library's number of live periodical subscriptions. We were unable to obtain details of the pricing matrix but the price does appear to involve a substantial premium over the price of the basic Ulrichsweb product. The Serials Analysis System cannot be acquired separately from Ulrichsweb, which is understandable given that the former is essentially an additional piece bolted on to the latter.

In spite of our reservations, Ulrich's Serials Analysis System is a major addition to the librarian's array of collection management tools. We expect that it will be widely used and that it will enable many libraries to make much better informed decisions about how to spend their subscription budgets than they have previously been able to do. The system has the advantage of a well-conceived framework which could lend itself relatively easily to further enhancement. For example, it could be developed into a tool for comparing the journal collections of different libraries, making it a more direct counterpart of OCLC's ACAS. Similarly, it could be developed into a tool for comparing the offerings of different publishers, or for identifying titles to review for cancellation. Whether the business conditions exist for Ulrich's to pursue these possibilities is, of course, another matter.

Ulrich's Serials Analysis System is published by R.R. Bowker, 630 Central Avenue, New Providence, NJ 07974. For more product information please contact the publisher at info@bowker.com or consult the product web site at http://www.ulrichsweb.com/ulrichsweb/analysis/. A preview trial is available.

References

Astle, D.L. 2002. Ulrichsweb.com. Charleston Advisor 3(4): 37-40.

Hart, A. 2003. Collection analysis: powerful ways to collect, analyze, and present your data. Library Media Connection 21(5): 36-39.

King, D.W. et al. Library econometric metrics: comparison of electronic and print journal collections and collection services. Library Trends 51(3): 376-400.

McDermott, I.E. Confessions of a serial clicker: Ulrich's on the web. Searcher 10(9): 8-12.

OCLC. 2002. ACAS: automated collection assessment & analysis services. [Online]. Available: {http://www.oclc.org/western/products/aca/default.htm} [July 26, 2003].

Ulrich's. 2003. Serials Analysis System. [Online]. Available: http://www.ulrichsweb.com/ulrichsweb/analysis/ [July 26, 2003]

Previous

Contents

Next