Brief Communication: Is citation normalization realistic?

Abstract

One method for assessing quality of research outputs across different technical disciplines is comparing citations received by the research output documents. However, crossdiscipline citation comparison studies require discipline normalization, in order to eliminate discipline differences in cultural citation practices and discipline differences in the number of active researchers available to cite. The 'definition' of, and number of documents used to represent, a discipline become critical. This study attempted to determinewhether the citation characteristics (average, median) ofa discipline's domain stabilized as the domain's size was decreased. A sample of papers (classified as research articles only, not review articles, by the Institute for ScientificInformation) published in the journal Oncogene in 1999 wasclustered hierarchically, and the citation averages and medians were computed for each cluster at different cluster hierarchical levels. The citation characteristics became increasingly stratified as the clusters were reduced in size, raising serious questions about the credibility of a selected denominator for normalization studies. An interesting side result occurred when all the retrieved articles were sorted by number of citations. Thirteen ofthe fifty most highly cited research articles had 100 or more references, whereas zero of the fifty least cited research articles had 100 or more references. Keywords: citation analysis; citation normalization; document clustering; research evaluation