More and more, academics are becoming open to the possibilities of using citation counts and h-index scores in considerations of measuring academic impact. As little as 3 years ago, most faculty involved in deliberations involving promotion and tenure would not consider using automatic metrics, such as those provided by google Scholar. There were many reasons for this reluctance, including: these metrics could be gamed, they do distinguish between being cited for good and bad reasons, they do not index all journals and fields equally, there is lag-time between publishing and citations that follow, and overall angst about machines involved in the process of determining academic quality.
Most recently, I witnessed those same promotion and tenure deliberations consider citation and h-index counts just as any other piece of data in the overall case being made. That is, they considered these metrics as important, but not everything.
Inevitably, this begs the question: What should my citation count or h-index look like as I go up for tenure? What about full? These questions go hand in hand with the usual questions such as : “How many publications do I need?” The answer to all of these questions is, of course, “It depends.”
In this look at the data – I present what numbers actually exist out there in the College of Education @ Michigan State University. This is in no way prescriptive – instead, this is entirely, one hundred percent descriptive.
Using the sample of college of education faculty who have google scholar profiles, I graphed each faculty’s total citation count (x-axis), h-index (y-axis), and academic rank (by shape and color). See the graph on the right. What it shows is there is no hard and fast rule for what a citation count or h-index should be at any rank. Instead, there is a distribution (range) of scores. Furthermore, those distributions overlap considerably. Yet there is considerable grouping by rank, and a discernible trend – higher citation counts and h-index scores certainly do associate with higher rank.Of course, this looks like a strong relationship. However, when years in academia are considered, another strong factor emerged. Using the “first year of publication” as a predictor, I re-organized the display. Now, looking at the first year of publication (x-axis), h-index (y-axis), and faculty rank (shape / color) it seems that faculty rank, h-index, and citations are very strongly associated with time in the field. Of course, correlation is not causation, but it certainly seems plausible that more time in the field leads to promotion, citations, and h-index counts. Only minor variation between individuals with same time in the field seems evident.
Cautionary Words There are some very prominent limitations to this data and analysis, including the issue of selection bias. Roughly 1/3 of College of Education Faculty have a Google Scholar profile and were included in the analysis. It is quite possible that those with a profile differ considerably from those without a profile. Faculty are likely more wiling to publicly share publication records on google scholar when: a) their records are noteworthy, b) are younger, c) more experienced with technology, d) see value in doing so, e) closer to a promotion decision. Analyses may have differed considerably if data were available for all faculty in the College.