Google Scholar Citations: Serious Errors » Dean Keith Simonton, PhD

Initial Observations:

Google’s excellent service is still a work in progress. Because the “citation counts are estimated and are determined automatically by a computer program,” the results sometimes indicate more artificiality than intelligence. When I conducted an initial scan on 11/20/11, I found numerous errors both major and minor. An example of a minor error appeared for my 1984 Genius, Creativity, and Leadership which Google said was published by “Cambridge.” In fact, the book was published by Harvard University Press, which just so happens to be located in Cambridge MA.

An instance of a major error occurred much farther down the list: the title “Disposition” published by APA Books in 2002. My CV lists over 400 publications, but not one has this title, and certainly no 2002 publication is so titled! It soon became evident that this reference was actually to a chapter called “Disposition” in my 2002 book Great Psychologists and Their Times – which was nowhere to be found on the Google publication list. Oops! For some reason, one chapter was taken to represent the whole book! How does that influence the citations received? Has the book been orphaned?*

These are not the only issues with the service. If you click on a given title, you’ll be sent to a page that quite nicely gives the citations per year for that particular title. The page also includes a “Description.” This is how Google Scholar describes Genius, Creativity, and Leadership:

In 1804, Ludwig van Beethoven completed the composition of the famous Third Symphony, now known as the Eroica (‘Heroic’). Initially dedicated to Napoleon Bonaparte, who the composer greatly admired as the champion of freedom, Beethoven had a sudden change of heart upon learning that Napoleon had just crowned himself Emperor of the French. Beethoven immediately obliterated his hero’s name from the title page–so violently that it left a hole in the paper of the original score. A couple of years later, when Beethoven just …

This passage does not describe my 1984 book! So where did Google get their description? – from a book chapter with the same title that I published 25 years later (Simonton, 2009m). That’s how my chapter opens – and the opening doesn’t even describe that publication! Apparently, because this chapter is not listed as a publication, the algorithm got confused, and collapsed the two references into one. Presumably, a few of the citations listed for the book might actually be for the later chapter, albeit not enough time has elapsed for the latter to accrue many citations. Furthermore, the “computer program” automatically takes the opening lines of the publication as descriptive of the whole. This decision rule is unfortunate to the extent that I often begin my articles and chapters with an anecdote! Only after reading the entire paragraph would anyone understand why this story relates to the title of the chapter. It actually gets worse than this. The “Description” for my 1988 Scientific Genius? – the list of the book’s figure captions! C’Mon Man! [No wonder, then, that Google Scholar seems to have corrected the Description feature at some later date, for the most part using the marketing blurbs instead.]

I could list other mistakes both big and small, but the above should suffice. Because I am currently working on correcting the major errors – to the extent that I’m allowed to do – I hope that some of my remarks above are already obsolete. In the meantime, take Google with a grain of salt.

Subsequent Observations:

On 12/13/11, I took advantage of my winter break to compare Google’s publication list directly with my CV. Although there were relatively few errors of commission – I only had to delete a few entries – there were many errors of omission. In particular, about 100 publications were missing. Nonetheless, most of these were minor items that would not be expected to attract citations (e.g., editorials, book/film reviews, replies, and encyclopedia articles). Even so, I also found several other publications, mostly book chapters, that were seriously overlooked, including one with 21 citations, and nine others with 10 citations or more. Their addition obviously substantially raised the total citation count, and even increased my i10-index by 10 publications. Interestingly, the additions and subtractions did not alter the h–index one iota, suggesting that this indicator is rather robust.

On 2/20/12 a curious change was noted: My total citations declined from 9863 to 9852. Given that these citations are cumulative, this loss is hard to explain – particularly given my “since-2007” count was unchanged from before. That means that the subtracted 11 citations all came from pre-2007 publications. Perhaps the algorithm was correcting mistakes made in earlier runs. Fortunately, the h-index and other measures did not change in this revision. Still, the decline is odd, especially given that it has happened more than once. How can a positive monotonic measure be nonmonotonic?

On 9/19/12 I discovered a dramatic loss of nearly 400 total citations in a single day, a decline that even reduced my h-index from 49 to 48! This temporal instability in Google Scholar is somewhat disconcerting, to say the least.

On 7/17/13 another troubling occurrence. For some inexplicable reason, Google’s algorithm decided that my 1984 Genius, Creativity, and Leadership published by Harvard University Press was exactly the same book as my 2009 Genius 101 published by Springer Publishing! Hence, all citations to the latter were assigned to the former, and the latter became completely uncited. Given the distinct publication dates, publishers, and titles, this conflation seems really stupid. Although I have since corrected the mistake, it must make one think that the algorithm will certainly commit another act of stupidity. Somehow the programmers must include lines of code that implement the deduction that two publications defined by different titles, different publication dates, and different publishers cannot be the same publication. Basic logic, really.

On 12/20/13 I decided to edit the newer entries, which contain many errors. But I can no longer do so. Don’t know why. Until I again get access, the site will become increasingly untrustworthy. CAVEAT EMPTOR!

On 9/29/14 I noticed that my single most cited publication, the book Origins of Genius (OUP, 1999), showed a substantial decrease in citations. I then spotted the culprit: a 2006 book review published in APA’s PsyCRITIQUES that was also called “Origins of genius” (concerning a collection of Charles Darwin’s major works edited by E. O. Wilson). It is impossible that a mere book review would receive 165 citations, so obviously almost all of these citations should go to the earlier book. Google’s algorithm cannot tell a book from a book review or even note the differences in publishers and publication dates. Now that authors cannot correct these faulty tabulations (see 12/20/13) scholars are at the mercy of Google’s artificial stupidity. In any case, it’s obvious that Origins remains my most cited publication, with well over 800 citations.

***

On 1/28/23 I decided to add an entry after a long lapse. Google’s algorithm went nuts! I first noticed that my citation count had increased by about 5,000 citations in less than a week – the biggest single increase since I started following Google Scholar. Even more strange, my single most cited publication had acquired three times as many citations. Stranger still, that publication had switched from my most cited book to a puny entry in a relatively obscure encyclopedia. A little inquiry revealed that the latter had picked up most of the citations to entries in other encyclopedias and handbooks. Google’s algorithm doesn’t realize that just because various entries or chapters share the title “creativity” that does not signify that the publications are identical. On the contrary, the titles of the encyclopedias and handbooks, plus their distinct publication dates, should give a clue that should be written into the program. Certainly an entry in a 1996 encyclopedia of gerontology would not be the same as a chapter in a 2009 handbook of positive psychology. Perhaps ChatGPT would do a more intelligent job of making the requisite distinctions.

On 2/23/23 I looked more closely at what Google Scholar claims to be my far most cited publication. According to their algorithm, the honoree is the following brief 1996 entry:

Creativity DK Simonton Encyclopedia of gerontology, 341-351.

This supposedly received 6319 citations as of this date – 3.5 times more than my next most cited publication. Yet the asterisk reveals there’s something tentative about this attribution. If you click on the symbol you get:

This “Cited by” count includes citations to the following articles in Scholar. The ones marked * may be different from the article in the profile.

Handbook of positive psychology
CR Snyder, SJ Lopez
Oxford university press, 2001
4786*

Creativity: Cognitive, personal, developmental, and social aspects.
DK Simonton
American psychologist 55 (1), 151, 2000
1564*

These two attributions are catastrophically wrong. Although I published a chapter in the handbook, I certainly don’t deserve all 4786 of those credited but probably only a small subset (plus the handbook was published in 2002, not 2001). I can’t tell how many citations my chapter earned because that chapter no longer exists according to Google Scholar. Next, the major journal article has no business being included with a handbook or handbook chapter. The subtitle, the journal data, and the publication date should reveal as much to any human intelligence. Yet, in effect, that journal article has also vanished, subsumed under an obscure entry in an encyclopedia of gerontology!

When I see algorithms so utterly stupid, I can’t help but think of the Tesla crash fatalities or the ChatGPT absolute idiocies. AI is an abbreviation for Arrogance Illustrated. These programs are uniformly pre-BETA.

*Although I managed to correct this error in my personal profile – merging six “separately cited” chapters into one book – that correction doesn’t carry over to Google Scholar as a whole. As far as Google Scholar is concerned, I never wrote a book titled Great Psychologists and Their Times! That saddens me because I consider this work to be the greatest single work of pure scholarship that I have ever published. Yet some more recent books are cited many times more. Hence, to resolve this difficulty, I tried to approach Google directly. The attempt failed miserably. Even real human beings cannot override their computer algorithm. The exact unedited record follows:

From: “Dean Keith Simonton” <dksimonton@ucdavis.edu>
Subject: Re: [#987307887] Major Book Publication Utterly Missing!!!!
Date: Fri, 16 Mar 2012 17:27:29 -0700 (PDT)

TWIMC:

My 2002 Great Psychologists and their Times: Scientific Insights into
Psychology’s History has yet to have its existence recognized in Google
Scholar. All citations to the book are credited to individual chapters,
but not to the book as a whole. Although I consolidated the chapters
under the single book chapter title at My Citations, these corrections did not
carry over to Google Scholar as a whole.

Perhaps the error originated with the publisher, APA Books, which
originally but erroneously had the book listed as an edited volume with
separate chapters all (coincidentally) written by me. Although APA long
ago corrected the mistake, Google did not follow suit. As a result,
Google cannot find either citations or book reviews (I uploaded some excerpts
myself). In effect, my best single scholarly work has been ignored. I
can only wonder how this affects my h-index and other citation indices.
Perhaps the current 59 citations is an underestimate!

Thanks for your attention to this matter.

Best, Dean
From:   Scholar Support <scholar-support@google.com>
Subject:   Re: [#987307887] Major Book Publication Utterly Missing!!!!
To:   dksimonton@ucdavis.edu
Date:   Sun, 25 Mar 2012 05:33:32 -0000

Hello Dean,

books that are presented as separate chapters can be challenging to index
well. In many cases, the chapters are unrelated and it is better to index
each chapter separately. In many other cases, the book is closer to a
monograph and it is better to index it as a single unit. In such cases, we
follow the lead of the publisher, who ,as someone close to the volume (and
hopefully the field), can make the best decision. In this case, the
chapters are marked as separate entries. See for example,
http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2002-17019-007
and
http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=2002-17019-016

Sincerely,

The Google Scholar Team
From:   Dean Keith Simonton <dksimonton@ucdavis.edu>
Subject:   Re: [#987307887] Major Book Publication Utterly Missing!!!!
To:   scholar-support@google.com
Date:   Sun, 25 Mar 2012 09:43:49

Hi, Team ~

Thank you very much for your response, but you need to search further on
the internet.
The publisher APA Books lists this as a complete book, as you can see at
http://www.apa.org/pubs/books/431676A.aspx
So does your very own Google Books at
http://books.google.com/books/about/Great_psychologists_and_their_times.htm
l?id=AxFtQgAACAAJ
and Amazon.com at
http://www.amazon.com/Great-Psychologists-Their-Times-
Psychologys/dp/155798896X

If you look carefully at the links that you provided me below, you’ll
realize that they are 10 years old. As I mentioned earlier, APA originally
made the mistake of treating the book as an edited volume, and listed the
chapters separately. It also listed the chapters separately in PsycINFO.
They corrected that error.

I was the sole author of the entire book. None of the individual chapters
contain my name because they are all written by me. This is just as much a
single-authored composed-through book as any other I’ve written.

Question: What happens when someone cites the entire book? To what chapter
does the citation go? Or does that citation disappear? If the latter, then
how accurate is Google Scholar?

Thanks in advance!

~ Best, Dean
From:   Scholar Support <scholar-support@google.com>
Subject:   Re: [#987307887] Major Book Publication Utterly Missing!!!!
To:   dksimonton@ucdavis.edu
Date:   Sun, 25 Mar 2012 22:11:39 -0000

Hello Dean,

I do understand the issue and would like to help to the extent I can.

As I mentioned, indexing of books with identifiable chapters is indeed
challenging. There is no uniformly applicable approach – the right thing
to do depends on each situation. And for that, we have to work with the
information from the publishers. Given that Scholar works across all
areas, and all languages, it wouldn’t be feasible to do it in other ways.
In this case, the APA records still indicate each chapter as a separate
item. Which is the approach we follow here. I do realize that this is not
ideal in this case. Our apologies.

Sincerely,

Thanks, Team ~

Doesn’t it concern you that a book that you have listed in Google Books,
that is given in the PsycINFO database (the official database of the
American Psychological Association), that is still in press with APA
Books, that is available for purchase on Amazon.com and other online
sources, that has no other author besides me for any of the chapters, and
that has received 59 citations according to Google Scholar – is not
recognized as a book? The latter fact is the clincher for me. All of the
citations that you have assigned to the chapter “Disposition” are actually
citations to the entire book “Great Psychologists and Their Times.” I
know. I’ve checked them. And a very large number of the citations are my
own, as you can easily verify. I have never cited “Disposition” in any of
my publications. That means that anytime someone cites the book as a
whole, you assign the credit to a single chapter in the book. If that
seems like a “scholarly” policy to you, then you are going to face tough
competition from the Web of Science, CSA Illumina, and other databases
that are willing to acknowledge that I published a 2002 APA Book
entitled “Great Psychologists and Their Times” that received at least 59
citations.

Thanks again for this fascinating exchange. It will make for a great
article in a professional journal!

~ Best, Dean