Three Cool Things You Wish You Knew Before About Bibliometrics

Biblio… WTF?
Yeah, you think, here is the science guy with his words again, but actually bibliometrics is a a good set of methods to understand and study research in your field of interest. Usually, all it requires is a publication search engine, like Scirus, ISI Web of Science (you need an institutional login), Google Scholar, BioMed Central, DOAJ, OAIster, or Faceted DBLP. And then you search your favorite terms and count the results in different clusters. You put it in your favorite spreadsheet software and plot a graph to see a general trend and compare.

Example: HCI & Game Bibliometrics
I was actually inspired to do this by Gareth over at Game Player Interaction, who always finds way more publications than you are able to read and who recently posted a DBLP search query on games. So, I sat down and gave my 5 favorite search terms a try in Faceted DBLP (which focuses on computer science). First, I put in “game* play” (note that the asterisk usually indicates any kind of affix), usabililty, hci (human-computer interaction), gameplay, and finally “digital games“. The following chart shows how they compare:

Game Bibliometrics

First, we notice of course the rising trend in the amount of research papers published that contain the keywords game and play. Second, we see some sort of similar trend for usability papers, also rising more slowly. Then, when looking at the hci data, we instantly notice two things: (1) the trend is not a straight line, but vibrates periodically (we call that oscillatory), (2) which also means it shows spikes, one of which is rather large for 2007. Then again, if you look at the curves usability, games and play you also see a spike in 2007. I was puzzled. Something must have happened in 2007 that must have shook the games interaction and especially the HCI community that year. Something big.

Exploring the Statistics
To find out what it was, I had a look at more statistics related to the search queries. For example, what publication venues were listed for digital games? It is clear that this search term is usually used by the DiGRA community, so as no surprise, 86.7% of all listed digital games publications where published at a DiGRA conference. Then, I looked at my gameplay keyword, at it turns out – to my surprise actually – that the main publishing venue for this was the International conference on Advances in Computer Entertainment Technology (ACE) with 21.7% of all publications listed there. This was interesting, but did not give me the answer I searched for: What happened in 2007?

The Secret of HCI
You would of course naturally proceed and look at the hci and usability keywords next, which I did. Turns out, the two biggest HCI conferences are HCI International with 41.5% of all hci publications listed there and INTERACT conference with 11% publication share. For usability 7.6% of all publications come from the CHI conference, which is only a bit more than 7.4% that come from HCI International (which also shows that there is no established scientific venue for usability research as all other publications are shattered over different journals and conferences). The HCI International conference was founded in 1984 and is held every two years, then in 1995 the INTERACT conference was established and is also held every other year. And this is why we have this waveform of the curve for the hci keyword (- does this also mean that human-computer interaction researchers produce their research outcomes in waves?). Further investigation on the HCI conference website yielded an final insight to the question what happened in 2007!

Something Chinese
Yes, you remember the olympic games – that minor event there last year. Turns out, China (Beijing) was also hosting the HCI International conference in 2007 jointly with a bunch of other related conferences. To quote them “this event was one of the biggest ever organized in the fields related to Human-Computer Interaction and Information Society Technologies”. 2300 participants sounds truly olympic to me for a scientific conference. But there it is, the answer to a question, we never would have asked had we not looked at some statistics. 🙂

3 Cool Things You Know Now About Bibliometrics

  1. You are able to see a research trend in numbers of publications (games and usability have both rising publication numbers)
  2. You understand what conference venues relate to what keywords (we know that HCI International is the biggest HCI conference)
  3. You understand a general tendency in publication frequency in your research field (HCI research is published every other year)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.