Biometrics, Psychophysiology, Research

Caution with Biometrics, Game Evaluation and UX

My colleague Steve Fairclough recently posted an article on in which he discusses the potential pitfalls of biometric research and how it is currently being sold to the game industry. I will present some of his ideas here.

Steve outlines that “psychophysiological methods are combined with computer games in two types of context: applied psychology research and game evaluation in a commercial context.  With respect to the former, a researcher may use a computer game as a platform to study a psychological concept, such as effects of game play on aggression or how playing against a friend or a stranger influences the experience of the player.”

Similar to Mike Ambinder’s presentation of user research and game design at Valve (PDF), he makes the point that games in this context are analysed using principles of experimental psychology.

They are used as tasks or virtual worlds within which a research can study the behavior of players (you might recall John Hopson’s Gamasutra article on behavioral game design).

He characterises the experimental psychology approach by 4 features:

  1. Comparing controlled conditions
  2. Importance of statistical power (large N)
  3. Controlled participant sample
  4. Counterbalanced design (removing order effects)

He makes a point about the sensitivity of physiological data as being volatile, variable and difficult to interpret without a high level of experimental control. He warns that the use of think aloud protocol might influence the physiological data being recorded because it influences heart rate and respiration considerably.

He also warns about the oversimplification regarding the interpretation of physiological data (something I have seen way too often), regarding its one-to-many relationship to psychological impact.

For example, galvanic skin response is used too often to infer emotional qualities although it is a highly ambiguous measure regarding emotional labels.

Steve closes with a discussion of the question of how to make physiological data and experimentation valuable to the game industry. Meaning, what questions can we answer with this type of data that the game industry does not already know?

This summary was first posted on my Gamasutra blog.

(Please read Steve’s original post here, too)

Advent Calendar 2009

21 – Game User Research: Making Games Better

Today, we have more of a slide collection. But the main featured presentation is the one from Graham McAllister, who is a researcher in video game usability at the University of Sussex in the UK and also runs the company Vertical Slice that specializes in User Experience (UX; human perspective, not quality assurance) testing for games. Most of what I have been researching in the past 4 years is already starting to be employed in practice by them (quite fascinating, really).

First, he explains the different meanings of UX jargon, such as usability (can I do it?), user experience (do I like it?), user interface (how does it look?), interaction design (how is the interface used?). Then he mentions that UX is a key factor driving review scores of games (not the technical functionality alone), which then drive the sales. He backs up his claims with sales data. However, some games with good reviews may still fail financially. On the other hand, games with bad reviews are not very likely to sell well. He then discusses two case studies (Assassin’s creed and Bioshock) in terms of successful design intent or gameplay flaws. The rise of episodic gaming demands a higher level of quality even for vertical slices of games. He goes on to analyze UX flaws of games defaced by gaming magazine reviews. Continue reading