• Skip to main content
  • Skip to header left navigation
  • Skip to site footer
This website is moving.


  • Newsletters
  • About
  • Courses
  • Free Course Materials
    • Basic Intro to Game Design
    • How to Write CHI Papers
  • Log In
The Acagamic

The Acagamic

User Experience Research & Design in Games

You are here: Home / Publications / Critic-Proofing: Robust Validation Through Data-Mining

Critic-Proofing: Robust Validation Through Data-Mining

May 5, 2022 by

by Ian J Livingston, Lennart E Nacke, Regan L Mandryk
Abstract:
Critic-proofing is a modified heuristic evaluation technique, specifically designed to provide a fine-grained, prioritized list of heuristic violations. The critic-proofing technique weights the severity of a problem based on the frequency that similar problems are found in similar games. The severity ratings are calculated using data collected from game reviews, and the severity assigned to a problem during the heuristic evaluation process. However, heuristic techniques have had limited adoption within the video game industry. One reason for this is the perceived lack of validity and robustness of game specific heuristic principles. In this paper, we introduce and outline a new data- mining project designed to validate game-specific heuristic techniques, especially the critic-proofing technique by using the popular game-review aggregation website Metacritic.
View PDF
Reference:
Critic-Proofing: Robust Validation Through Data-Mining (Ian J Livingston, Lennart E Nacke, Regan L Mandryk), In , 2010.
Bibtex Entry:
@inproceedings{livingston2010critic,
abstract = {Critic-proofing is a modified heuristic evaluation technique, specifically designed to provide a fine-grained, prioritized list of heuristic violations. The critic-proofing technique weights the severity of a problem based on the frequency that similar problems are found in similar games. The severity ratings are calculated using data collected from game reviews, and the severity assigned to a problem during the heuristic evaluation process. However, heuristic techniques have had limited adoption within the video game industry. One reason for this is the perceived lack of validity and robustness of game specific heuristic principles. In this paper, we introduce and outline a new data- mining project designed to validate game-specific heuristic techniques, especially the critic-proofing technique by using the popular game-review aggregation website Metacritic.},
address = {Leuven, Belgium},
author = {Livingston, Ian J and Nacke, Lennart E and Mandryk, Regan L},
journal = {Proceedings of a Workshop at Fun and Games 2010},
pages = {81--94},
title = {{Critic-Proofing: Robust Validation Through Data-Mining}},
url = {http://www.ianlivingston.ca/storage/pubs/Metacritic-FinalPaper_submitted.pdf},
year = {2010}
}

The public part of this work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International LicenseCC-BY-NC-ND 4.0 license, all paid parts are copyright by Lennart Nacke · The Acagamic · Privacy Policy