Examples are important tools for programming education. In this paper,
we investigate desirable properties of programming examples from a cognitive
and a measurement point of view. We argue that some cognitive aspects of
example programs are "caught" by common software measures, but they are
not sufficient to capture all important aspects of understandability.
We propose a framework for measuring the understandability of example programs
that also considers factors related to the usage context of examples.
Research shows that examples play an important role for cognitive
skill acquisition. Students as well as teachers rank examples as important
resources for learning to program.
Students use examples as templates for their work. Therefore examples
must be consistent with the principles and rules of the topics we are
teaching; free of any undesirable properties or behaviour. By repeatedly
exposing students to "exemplary" examples, desirable properties are reinforced
many times. Students will eventually recognize patterns of "good"
design, gaining experience in telling desirable from undesirable properties.
We therefore need to be able to tell apart "good" and "bad" examples.
To do so objectively and consistently requires some kind of measurement
In this paper, we describe the development and initial validation of an
evaluation instrument for example programs, based on three aspects of
quality; technical quality, object-oriented quality, and didactical quality.
Validation was performed by six experienced educators using examples
from popular introductory programming textbooks.
Results show that the instrument helps indicating particular strengths
and weaknesses of example programs. In all but one example program, we
identified aspects that need to be improved. However, inter-rater agreement
is low. The instrument can therefore not be considered reliable
enough for evaluations on a larger scale. Further work is necessary to
fine-tune the instrument and provide guidance for its usage.