A multi-institutional, multinational study of programming concepts using card sort data


Brad Richards, University of Puget SoundFollow
Kate Sanders, Mathematics and Computer Science, Rhode Island College, USA
Dennis Bouvier, Department of Electrical and Computer Engineering, Parks College of Engineering, Aviation, and Technology, Saint Louis University, USA
Sally Fincher, Computing Laboratory, University of Kent, UK
Gary Lewandowski, Mathematics and Computer Science Department, Xavier University, Cincinnati, USA
Briana Morrison, Department of Computer Science, Southern Polytechnic State University, USA
Laurie Murphy, Computer Science and Computer Engineering Department, Pacific Lutheran University, USA
Marian Petre, Computing Department, Open University, UK
Josh Tenenberg, Computing and Software Systems, Institute of Technology, University of Washington, Tacoma, USA
Lynda Thomas, Department of Computer Science, University of Wales, Aberystwyth, UK
Richard Anderson, Computer Science and Engineering, University of Washington, Seattle, USA
Ruth Anderson, Department of Computer Science, University of Virginia, USA
Sue Fitzgerald, Information and Computer Sciences, Metropolitan State University, USA
Alicia Gutschow, Information Technology and Computer Science, Blue Ridge Community College, Virginia, USA
Susan Haller, Computer Science Department, University of Wisconsin – Parkside, USA
Raymond Lister, Department of Software Engineering, University of Technology, Sydney, Australia
Renée McCauley, Department of Computer Science, College of Charleston, USA
John McTaggart, Department of Mathematics and Computer Science, Drake University, USA
Christine Prasad, School of Computing and Information Technology, UNITEC, New Zealand
Terry Scott, Department of Mathematical Sciences, University of Northern Colorado, USA
Dermot Shinners-Kennedy, Department of Computer Science and Information Systems, University of Limerick, Ireland
Suzanne Westbrook, Department of Computer Science, University of Arizona, USA
Carol Zander, Computing and Software Systems, University of Washington, Bothell, USA

Document Type


Publication Date


Publication Title

Expert Systems


Mathematics and Computer Science


This paper presents a case study of the use of a repeated single-criterion card sort with an unusually large, diverse participant group. The study, whose goal was to elicit novice programmers' knowledge of programming concepts, involved over 20 researchers from four continents and 276 participants drawn from 20 different institutions. In this paper we present the design of the study and the unexpected result that there were few discernible systematic differences in the population. The study was one of the activities of the National Science Foundation funded Bootstrapping Research in Computer Science Education project (2003).