A Cognitive Model and Gaze-Based Evaluation of Multiple Representation use during Program Comprehension and Debugging

Abstract

Integrated Development Environments (IDE) for software development offer a wide variety of software visualization tools, which facilitate navigation and analysis, and generate multiple graphical presentations of program code. Co-ordination of these representations during program comprehension can be a complex task for a novice programmer, and at times be detrimental to debugging performance. This thesis develops a cognitive model of how multiple representations including visualizations are used by programmers to comprehend and debug a program in an IDE for object oriented programming. The model, based on literature review and analyses of the shortcomings of existing research, is more detailed than any model of program comprehension and debugging hitherto offered in the literature. The model was evaluated empirically with two debugging studies during which visual attention of participants was tracked with an eye-tracker. The first study found that a mental model created by static visualizations is not as extensive as the mental model created by dynamic visualizations. Mental model strength of programming constructs like data structure and function was consistently higher for dynamic visualizations, whereas strength of control flow and data flow was consistently stronger for static visualizations. On analyzing visual attention patterns during the second study, we found that program code and dynamic representations (viewer, variable watch and output) attracted the most attention. Static representations like UML and Control Structure diagrams saw significantly lesser usage. Gaze patterns were analyzed by breaking down the debugging sessions into segments of three, five and fifteen minute intervals, and classifying gaze durations as short and long gazes. Data mining techniques were used to detect high frequency patterns from eye tracking data of participants. A significant pattern difference was found among the participants based on programming experience, familiarity with the IDE and debugging performance. These results are consistent with the proposed cognitive model and open up many more intriguing questions for future research.