Eye Catching Technology: Eye Tracking Studies Help Future Programming Students

By Alyssa Pawluk

YSU students participated in a study that uses a remote eye tracker to track eye movement using infrared light. The image shows a Java program with a student’s eye gaze placed on a heat map to show where he or she looked when viewing source code. Photo courtesy of Bonita Sharif.

Bonita Sharif, assistant professor in the department of computer science and information systems at Youngstown State University, has been researching and developing eye tracking technology to benefit computer science educators and industry professionals alike.

Sharif, with the help of graduate and undergraduate students at YSU and academic groups in Switzerland, Germany and Finland, has been collecting data and conducting studies since 2009.

Eye tracking technology uses a hardware device — aptly named an eye tracker — capable of determining where a person is looking on a screen.

These remote hardware devices connect to a computer and operate using infrared light and a camera to track where participant’s eyes move while they perform tasks, such as computer program comprehension or computer bug fixes.

Six Years of Research

Sharif and her team have used eye tracking devices at YSU and Kent State University. Sharif explained where these eye tracking studies all began.

“This started when I was a graduate student at Kent State University and I was working with my adviser Dr. Jonathan Maletic. I basically did a study there that tried to compare if one type of identifier style was better than the other,” Sharif said. “We did a very small study on that, and it kind of branched from there.”

Sharif’s most recent study began in July of 2014 concerning iTrace, a new comprehensive software traceability environment developed by Sharif and her colleagues that utilizes eye tracking technology and theories to make software engineering simpler for developers. The technology provides recommendations based on the movements of developers’ eyes during development.

“The project that I’m currently working on is called iTrace, and that originated from our Software Engineering Research and Empirical Studies Lab in collaboration with Huzefa Kagdi at Wichita State University. We are looking at incorporating eye tracking within the working environment of a developer. … His is an ongoing research area,” Sharif said.

Altogether, Sharif and her collaborators have published around 10 peer-reviewed conference and journal studies since 2009.

Sharif said that among the studies conducted, one focused on how developers fixed bugs on computers and how they determined errors in computer code.

“The second study we did was we looked at some snippets of code and tried to see how long it takes for a person to find a defect in that code. That was published in 2012, and we found that the typical person who is an expert would read the entire code once and go directly to the place that had the defect. So that was the scan time, and we measured scan time with respect to finding source code defects,” Sharif said.

The most recently submitted paper for review was a comparison of beginning programmers and computer programming experts on how each one differentiates between the English language and source code on a computer.

“We saw that novices read code the same way as natural language. When you are really a novice, you don’t differentiate between English and source code. Experts do. It’s a huge difference, and they literally jump around in important parts, but novices just read it from top to bottom, left to right,” Sharif said. “Obviously, we need to train students to learn to read code right from the get go in a certain way, rather than them learning in an x amount of years. I don’t know how many times I’ve heard students say ‘Oh, if I only knew that in my first class of programming, I would have done so much better.’ That’s what we are trying to do with eye tracking — increase the students’ awareness of their code reading behavior.”

Eye Tracking’s Impact

Sharif said eye tracking technology would help experts produce curricula for students in the computer science and computer information technology departments.

“So there are two aspects. One is helping the developer in their work — and that is more practitioner based — and the other is pedagogy. Pedagogy is informing the curriculum. It’s how we use eye tracking as an evidence-based method to inform our curriculum. Should we change the way we teach programming because novices look at things entirely different? Should we ask them to look at specific things?” Sharif said.

She added that she would like to see more beginners and students in computer studies using the technology to help themselves.

“We want more people to use it as an evaluation tool, but not just an evaluation tool, but incorporate it as part of their day. If we do that, I think we get a lot more information than we have ever gotten before. With eye tracking we know not only the end result — the answer given — but also the process used to get to the answer. What if someone uses the right process, but gets the final answer wrong? You used the right process, but you just missed the last step. Here, we can see it in the eye movements,” Sharif said.

Looking Toward the Future

The most recent study was conducted in the Software Engineering Research and Empirical Studies Lab at the university. Sharif said she and her colleagues are in the analysis process and will submit their results to a peer-reviewed conference in March.

Braden Walters, an undergraduate student majoring in computer science, helped Sharif, along with a graduate student John Meinken, to further the studies.

Walters said that his role in the most recent study was in developing the traceability software iTrace.

“I’ve taken a role in the development of our eye tracking traceability software iTrace, which was used in the studies and in conducting the studies. At the time, it was really a matter of who was available at the time we needed to collect data. I actually enjoy working with the collected data more than collecting it,” Walters said.

Although Sharif and her colleagues have conducted studies in recent years, she said that they have only scratched the surface in the computer science and software engineering fields.

“There is a lot more work to be done. It is exciting research that I hope more students can get involved in. Right now, this is a hot topic in software engineering and computing education,” Sharif said.

Walters expressed his excitement for the future possibilities that the research and studies could bring to the computer science and software engineering fields in academic universities.

“I feel excited to contribute to the development of new technology in general. Eye tracking isn’t my focus, but the contributions I’ve made to that field definitely excite me. Although I think it could take some time for our research to be used outside of academic contexts, I’m excited to see when that happens,” Walters said. “I intend to do future work in computer science research, but I doubt I’ll be making any notable contributions to this particular project after graduation, but that can’t be said with absolute certainty.”