Gradebot speeds up grading experience

The notion of a future where learning environments are dictated by machines is an uneasy one at best. However, the Gradebot, a new automated grading program developed by UC Davis Computer Science Associate Professor Hao Chen, is being used to reduce an ever-increasing workload for professors and faculty and provide more time for one-on-one interaction with students.

“The Gradebot is a tool and a service which allows the system to grade a massive number of homework assignments quickly,” Chen said. “Whenever the student submits a homework assignment, the Gradebot will grade it immediately and will provide feedback to the student, so the student can receive immediate feedback.”

When a student registered in either Engineering Computer Science (ECS) 30, 40 or 60 needs to complete programming homework, they can log on to the Gradebot website, gradebot.org, and work on it an unlimited amount of times before the deadline. Each time a student submits a program, Gradebot runs testing scripts on the work and immediately gives the student feedback on which test cases their programs passed and failed.

“You can get a lot of the tests right, but if one of the tests is wrong and you keep on changing your program and it doesn’t work, it’s frustrating,” said third-year managerial economics major and ECS 30 student Vinzent Davies. “I feel like Gradebot is a small child that is really picky about what they eat. Gradebot won’t eat your program unless it’s exactly what it wants.”

In winter of 2011, Chen spent two weeks creating the core functions of the Gradebot, mainly for the purpose of combating increased enrollment in computer science and engineering classes. This, paired with budget cuts, has decreased the department’s teaching assistant count as well, creating an even larger problem.

“I taught programming classes and these are the problems that I observed myself, and so those became the motivation for developing the Gradebot,” Chen said. “When these problems happen, students are not happy and I’m not happy. I wanted to make the classes more efficient and more effective.”

An example of a large class in need of teaching assistants can be seen in computer science professor Premkumar Devanbu’s ECS 30 class, in which the 225 students are given two to five programming assignments each week. Before Devanbu became one of the first professors to use the automated grading program in his classroom last quarter, his teaching assistants were spending 20 to 25 hours grading 1,000 programs every week.

“I think it’s a perfect tool to solve a pressing problem. On one hand, we have the budget cuts, so we have fewer and fewer instructional resources. On the other hand, we have more and more students,” Chen said. “The Gradebot is the perfect tool to solve both of these problems at once.”

When students have a problem with a programming assignment on Gradebot, they are encouraged to seek help from undergraduate tutors, teaching assistants and their professors, who now all have more time for these interactions. Davies said that with Gradebot in his ECS 30 class, he has collaborated with his fellow students and outside help more than ever.

“Face-time is really important, especially in these big classes and for freshmen that are new to this whole university experience,” Devanbu said. “I’m grateful for Gradebot because it promotes this opportunity for one-on-one interaction with the students.”

For some staff at UC Davis, the Gradebot demonstrates a huge step forward in the development of a detailed and structured grading system, in which students benefit from innovative teaching strategies for large class sizes.

“If you use an online teaching tool well, it can really improve teaching and learning,” said Rosemary Capps, assistant director at the Center for Excellence in Teaching and Learning. “Auto-graders are really good at working on assignments where details matter and details need to be exactly right.”

In fact, the program is still evolving to fit the needs of both students and faculty. Since its inception, Chen has been adding features to the Gradebot, specifically in securing the system from outside threats, including blocking cheating between students.

Other features of the current Gradebot include the teacher’s ability to track their students’ homework progress, a scoreboard for students to see how their peers are performing in the class, designed to encourage competition, as well as program plagiarism recognition.

The newest teaching method added to the program, to be implemented in Devanbu’s ECS 30 class in the coming weeks, is the in-class quick quiz, where students submit a short assignment at the beginning of a class period. With the immediate results reported to the instructor, lectures can be modified to fit the students’ needs.

Although Gradebot is quick to give students feedback on their work, it is not yet fully designed to say which parts of their programs specifically have errors. Rather, the results will tell the student what test cases they failed, and it is up to the student to figure out another way to solve the problem.

Devanbu said he thinks developing a part of the Gradebot to diagnose student failures by differential analysis will take one to two years to happen.

Integrating the Gradebot’s techniques into other teaching methods, such as the flipped model of learning, has been of particular interest for the future.

According to Capps, students traditionally come into class to learn new information, and then are responsible for processing that information after the class session. However, in the flipped model, students get new information before they come to class via online resources. They then come into class to process the new information with their professor and peers.

“Faculty can use the in-class time to model the ways that colleagues in their field approach projects and then to coach students in practicing those approaches,” Capps said. “The students can use their in-class time to work on complex problems and ask questions to their teacher and peers.”

Even though the flipped model has proven to be successful at many universities across the country, Devanbu said he is skeptical of replacing all lectures with online videos and homework assignments, and would rather have online resources complement students in the classroom.

Thinking about the evolution of teaching and how Gradebot can have a direct role in any coming transitions is important to Chen, but for now, he is focused on honing Gradebot into something that can assist the UC Davis community in the coming years, on a more short-term scale of time.

Both Chen and Devanbu are hopeful for the future of Gradebot in their department, as well as its spillover effect into other colleges and departments on campus.

“As long as the answers can be graded objectively, such as programs or clicks or multiple choices, then they can use the Gradebot,” Chen said. “I would love other people to use it. The more people who use it, the more kinks can be found and so that it can be improved.”

RITIKA IYER can be reached at features@theaggie.org.