Autograding is a valuable mechanism for assessment which facilitates scaling a curriculum to a much broader audience. Perhaps the best example can be found in the case of massive open online courses through platforms like Coursera and edX. However, developing autograders for lively, blocks-based programming environments can pose a number of unique challenges. These include the lack of clearly defined start state, often visual program output, and required user-interaction. How can we automatically grade assignments in these sorts of environments? What types of assignments can be graded easily?
Furthermore, as autograders are certainly useful for educators, it would also be compelling to be able to empower the educators to configure autograders for their curriculum. This would not only assist educators with grading student assignments but also could facilitate sharing curriculum as the teachers could reuse and “remix” both the curriculum and the corresponding autograders.
Blocks-based programming environments can certainly express rich concepts and powerful ideas as evident by their use for mobile app development, stitching, 3D printing, networking, and many others. What if they could also be used to design autograders?
In this lightning talk, we will discuss the challenges of autograding in lively, blocks-based programming environments and some of our recent work facilitating the creation of custom autograders for NetsBlox assignments within NetsBlox itself.