Languages, automata, and logic
Handbook of formal languages, vol. 3
Introduction to the Theory of Computation
Introduction to the Theory of Computation
Mona: Monadic Second-Order Logic in Practice
TACAS '95 Proceedings of the First International Workshop on Tools and Algorithms for Construction and Analysis of Systems
A survey on tree edit distance and related problems
Theoretical Computer Science
Introduction to Automata Theory, Languages, and Computation (3rd Edition)
Introduction to Automata Theory, Languages, and Computation (3rd Edition)
JFLAP: An Interactive Formal Languages and Automata Package
JFLAP: An Interactive Formal Languages and Automata Package
Conditional Densities of Regular Languages
Electronic Notes in Theoretical Computer Science (ENTCS)
A survey of graph edit distance
Pattern Analysis & Applications
Regular Repair of Specifications
LICS '11 Proceedings of the 2011 IEEE 26th Annual Symposium on Logic in Computer Science
Symbolic automata: the toolkit
TACAS'12 Proceedings of the 18th international conference on Tools and Algorithms for the Construction and Analysis of Systems
Hi-index | 0.00 |
One challenge in making online education more effective is to develop automatic grading software that can provide meaningful feedback. This paper provides a solution to automatic grading of the standard computation-theory problem that asks a student to construct a deterministic finite automaton (DFA) from the given description of its language. We focus on how to assign partial grades for incorrect answers. Each student's answer is compared to the correct DFA using a hybrid of three techniques devised to capture different classes of errors. First, in an attempt to catch syntactic mistakes, we compute the edit distance between the two DFA descriptions. Second, we consider the entropy of the symmetric difference of the languages of the two DFAs, and compute a score that estimates the fraction of the number of strings on which the student answer is wrong. Our third technique is aimed at capturing mistakes in reading of the problem description. For this purpose, we consider a description language MOSEL, which adds syntactic sugar to the classical Monadic Second Order Logic, and allows defining regular languages in a concise and natural way. We provide algorithms, along with optimizations, for transforming MOSEL descriptions into DFAs and vice-versa. These allow us to compute the syntactic edit distance of the incorrect answer from the correct one in terms of their logical representations. We report an experimental study that evaluates hundreds of answers submitted by (real) students by comparing grades/feedback computed by our tool with human graders. Our conclusion is that the tool is able to assign partial grades in a meaningful way, and should be preferred over the human graders for both scalability and consistency.