CNGL: Grading Student Answers by Acts of Translation

Ergun Biçici and Josef van Genabith. CNGL: Grading Student Answers by Acts of Translation. In *SEM 2013: The Second Joint Conference on Lexical and Computational Semantics and Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), Atlanta, Georgia, USA, 14-15 June 2013. Association for Computational Linguistics. [WWW ] [PDF ] Keyword(s): Machine Translation, Machine Learning, Quality Estimation, Natural Language Processing. [Abstract] [bibtex-entry]

We invent referential translation machines (RTMs), a computational model for identifying the translation acts between any two data sets with respect to a reference corpus selected in the same domain, which can be used for automatically grading student answers. RTMs make quality and semantic similarity judgments possible by using retrieved relevant training data as interpretants for reaching shared semantics. An MTPP (machine translation performance predictor) model derives features measuring the closeness of the test sentences to the training data, the difficulty of translating them, and the presence of acts of translation involved. We view question answering as translation from the question to the answer, from the question to the reference answer, from the answer to the reference answer, or from the question and the answer to the reference answer. Each view is modeled by an RTM model, giving us a new perspective on the ternary relationship between the question, the answer, and the reference answer. We show that all RTM models contribute and a prediction model based on all four perspectives performs the best. Our prediction model is the $2$nd best system on some tasks according to the official results of the Student Response Analysis (SRA 2013) challenge.