EVALUATING SEMANTIC ANALYSIS METHODS FOR SHORT ANSWER GRADING USING LINEAR REGRESSION
DOI:
https://doi.org/10.20319/pijss.2017.32.437450Keywords:
Semantic Analysis, Linear Regression, Automatic Grading, Automatic Short Answer GradingAbstract
The assessment of free-text answers may demand significant human effort, especially in scenarios with many students. This paper focuses on the automatic grading of short answer written in Portuguese language using techniques of natural language processing and semantic analysis. A previous study found that a similarity scoring model might be more suitable to a question type than to another. In this study, we combine latent semantic analysis (LSA) and a WordNet path-based similarity method using linear regression to predict scores for 76 short answers to three questions written by high school students. The predicted scores compared well to human scores and the use of combined similarity scores showed an improvement in overall results in relation to a previous study on the same corpus. The presented approach may be used to support the automatic grading of short answer using supervised machine learning to weight different similarity scoring models.
References
Burrows, S., Gurevych, I. & Stein, B. (2015). The eras and trends of automatic short answer grading International Journal of Artificial Intelligence in Education. International Journal of Artificial Intelligence in Education, 25(1), 60-117. https://doi.org/10.1007/s40593-014-0026-8
Fellbaum, C. (1998). WordNet: An electronic lexical database. Massachusetts: MIT Press.
Landauer, T. K. & Dutnais, S. T. (1997). A Solution to Plato’s Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. Psychological Review, 104(2), 211–240. https://doi.org/10.1037/0033-295X.104.2.211
Leacock, C., & Chodorow, M. (1998). Combining local context and WordNet sense similarity for word sense identification. In Fellbaum, C. (Ed.), WordNet: An electronic lexical database (pp. 265-284). Massachusetts: MIT Press.
Lin, D. (1998). An Information-Theoretic Definition of Similarity. Proceedings of International Conference on Machine Learning, 296–304.
Meng, L., Huang, R. & Gu, J. (2013). A Review of Semantic Similarity Measures in WordNet. International Journal of Hybrid Information Technology, 6(1), 1-12.
Mohler, M., & Mihalcea, R. (2009). Text-to-text semantic similarity for automatic short answer grading. In Proceedings of the 12th conference of the European Chapter of the Association for Computational Linguistics: 30 March-3 April 2009, Megaron Athens International Conference Centre, Athens, Greece (pp. 567-575). Stroudsburg, PA: Association for Computational Linguistics. https://doi.org/10.3115/1609067.1609130
Oliveira et al. (2015). As Wordnets do Português. Oslo Studies in Language, 7(1), 397-424.
Paiva, V., Rademaker, A., & Melo, G. (2012). OpenWordNet-PT: An Open Brazilian Wordnet for Reasoning. In Proceedings of COLING 2012: Demonstration Papers (pp. 353–360). Mumbai, India: The COLING 2012 Organizing Committee.
Passero, G., Filho, A. H., & Dazzi, R. (2016). Avaliação do Uso de Métodos Baseados em LSA e WordNet para Correção de Questões Discursivas. Proceedings of the 17th Brazilian Symposium on Computers in Education, 1136-1145. https://doi.org/10.5753/cbie.sbie.2016.1136
Pedersen, T., Patwardhan, S., & Michelizzi, J. (2004). WordNet::Similarity: Measuring the Relatedness of Concepts. Demonstration Papers at HLT-NAACL. https://doi.org/10.3115/1614025.1614037
Santos, J. C., & Favero, E. L. (2015). Practical use of a latent semantic analysis (LSA) model for automatic evaluation of written answers. Journal of the Brazilian Computer Society, 21(1). https://doi.org/10.1186/s13173-015-0039-7
Silva, W. D. C. M. (2013). Aprimorando o corretor gramatical CoGrOO. Dissertação de Mestrado, Instituto de Matemática e Estatística, Universidade de São Paulo, São Paulo. doi:10.11606/D.45.2013.tde-02052013-135414. Retrieved from: www.teses.usp.br https://doi.org/10.11606/D.45.2013.tde-02052013-135414
Widdows, D., & Ferraro, K. (2008). Semantic Vectors: A Scalable Open Source Package and Online Technology Management Application. Proceedings of the 6th International Conference on Language Resources and Evaluation, 1183-1190.
Wu, Z., & Palmer, M. (1994). Verb semantics and lexical selection. Proceedings of the 32nd annual meeting on Association for Computational Linguistics (pp. 133-138). https://doi.org/10.3115/981732.981751
Ziai, R., Ott, N., & Meurers, D. (2012). Short Answer Assessment: Establishing Links Between Research Strands. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP (pp. 190-200). Montreal, Canada.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2017 Authors
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
