![VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019)
ISSN(Online): 2581-7280 Article No. 10
PP 1-6
1
www.viva-technology.org/New/IJRI
An Adaptive Approach for Subjective Answer Evaluation
Vishal Bhonsle1
, Priya Sapkal2
, Dipesh Mukadam3
, Prof. Vinit Raut4
Department of Computer Engineering, Viva Institute of Technology, Virar
Abstract: In current academic environment, assignments and home works are very necessary so that students
can increase their final grades. This assignments and home works are checked manually by teachers, due to this
it consumes lots of time and efforts. Due to manual checking sometimes human error may occur which may affect
to student’s grades. Students may misplace their hard copies of assignments because of this they have to rewrite
it again. In order to overcome these problems, the proposed system will convert the manual work to digital, in
which student will submit their assignments to the system and the system will generate and assign appropriate
grades. In the proposed system, by using K Nearest Neighbor Algorithm, it will collect keywords check for the
similarity and will generate similarity score. It will also check the relation of the keyword with respect to sentence.
To comparing the keywords with synonyms and similar meaning words Semantic Similarity Measure algorithm
will be used. After getting the similarity score the grades are assigned accordingly.
Keywords – data mining, duplication, grading system, KNN, semantics, subjective evaluation.
1. INTRODUCTION
In most of the online answer assessment systems almost all of the questions are multiple choice questions
which can be easily evaluated using binary matching. But in our education system most of the question and
answers are not in multiple choice format rather they are long answers which cannot be easily evaluated using
binary matching. These long answers are written by students so these answers are bound to be different for each
student and in order to evaluate this kind of answers a subjective evaluation of the answers is needed. By subjective
evaluation means that the answer written by student should be logically right as each person can write same thing
in their own different way. Previous attempts to evaluate this type of answers only used keyword matching
approach for checking the correctness of the answer, which in on itself is not wrong approach but it doesn’t give
as accurate results as we can expect from a human who is checking the answers. So in order to increase this
accuracy the proposed system will incorporate a synonyms matching approach along with the previously
mentioned method, which is explained in section 3 of this paper. By using the proposed system in our education
system, the tremendous load of evaluating all the term-work of all the student will be reduced from teacher’s side.
Also it will help students in long run as they would have to go through the study material to give answers instead
of just copying the whole work without going through it. And this system will help students to get a fair evaluation
for their work on the quality of the efforts they put in it.
2. RELATED WORKS
An There are some automatic scoring systems found in literature. These system uses K- Nearest
Neighboring algorithm [3] [6] for subjective scoring. To assign the scores the main process involves generation
of supplied test set, split the answer text to the list of words or phrases and lastly apply KNN classifier. To check
the similarity, word matching and word ordering is done if word gets matched and have the same ordering the
result shows the similarity is high. As per the similarity scores are generated. W. Xueli, J. Zhiyong, Y. Dahai [10]
proposed a system where K-nearest neighboring algorithm based kernel method and attribute reduction is used
for characteristics simplicity and efficiency so that accurate result can obtain. By comparing the base KNN
algorithm and purpose algorithm by using four UCI dataset. Then the best result is shown that stepwise KNN
algorithm works better than the original KNN algorithm.
R. Goswami, S. Karmakar, A. Bisai, A.R. Pal proposed a system for a long answer evaluation that has
been done by using lexical and semantic measures [2]. In this user answers are paired or matched with the standard](https://cdn.statically.io/img/image.slidesharecdn.com/10-190607014534/85/An-Adaptive-Approach-for-Subjective-Answer-Evaluation-1-320.jpg)
![VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019)
ISSN(Online): 2581-7280 Article No. 10
PP 1-6
2
www.viva-technology.org/New/IJRI
answer which the examiner will give by using lexical similarity measure. The system resolves and find accuracy
according to human decision. Now in next step both the answers are compared using semantic similarity
synonymous words of the keywords from both the answers are taken from the semantic dictionary. K. Meena, R.
Lawrance proposes Assessment algorithm [4] for evaluation of detailed type answer. For reducing the size of
answer preprocessing is done. It includes pruning and stemming. After this clustering is done using self-organizing
map. The cosine similarity formula is applied in each cluster and checked for the similarity by comparing the
vectors. After getting the result the scores are assigned to the detailed type answers.
P.A.A. Dumarl, W.K.D Shanika, S.A.D. Pathinayake [5] proposed an E-learning systems which has gain
a greater attraction in the field of using online educational resources. This research study has based on developing
a system to optimize the evaluation of text based answers automatically and to overcome the issues which has not
been addressed by the existing systems. In this research it has been addressed structured and essay type questions.
By referencing to prior work, this project solution has come up with a total solution where the tutors can add
questions to the question-bank, creating assignments and get the final scores automatically. The learners are
facilitated in providing the answers for the assessments and get feedback immediately.
For duplication detection, R. Sutoyo, I. Ramadhani, A.D. Ardiatma [1] proposed two algorithms which
are used that is winnowing algorithm and web-based k-gram for detection of documents similarity. The work of
robin karp is to search for the substring pattern in text using hashing algorithm and k gram method. Work of
winnowing is that it uses the window containing the hash value. The smallest values of each winnow is selected.
Plagiarism level can be determined based on the degree of similarity in the use of words that appear in sequence
in document. E.M.B. Nagoudi, H. Cherroul, A. Alshehri proposed the two approaches that have been done to
assist uses in detecting plagiarism in Arabic natural language text. The first approach is on word embedding [12],
word alignment and words weighting so that semantic similarity relation among the text will be searched. The
second approach was on machine learning where the characters will get performed at the sentence level. After the
lexical feature is used to assist the detection task.
For Spell checking V.V. Bhaire, A.A. Jadhav, P.A. Pashte proposed a system using autosuggestion
technique [8] for spell checking and correction. The main goal of the spell checker is to provide unified treatment
of various spell correction. Y. Bassi, M. Alwani proposed ‘spell checking’ which is the process which in any text
document detects and suggest corrections for incorrectly spelled words. This paper proposes context-sensitive
spelling correction method [7] for detection along with correction for misspelled words in digital text documents.
The proposed method comprises an error detector that detects misspellings, a candidate spellings generator based
on a character 2-gram model that generates correction suggestions, and an error corrector that performs contextual
error correction. F Ahmed, E. Willian De Luca, A. Nürnberger [9] proposed a system called ‘language-
independent spell-checker’ is a tool that is based on an enhanced n-gram model. N. Gupta, P. Mathur [11]
proposed a system in which various spell checking techniques are used to find the text with error. In developing
upon the type of error, non-word error and real word error. If error is detected for any word. In the sentence spell
checker search the proper word or matches the word from the dictionary that resemble the error of the word.
3. PROPOSED METHODOLOGY
From the above literature study the system is proposed for subjective answer evaluation. The system
first receives solution sets from admin and student’s answers. Then stop words are removed from them in order
to generate keywords. After keyword generation it checks for similarity and by calculating similarity it also checks
the relation of the keywords along with sentences with the sentences in the dataset which finds the exact similarity
and correctness of the sentence with the datasets. If the sentences match with the datasets it generates similarity
score as per the overlapping percentage. It also checks the synonyms and similar words before relating the
keywords in order to increase the accuracy of the overlap. Data duplication technique is used to compare the
previous answers submitted by students and on the basis of uniqueness of answers, grades are generated. The
proposed system is explained using fig 1, fig 2 and fig 3.](https://cdn.statically.io/img/www.slideshare.net/data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
More Related Content
What's hot
What's hot (20)
Similar to An Adaptive Approach for Subjective Answer Evaluation
Similar to An Adaptive Approach for Subjective Answer Evaluation (20)
More from vivatechijri
More from vivatechijri (20)
Recently uploaded
Recently uploaded (20)
An Adaptive Approach for Subjective Answer Evaluation
- 1. VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019) ISSN(Online): 2581-7280 Article No. 10 PP 1-6 1 www.viva-technology.org/New/IJRI An Adaptive Approach for Subjective Answer Evaluation Vishal Bhonsle1 , Priya Sapkal2 , Dipesh Mukadam3 , Prof. Vinit Raut4 Department of Computer Engineering, Viva Institute of Technology, Virar Abstract: In current academic environment, assignments and home works are very necessary so that students can increase their final grades. This assignments and home works are checked manually by teachers, due to this it consumes lots of time and efforts. Due to manual checking sometimes human error may occur which may affect to student’s grades. Students may misplace their hard copies of assignments because of this they have to rewrite it again. In order to overcome these problems, the proposed system will convert the manual work to digital, in which student will submit their assignments to the system and the system will generate and assign appropriate grades. In the proposed system, by using K Nearest Neighbor Algorithm, it will collect keywords check for the similarity and will generate similarity score. It will also check the relation of the keyword with respect to sentence. To comparing the keywords with synonyms and similar meaning words Semantic Similarity Measure algorithm will be used. After getting the similarity score the grades are assigned accordingly. Keywords – data mining, duplication, grading system, KNN, semantics, subjective evaluation. 1. INTRODUCTION In most of the online answer assessment systems almost all of the questions are multiple choice questions which can be easily evaluated using binary matching. But in our education system most of the question and answers are not in multiple choice format rather they are long answers which cannot be easily evaluated using binary matching. These long answers are written by students so these answers are bound to be different for each student and in order to evaluate this kind of answers a subjective evaluation of the answers is needed. By subjective evaluation means that the answer written by student should be logically right as each person can write same thing in their own different way. Previous attempts to evaluate this type of answers only used keyword matching approach for checking the correctness of the answer, which in on itself is not wrong approach but it doesn’t give as accurate results as we can expect from a human who is checking the answers. So in order to increase this accuracy the proposed system will incorporate a synonyms matching approach along with the previously mentioned method, which is explained in section 3 of this paper. By using the proposed system in our education system, the tremendous load of evaluating all the term-work of all the student will be reduced from teacher’s side. Also it will help students in long run as they would have to go through the study material to give answers instead of just copying the whole work without going through it. And this system will help students to get a fair evaluation for their work on the quality of the efforts they put in it. 2. RELATED WORKS An There are some automatic scoring systems found in literature. These system uses K- Nearest Neighboring algorithm [3] [6] for subjective scoring. To assign the scores the main process involves generation of supplied test set, split the answer text to the list of words or phrases and lastly apply KNN classifier. To check the similarity, word matching and word ordering is done if word gets matched and have the same ordering the result shows the similarity is high. As per the similarity scores are generated. W. Xueli, J. Zhiyong, Y. Dahai [10] proposed a system where K-nearest neighboring algorithm based kernel method and attribute reduction is used for characteristics simplicity and efficiency so that accurate result can obtain. By comparing the base KNN algorithm and purpose algorithm by using four UCI dataset. Then the best result is shown that stepwise KNN algorithm works better than the original KNN algorithm. R. Goswami, S. Karmakar, A. Bisai, A.R. Pal proposed a system for a long answer evaluation that has been done by using lexical and semantic measures [2]. In this user answers are paired or matched with the standard
- 2. VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019) ISSN(Online): 2581-7280 Article No. 10 PP 1-6 2 www.viva-technology.org/New/IJRI answer which the examiner will give by using lexical similarity measure. The system resolves and find accuracy according to human decision. Now in next step both the answers are compared using semantic similarity synonymous words of the keywords from both the answers are taken from the semantic dictionary. K. Meena, R. Lawrance proposes Assessment algorithm [4] for evaluation of detailed type answer. For reducing the size of answer preprocessing is done. It includes pruning and stemming. After this clustering is done using self-organizing map. The cosine similarity formula is applied in each cluster and checked for the similarity by comparing the vectors. After getting the result the scores are assigned to the detailed type answers. P.A.A. Dumarl, W.K.D Shanika, S.A.D. Pathinayake [5] proposed an E-learning systems which has gain a greater attraction in the field of using online educational resources. This research study has based on developing a system to optimize the evaluation of text based answers automatically and to overcome the issues which has not been addressed by the existing systems. In this research it has been addressed structured and essay type questions. By referencing to prior work, this project solution has come up with a total solution where the tutors can add questions to the question-bank, creating assignments and get the final scores automatically. The learners are facilitated in providing the answers for the assessments and get feedback immediately. For duplication detection, R. Sutoyo, I. Ramadhani, A.D. Ardiatma [1] proposed two algorithms which are used that is winnowing algorithm and web-based k-gram for detection of documents similarity. The work of robin karp is to search for the substring pattern in text using hashing algorithm and k gram method. Work of winnowing is that it uses the window containing the hash value. The smallest values of each winnow is selected. Plagiarism level can be determined based on the degree of similarity in the use of words that appear in sequence in document. E.M.B. Nagoudi, H. Cherroul, A. Alshehri proposed the two approaches that have been done to assist uses in detecting plagiarism in Arabic natural language text. The first approach is on word embedding [12], word alignment and words weighting so that semantic similarity relation among the text will be searched. The second approach was on machine learning where the characters will get performed at the sentence level. After the lexical feature is used to assist the detection task. For Spell checking V.V. Bhaire, A.A. Jadhav, P.A. Pashte proposed a system using autosuggestion technique [8] for spell checking and correction. The main goal of the spell checker is to provide unified treatment of various spell correction. Y. Bassi, M. Alwani proposed ‘spell checking’ which is the process which in any text document detects and suggest corrections for incorrectly spelled words. This paper proposes context-sensitive spelling correction method [7] for detection along with correction for misspelled words in digital text documents. The proposed method comprises an error detector that detects misspellings, a candidate spellings generator based on a character 2-gram model that generates correction suggestions, and an error corrector that performs contextual error correction. F Ahmed, E. Willian De Luca, A. Nürnberger [9] proposed a system called ‘language- independent spell-checker’ is a tool that is based on an enhanced n-gram model. N. Gupta, P. Mathur [11] proposed a system in which various spell checking techniques are used to find the text with error. In developing upon the type of error, non-word error and real word error. If error is detected for any word. In the sentence spell checker search the proper word or matches the word from the dictionary that resemble the error of the word. 3. PROPOSED METHODOLOGY From the above literature study the system is proposed for subjective answer evaluation. The system first receives solution sets from admin and student’s answers. Then stop words are removed from them in order to generate keywords. After keyword generation it checks for similarity and by calculating similarity it also checks the relation of the keywords along with sentences with the sentences in the dataset which finds the exact similarity and correctness of the sentence with the datasets. If the sentences match with the datasets it generates similarity score as per the overlapping percentage. It also checks the synonyms and similar words before relating the keywords in order to increase the accuracy of the overlap. Data duplication technique is used to compare the previous answers submitted by students and on the basis of uniqueness of answers, grades are generated. The proposed system is explained using fig 1, fig 2 and fig 3.
- 3. VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019) ISSN(Online): 2581-7280 Article No. 10 PP 1-6 3 www.viva-technology.org/New/IJRI Fig. 1. Admin Model As shown in fig. 1, admin first inputs question and answers in the system, which are saved in database. Along with it stop words such as “a”, “an”, “the”, “but” etc. are removed and remaining answer is send to Answer Evaluation Section. Fig. 2. Student Model As shown in fig. 2 student types their answers as a input in the system, which are saved in database. Along with it, stop words such as “a”, “an”, “the”, “but” etc. are removed and remaining answer is send to Answer Evaluation Section.
- 4. VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019) ISSN(Online): 2581-7280 Article No. 10 PP 1-6 4 www.viva-technology.org/New/IJRI Fig. 3. System Flow In this system, firstly the solution set from admin and student answers are collected in text format from this solution sets stop words are removed from both administrators as well as student’s solution set. After that keywords are extracted from the remaining words. While keyword matching, keywords are matched with both the sets along with this relation of the keywords are analysed using K- Nearest Neighboring algorithm. The remaining words which are not matched in previous steps are checked for synonyms and similar meaning words using semantic similarity measuring algorithm. For duplication checking which is based on timestamp provided to each answers submitted by students the solution of the students are checked. Depending on the duplication level and the similarity score generated by KNN and Semantic Analysis Grades are assigned to each student. A detailed report of the solution of the student is generated by system and is saved in .pdf format also this report is sent to respective student.
- 5. VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019) ISSN(Online): 2581-7280 Article No. 10 PP 1-6 5 www.viva-technology.org/New/IJRI 4. IMPLEMENTATION The proposed system is implemented in ASP.NET. The .NET Framework is a computing platform that simplifies application development in the highly distributed environment of the Internet. ASP .NET can use to develop applications ranging from traditional command-line or graphical user interface (GUI) applications to applications based on the latest innovations provided by ASP.NET, such as Web Forms and XML Web services. 5. RESULT AND ANALYSIS The below Table 1, shows the results of the Features analysis on various parameters for Subjective Answer Evaluation. The table shows that the proposed system derives better results than all the systems referred so far. TABLE 1. FEATURES ANALYSIS ON VARIOUS PARAMETERS FOR SUBJECTIVE ANSWER EVALUATION Features Keywords Extraction Synonyms mapping Duplication checking K.Sriwanna [3] More Not Available Not Available R. Goswami [2] More More Not Available P.A.A Dumal [5] Less More Not Available Proposed system More More Available 6. CONCLUSION AND FUTURE SCOPE In this paper, we have studied various methods for evaluating answers which contained various keyword matching algorithms, semantics approach toward the problem as well as different duplication detection method to further advanced the system for subjective answer evaluation. As for evaluation of any long type answer, only keyword matching is not sufficient but the relation of those keywords for their given sentence should be subjective or in relation with each other. Among all these K-Nearest Neighboring algorithm along with semantic similarity measure gives better accuracy for evaluating subjective type answers. As in this system, while pre-processing most of the notations and symbols are removed it becomes easy to overlap the text but in case of mathematical problems the pre-processing removes most of the important symbols and features. So for mathematical questions it gives poor results as compared to textual format questions. This system can be implemented for an education institutes for evaluation of term work of students. As manually checking all students term work can be very hectic work for teachers. In future work, the system should be able to evaluate the mathematical type content and should give the score. REFERENCES [1] R. Sutoyo, I. Ramadhani, A.D. Ardiatma, “Detecting Documents Plagiarism using Winnowing Algorithm and K-Gram Method”, IEEE International Conference on Cybernetics and Computational Intelligence 2017. [2] R. Goswami, S. Karmakar, A. Bisai, A.R. Pal, “A Knowledge based Approach for Long Answer Evaluation”, 2017 2nd IEEE International Conference On Recent Trends in Electronics Information & Communication Technology (RTEICT). [3] K. Sriwanna, “Text Classification for Subjective Scoring Using K-Nearest Neighbours”, The 3rd International Conference on Digital Arts, Media and Technology (ICDAMT2018). [4] K. Meena, R. Lawrance, “Semantic Similarity Based Assessment of Descriptive Type Answers”, 2016. [5] P.A.A Dumal, W.K.D Shanika, S.A.D Pathinayake, T.C. Sandanayake, “Adaptive and Automated Online Assessment Evaluation System”, 2017 11th International Conference on Software, Knowledge, Information Management and Applications (SKIMA). [6] Y. Zhao, Y. Qian, C. Li, “Improved KNN Text Classification Algorithm with MapReduce Implementation”, The 2017
- 6. VIVA-Tech International Journal for Research and Innovation Volume 1, Issue 2 (2019) ISSN(Online): 2581-7280 Article No. 10 PP 1-6 6 www.viva-technology.org/New/IJRI 4th International Conference on Systems and Informatics (ICSAI 2017). [7] Y. Bassil, M. Alwani, “Context-sensitive Spelling Correction Using Google Web 1T 5-Gram”, LACSC - Lebanese Association for Computational Sciences, Beirut, Lebanon. [8] V. Bhaire, A. Jadhav, P. Pashte, P.G. Magdum, “Spell checker”, International Journal of Scientific and Research Publications, April 2015. [9] F. Ahmed, E.W. De Luca, A. Nürnberger, “Revised N-Gram based Automatic Spelling Correction Tool to Improve Retrieval Effectiveness”, 2009. [10] W. XueliˈJiang, Z. Dahai, “An Improved KNN algorithm Based on Kernel Methods and Attribute Reduction”, 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control. [11] N. Gupta, P. Mathur, “Spell Checking Techniques in NLP”, International Journal of Advanced Research in Computer Science and Software Engineering, 2012. [12] E.M.B. Nagoudi, H. Cherroun, A. Alshehri, “Disguised Plagiarism Detection in Arabic Text Documents”, 2018. 1.