AUTOMATED BLOOM’S TAXONOMY-BASED QUESTION GENERATION FOR COURSE OUTCOME ATTAINMENT IN OBE FRAMEWORKS
DOI:
https://doi.org/10.29121/shodhkosh.v5.i5.2024.6108Keywords:
Question Generation, Outcome-Based Education (OBE), Course Outcomes (COs), Retrieval-Augmented Generation (RAG) Model, Large Language Model (LLM)Abstract [English]
The creation of assessment questions that align with Bloom's taxonomy levels and achieve Course Outcomes (COs) is a critical yet complex task in Outcome-Based Education (OBE). Traditional manual methods, reliant on subject experts, are time-consuming and prone to gaps in addressing all COs or Bloom's levels. While Large Language Models (LLMs) like ChatGPT can generate questions, they lack access to private data, including prescribed textbooks and syllabi, potentially leading to questions beyond the scope of the curriculum. This paper presents a novel system leveraging Retrieval-Augmented Generation (RAG) to automate the generation of Bloom's taxonomy-based questions within the syllabus scope, ensuring comprehensive CO attainment. The proposed system integrates a vector database to store private data, including scanned textbooks, syllabi, Bloom's taxonomy levels, and COs. The RAG model, trained on this curated dataset, generates questions that fulfill the cognitive, psychomotor, and affective domain requirements specified in the syllabus. This approach not only ensures alignment with educational objectives but also significantly reduces the manual effort involved in question preparation. The system's efficacy is demonstrated through its ability to produce high-quality, targeted questions that effectively support OBE evaluation and enhance educational quality. This innovation addresses a critical gap in automated question generation for modern education systems.
References
Abd Rahim, T. N. T., Abd Aziz, Z., Ab Rauf, R. H., & Shamsudin, N. (2017, November). Automated exam question generator using genetic algorithm. In 2017 IEEE Conference on e-Learning, e-Management and e-Services (IC3e) (pp. 12-17). IEEE. DOI: https://doi.org/10.1109/IC3e.2017.8409231
Akram Sawiras, K. (2024). Evaluation and Development of Innovative NLP Techniques for Query-Focused Summarization Using Retrieval Augmented Generation (RAG) and a Small Language Model (SLM) in Educational Settings.
Butterfuss, R., & Doran, H. (2024). An application of text embeddings to support alignment of educational content standards. Educational Measurement: Issues and Practice. DOI: https://doi.org/10.1111/emip.12641
Chan, Y. H., & Fan, Y. C. (2019, November). A recurrent BERT-based model for question generation. In Proceedings of the 2nd workshop on machine reading for question answering (pp. 154-162). DOI: https://doi.org/10.18653/v1/D19-5821
Dhainje, S., Chatur, R., Borse, K., & Bhamare, V. (2018). An automatic question paper generation: using Bloom’s taxonomy. International Research Journal of Engineering and Technology, 5.
Gnanasekaran, D., Kothandaraman, R., & Kaliyan, K. (2021). An Automatic Question Generation System Using Rule-Based Approach in Bloom’s Taxonomy. Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science), 14(5), 1477-1487. DOI: https://doi.org/10.2174/2213275912666191113143335
Henkel, O., Levonian, Z., Li, C., & Postle, M. (2024). Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human preference. In Proceedings of the 17th International Conference on Educational Data Mining (pp. 315-320).
Hussein, H., Elmogy, M., & Guirguis, S. (2014). Automatic english question generation system based on template driven scheme. International Journal of Computer Science Issues (IJCSI), 11(6), 45.
Kukreja, S., Kumar, T., Bharate, V., Purohit, A., Dasgupta, A., & Guha, D. (2023, December). Vector Databases and Vector Embeddings-Review. In 2023 International Workshop on Artificial Intelligence and Image Processing (IWAIIP) (pp. 231-236). IEEE. DOI: https://doi.org/10.1109/IWAIIP58158.2023.10462847
Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2020). A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30, 121-204. DOI: https://doi.org/10.1007/s40593-019-00186-y
Levonian, Z., Li, C., Zhu, W., Gade, A., Henkel, O., Postle, M. E., & Xing, W. (2023). Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human preference. arXiv preprint arXiv:2310.03184.
Mohandas, M., Chavan, A., Manjarekar, R., Karekar, D., Qing, L., & Byeong Man, K. (2015). Automated question paper generator system. International Journal of Advanced Research in Computer and Communication Engineering, 4(12), 2278-1021.
Revathi, A. S., & Modi, N. A. (2021, March). Comparative analysis of text extraction from color images using tesseract and opencv. In 2021 8th International Conference on Computing for Sustainable Global Development (INDIACom) (pp. 931-936). IEEE.
Seo, J., Lee, S., Liu, L., & Choi, W. (2022). TA-SBERT: token attention sentence-BERT for improving sentence representation. IEEE Access, 10, 39119-39128. DOI: https://doi.org/10.1109/ACCESS.2022.3164769
Singh, H., Mohammad, S., Yaseen, A., Molawade, M., Mohite, S. G., Jadhav, V., & Jadhav, R. (2024, April). Multilingual Education through Optical Character Recognition (OCR) and AI. In 2024 MIT Art, Design and Technology School of Computing International Conference (MITADTSoCiCon) (pp. 1-6). IEEE. DOI: https://doi.org/10.1109/MITADTSoCiCon60330.2024.10575015
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023). Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering. Transactions of the Association for Computational Linguistics, 11, 1-17. DOI: https://doi.org/10.1162/tacl_a_00530
Smelyakov, K., Karachevtsev, D., Kulemza, D., Samoilenko, Y., Patlan, O., & Chupryna, A. (2020, October). Effectiveness of preprocessing algorithms for natural language processing applications. In 2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T) (pp. 187-191). IEEE. DOI: https://doi.org/10.1109/PICST51311.2020.9467919
Suryadjaja, P. S., & Mandala, R. (2021, September). Improving the performance of the extractive text summarization by a novel topic modeling and sentence embedding technique using SBERT. In 2021 8th International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA) (pp. 1-6). IEEE. DOI: https://doi.org/10.1109/ICAICTA53211.2021.9640295
Vieira, S. M., Kaymak, U., & Sousa, J. M. (2010, July). Cohen's kappa coefficient as a performance measure for feature selection. In International conference on fuzzy systems (pp. 1-8). IEEE. DOI: https://doi.org/10.1109/FUZZY.2010.5584447
Wang, T., Yuan, X., & Trischler, A. (2017). A joint model for question answering and question generation. arXiv preprint arXiv:1706.01450.
Yu, D. (2016). Softmax function based intuitionistic fuzzy multi-criteria decision making and applications. Operational Research, 16, 327-348. DOI: https://doi.org/10.1007/s12351-015-0196-7
Žitko, B., & Ljubić, H. (2021). Automatic question generation using semantic role labeling for morphologically rich languages. Tehnički vjesnik, 28(3), 739-745. DOI: https://doi.org/10.17559/TV-20200402175619
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Blessy Paul P, Cini Kurian, John T Abraham
This work is licensed under a Creative Commons Attribution 4.0 International License.
With the licence CC-BY, authors retain the copyright, allowing anyone to download, reuse, re-print, modify, distribute, and/or copy their contribution. The work must be properly attributed to its author.
It is not necessary to ask for further permission from the author or journal board.
This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge.
