IRT Method for Measuring The Quality of High School Mathematics Mid-Semester Assessment Questions in Magelang

Gunawan; Kana Hidayati

Authors

Gunawan Universitas Negeri Yogyakarta
Kana Hidayati Universitas Negeri Yogyakarta

Keywords:

assessment, irt methods, mathematics

Abstract

Learning assessment is an important part of the learning process. The quality of the assessments carried out must of course be proven, namely by using item analysis, one of which is analysis using the IRT (Item Response Theory) method. This research aims to determine the quality of the question items that have been designed in mathematics mid-semester assessment questions using the IRT method. The method used in this research is descriptive analysis with a quantitative approach. The research subjects were 143 class XI students at one of the Magelang Regency high schools. The research instrument used was mathematics mid-semester assessment questions in Multiple Choice form, totaling 20 questions. Data analysis was carried out using the technique of analyzing the characteristics of the questions based on the IRT method, consisting of two stages, namely the assumption test and the analysis test. Based on the results of the analysis, it was found that the items met the assumption tests, namely the unidimensional, local independence, and invariance tests. Based on the model suitability test, a 2-PL model was obtained that was suitable for use, namely analyzing the different power parameters and the difficulty level of the questions. Based on the results of the estimation analysis on the question item parameters, it was concluded that 16 questions met the criteria for good quality questions and 4 questions that met the criteria for deficient questions.

References

Aiken, L. R. (1994). Psychological testing and assessment (8th ed.). Allyn & Bacon.

Apriyani, D. C. N., Susanto, H. P., & Hidayat, T. (2023). Analysis of Pre-Olympic Middle School Mathematics Test Instruments Based on Item Response Theory. AlphaMath : Journal of Mathematics Education, 9(2), 145. https://doi.org/10.30595/alphamath.v9i2.18021

Archer, E. (2017). The Assessment Purpose Triangle: Balancing the Purposes of Educational Assessment. Frontiers in Education, 2(August), 1–7. https://doi.org/10.3389/feduc.2017.00041

Arlinwibowo, J., Achyani, I., & Kurniadi, G. (2021). Multidimentional Item Respose Utilization for Validating Mathematics National Examination in Indonesia. Journal of Physics: Conference Series, 1764(1). https://doi.org/10.1088/1742-6596/1764/1/012113

Ayanwale, M. A., Chere-Masopha, J., & Morena, M. C. (2022). The classical test or item response measurement theory: The ftatus of the framework at the examination council of Lesotho. International Journal of Learning, Teaching and Educational Research, 21(8), 384–406. https://doi.org/10.26803/ijlter.21.8.22

Baird, J. A., Andrich, D., Hopfenbeck, T. N., & Stobart, G. (2017). Assessment and learning: fields apart? Assessment in Education: Principles, Policy and Practice, 24(3), 317–350. https://doi.org/10.1080/0969594X.2017.1319337

Barnes, N., Fives, H., & Dacey, C. M. (2017). U.S. teachers’ conceptions of the purposes of assessment. Teaching and Teacher Education, 65, 107–116. https://doi.org/10.1016/j.tate.2017.02.017

Bichi, A. A. (2015). Comparison of Classical Test Theory and Item Response Theory: A Review of Empirical Studies Test Items Development and Analysis Using Item Response Theory View project. Australian Journal of Basic and Applied Sciences, 9(7), 549–556. https://doi.org/10.13140/RG.2.1.1561.5522

Black, P., & Wiliam, D. (2018). Classroom assessment and pedagogy. Assessment in Education: Principles, Policy and Practice, 25(6), 551–575. https://doi.org/10.1080/0969594X.2018.1441807

Cai, L., Choi, K., Hansen, M., & Harrell, L. (2016). Item response theory. Annual Review of Statistics and Its Application, 3, 297–321. https://doi.org/10.1146/annurev-statistics-041715-033702

Earl, L. M. (2013). Assessment as learning: Using classroom assessment to maximize student learning. Sage Publications.

Gardner, J. (2012). Assessment and learning. Sage Publications.

Gunawan, & Asria, L. (2023). Analisis butir soal Penilaian Tengah Semester ( PTS ) matematika kelas XI berdasarkan teori klasik. MATH LOCUS: Jurnal Riset Dan Inovasi Pendidikan Matematik, 4(1), 1–11.

Gyamfi, A., & Acquaye, R. (2023). Parameters and Models of Item Response Theory (IRT): A review of literature. Acta Educationis Generalis, 13(3), 68–78. https://doi.org/10.2478/atd-2023-0022

Halik, A. S., Mania, S., & Nur, F. (2019). Analisis Butir Soal Ujian Akhir Sekolah (Uas) Mata Pelajaran Matematika Pada Tahun Ajaran 2015/2016 Smp Negeri 36 Makassar. Al Asma : Journal of Islamic Education, 1(1), 11. https://doi.org/10.24252/asma.v1i1.11249

Hamimi, L., Zamharirah, R., & Rusydy, R. (2020). Analisis Butir Soal Ujian Matematika Kelas VII Semester Ganjil Tahun Pelajaran 2017/2018. Mathema: Jurnal Pendidikan Matematika, 2(1), 57. https://doi.org/10.33365/jm.v2i1.459

Hartono, W., Hadi, S., Rosnawati, R., & Retnawati, H. (2022). Uji Kecocokan Model Parameter Logistik Soal Diagnosa Kemampuan Matematika Dasar. JNPM (Jurnal Nasional Pendidikan Matematika), 6(1), 125. https://doi.org/10.33603/jnpm.v6i1.5899

Khairil, L. F., & Mokshein, S. E. (2018). 21st Century Assessment: Online Assessment. International Journal of Academic Research in Business and Social Sciences, 8(1), 659–672. https://doi.org/10.6007/ijarbss/v8-i1/3838

Kiliç, A. F., Koyuncu, İ., & Uysal, İ. (2023). Scale Development Based on Item Response Theory : A Systematic Review. 10(1), 209–223.

Kong, S., & Lai, M. (2022). Computers & Education Validating a computational thinking concepts test for primary education using item response theory : An analysis of students ’ responses. 187(May), 1–18.

Kusumayanti, A., & Jannah, N. (2022). Analisis butir soal ujian masuk mandiri UIN Alauddin Makassar dengan teori tes modern. MaPan: Jurnal Matematika Dan Pembelajaran, 10(1), 159–174.

Looney, A., Cumming, J., van Der Kleij, F., & Harris, K. (2018). Reconceptualising the role of teachers as assessors: teacher assessment identity. Assessment in Education: Principles, Policy and Practice, 25(5), 442–467. https://doi.org/10.1080/0969594X.2016.1268090

Mahlambi, S. B., Studies, I., Africa, S., & Mahlambi, S. B. (2022). Mathematics teachers ’ use of assessment for learning to promote classroom diversity of learners. Pythagoras-Journal of the Association for Mathematics Education of Africa, 1–9.

Memarian, B., & Doleck, T. (2024). Computers in Human Behavior : Artificial Humans A review of assessment for learning with artificial intelligence. Computers in Human Behavior: Artificial Humans, 2(1), 1–11. https://doi.org/10.1016/j.chbah.2023.100040

Na, C., Clarke-Midura, J., Shumway, J., van Dijk, W., & Lee, V. R. (2024). Validating a performance assessment of computational thinking for early childhood using item response theory. International Journal of Child-Computer Interaction, 40(May 2023), 100650. https://doi.org/10.1016/j.ijcci.2024.100650

O. A., A., & E. R. I., A. (2016). Comparative Analysis of Classical Test Theory and Item Response Theory Based Item Parameter Estimates of Senior School Certificate Mathematics Examination. European Scientific Journal, ESJ, 12(28), 263. https://doi.org/10.19044/esj.2016.v12n28p263

Özdemir, B. (2015). A Comparison of IRT-based Methods for Examining Differential Item Functioning in TIMSS 2011 Mathematics Subtest. Procedia - Social and Behavioral Sciences, 174, 2075–2083. https://doi.org/10.1016/j.sbspro.2015.02.004

Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1–11. https://doi.org/10.1080/2331186X.2017.1301013

Raj Acharya, B. (2017). Factors Affecting Difficulties in Learning Mathematics by Mathematics Learners. International Journal of Elementary Education, 6(2), 8. https://doi.org/10.11648/j.ijeedu.20170602.11

Retnawati, H. (2014). Teori respon butir dan penerapannya. Nuha Medika.

Retnawati, H., & Hadi, S. (2014). Sistem bank soal daerah terkalibrasi untuk menyongsong era desentralisasi. Jurnal Ilmu Pendidikan, 20(2), 183–193.

Sainuddin, S. (2018). Analisis Karakteristik Butir Tes Matematika Berdasarkan Teori Modern (Teori Respon Butir). Jurnal Penelitian Matematika Dan Pendidikan Matematika, 1(1), 1–12.

Santoso, A., Kartianom, K., & Kassymova, G. K. (2019). Kualitas butir bank soal statistika (Studi kasus: Instrumen ujian akhir mata kuliah statistika Universitas Terbuka). Jurnal Riset Pendidikan Matematika, 6(2), 165–176. https://doi.org/10.21831/jrpm.v6i2.28900

Stage, C. (2003). Classical test theory or item response theory: The Swedish experience. 42, 1–29.

Stemler, S. E., & Naples, A. (2021). Rasch measurement v. item response theory: Knowing when to cross the line. Practical Assessment, Research and Evaluation, 26, 1–16. https://doi.org/10.7275/v2gd-4441

Sumaryanta. (2021). Teori klasik & teori respon butir: Konsep & contoh penerapannya. CV. Confident.

Susanto, H., Rinaldi, A., & Novalia. (2015). Analisis validitas reabilitas tingkat kesukaran dan daya beda pada butir soal Ujian Akhir Semester ganjil mata pelajaran matematika. 3(2), 203–217.

Treiblmaier, H., Rusch, T., Mair, P., & Lowry, P. B. (2016). Breaking Free from the Limitations of Classical Test Theory: Developing and Measuring Information Systems Scales Using Item Response Theory. Information & Management. https://doi.org/10.1016/j.im.2016.06.005

Wahiah, Z., Prabowo, S. M., & Safitri, H. A. (2023). Eksplorasi Efektivitas Tes Pilihan Ganda Berbasis Komputer Sebagai Evaluasi Pembelajaran. EDUCATIVO: Jurnal Pendidikan, 2(2), 342–347. https://doi.org/10.56248/educativo.v2i2.

Wilkinson, L. C. (2018). Teaching the language of mathematics: What the research tells us teachers need to know and do. Journal of Mathematical Behavior, 51(May), 167–174. https://doi.org/10.1016/j.jmathb.2018.05.001

Yoshita Cahyaningrum, I., Fuady, A., & Islam Malang, U. (2023). Analisis Butir Soal Sumatif Akhir Semester Ganjil Mata Pelajaran Matematika Kelas VII dengan Berbantuan Aplikasi Software Anates. Mathema Journal, 5(2), 67–81.

Zanon, C., Hutz, C. S., Yoo, H., & Hambleton, R. K. (2016). An application of item response theory to psychological test development. Psicologia: Reflexao e Critica, 29(1), 1–10. https://doi.org/10.1186/s41155-016-0040-x

Ziegler, G. M., & Loos, A. (2017). “What is Mathematics?” and why we should ask, where one should experience and learn that, and how to teach it. 63–77. https://doi.org/10.1007/978-3-319-62597-3_5