Designing of Numeracy Questions for Middle School Level Students

Numeracy is a skill that has begun to be paid attention to in recent years, including in Indonesia. When solving numeracy questions, students often find it difficult, especially if the stimuli used in the questions are unfamiliar to students. Teachers can bridge this by using numeracy questions starting with stimuli that the students are familiar with or are around the students. The aim of this research is to produce numeracy questions at the middle school level that are suitable for use. This research is Research and Development research with the ADDIE (Analyze, Design, Development, Implementation, Evaluation) model. The instruments used in this research were interview guides and questionnaires. Question validation was carried out by providing draft questions to 2 experts in the field of mathematics education at State Universities in Central Java, Indonesia. Data was analyzed descriptively, both qualitatively and quantitatively. The results of this research show that references to numeracy questions are needed. After preparing the numeracy questions through the design and development process, it was discovered that the questions prepared received a score of 4.27 in the very good category from expert validators.


INTRODUCTION
Numeracy ability is one of the skills that a person really needs to master in facing life.The numeracy ability referred to has now experienced a shift in meaning, from being narrowly defined as skills involving numbers and counting to the ability to apply mathematical concepts and skills to solve practical problems in various contexts of daily life (Resti et al., 2020;Susanto et al., 2021) as well as skills in analyzing information displayed in various forms (graphs, tables, charts, and so on), then using the interpretation of the analysis results to predict and make decisions (Resti et al., 2020).Numeracy can also be referred to as numeracy literacy or mathematical literacy.
Numeracy skills have received much attention internationally, including in Indonesia.Developing students' numeracy skills is considered a major goal of mathematics education (Adelia et al., 2024).Many assessments of numeracy abilities have also been carried out, one of which is through the PISA assessment.The PISA assessment is carried out every three years and Indonesia is one of the participating countries.The 2018 PISA results show that Indonesia is ranked 73rd out of 79 countries with an average score of 379 in mathematics (Hewi & Shaleh, 2020;Kemdikbud, 2018).It can be interpreted that the average ability of Indonesian students is at level 1 of 6 existing levels.Starting in 2021, the government conduct a Minimum Competency Assessment or "Asesmen Kompetensi Minimum (AKM)" where one of the competencies measured is numeracy.AKM is an assessment of the fundamental competencies needed by students to develop their own capacity and participate positively in society (Sensus et al., 2022).If the PISA test is carried out on students aged 15 years, the AKM test is carried out on students in grades 5, 8 and 11.However, this numeracy ability does not only need to be developed and measured on students in these grades, However, there needs to be continued efforts so that students' numeracy skills can develop better.
Based on the results of interviews with several mathematics teachers, teachers have limitations in creating questions to measure numeracy skills, both in terms of time and ideas in creating them.There are many training or mentoring activities in creating questions to measure numeracy abilities as carried out by Zukhrufurrohmah & Putri (2021), Novita et al. (2022), Rahman et al. (2022), Fiangga et al. (2019) and others, but such training activities are very limited in number.Likewise, research related to the development of numeracy questions has been widely carried out, but is still limited to certain materials or certain contexts (Aulia & Mutaqin, 2022;Kurniawan et al., 2022;Rachma Kurniasi et al., 2023;Susetyawati & Kintoko, 2022).It requires questions that are close to students or situations that are familiar to students.Apart from that, quoted from Peraturan Kepala Badan Standar, Kurikulum, dan Asesmen Pendidikan Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi Nomor 015/H/KP/2023 about Standard Operational Procedures for Implementing National Assessments in 2023 (2023) in each educational unit at the middle school level, a maximum of 45 students are selected to take part in National Assessment.This means that teachers cannot map the numeracy abilities of each student they teach.
Based on this, it is necessary to develop numeracy questions with the aim of assisting teachers in providing assessment questions that are suitable for measuring the numeracy abilities of middle school students.The results of this research are numeracy questions for SMP/MTs level which are prepared based on the AKM framework which can be an alternative for mathematics teachers to carry out numeracy assessments or train numeracy skills for their students which have been validated by experts.

METHODS
This research is development research carried out by following the ADDIE model development procedure which has five stages: Analyze, Design, Development, Implementation, and Evaluation.However, this article only reports up to the development stage.At the analysis stage, researchers carry out goal setting activities, competency analysis, and reference analysis.Goal setting activities are based on needs related to numeracy questions.In the competency analysis activity, researchers carried out an analysis related to the competencies expected to be achieved by grade 8 students which were sourced from learning progression as a reference in developing mathematicsnumeracy literacy questions contained in the Minimum Competency Assessment Framework (Tim Substansi Asesmen Akademik, Pusat Asesmen dan Pembelajaran, Badan Litbang dan Perbukuan et al., 2021).The reference analysis activity aims to find out the type of numeracy questions, the form of the questions, the cognitive level of the questions, the number of questions, the context used, as well as the domain content for the numeracy questions, and other important information.References analyzed include: (a) Framework "Asesmen Kompetensi Minimun (AKM)" (Tim Substansi Asesmen Akademik, Pusat Asesmen dan Pembelajaran, Badan Litbang dan Perbukuan et al., 2021); (b) Erlangga X-Press AKM SMP/MTs (Miftahudin & Khotimah, 2022); (c) "Asesmen Nasional -Asesmen Kompetensi Minimum"; (d) AKM Pusmendik Simulation (Pusmendik, 2023); dan (e) "Modul berkembang : Pengetahuan Numerasi: Proses, Konten, dan Konteks" (Fachrudin, 2022).
At the design stage, researchers create a question grid as a basis for preparing questions including domain, subdomain, competency, context, cognitive level, question form, and question number.In addition, at this design stage, the researcher collects material for the question stimulus.The materials used as stimuli in this research include news, posters, the web, personal sources, the environment, etc.At the development stage, the researcher carried out question writing activities based on a grid, validated the questions with the help of two experts in the field of mathematics education, carried out stage 1 revisions based on input from experts, conducted limited question trials on two mathematics teachers and five junior high school students, and carried out stage 2 revisions.The lecturers, teachers and students involved in this research all came from institutions in Central Java, Indonesia.Data collection was carried out using interview and questionnaire techniques.Data was analyzed descriptively qualitatively and quantitatively.
To determine the validation of questions from experts, this is done by recapitulating the validity assessment data into a table for each validator, determining the average score for the two validators, and categorizing the average for all aspects of the expert assessment into the following categories (Widoyoko, 2009) If the score is in the minimum good category, then the researcher continues to the next stage.As for analyzing the results of teacher and student assessments, it is done in the same way as before.

RESULTS AND DISCUSSION
In this research, researchers developed numeracy questions for junior high school level.This was decided on the basis that at the initial analysis stage, it was known from junior high school teachers that the Mid-Semester Assessment and Final Semester Examination questions in the school referred to AKM questions, the assessment of numeracy abilities among students in schools had not been carried out evenly, and there were limited references available regarding the questions.numeracy at school.On this basis, researchers developed numeracy questions that can be used as a reference for teachers when preparing numeracy questions or can be used as questions to assess numeracy skills, outside of those carried out by the government on a scheduled basis.
The development of these numeracy questions refers to the "Asesmen Kompetensi Minimum (AKM)" Framework published by "Tim Substansi Asesmen Akademik, Pusat Asesmen dan Pembelajaran, Badan Litbang dan Perbukuan".Based on information obtained from several literatures that have been reviewed by researchers, there are several provisions, including the existence of learning progression for certain class levels.This learning progression is divided into the domains of numbers, geometry and measurement, algebra, and data and uncertainty.This domain is still divided into several subdomains.Interpret linear functions and their graphs, as well as their properties.

Algebra Ratio and Proportion
Solve social arithmetic problems related to ratios/percentages.

Data and representation
Determine and use the mean, median, and mode.

Data and Uncertainty
Data and representation Compare and assess the effectiveness of various data views Data and Uncertainty

Uncertainty and probability
Calculating the probability of a simple event Researchers compiled numeracy questions for junior high school level consisting of 36 questions starting with a stimulus.The stimuli used consist of personal, sociocultural and scientific contexts.Researchers emphasize stimuli in the context surrounding students' lives.It is important to use a broad context in numeracy questions so that students can recognize the various roles of mathematics in everyday life (Tim Substansi Asesmen Akademik, Pusat Asesmen dan Pembelajaran, Badan Litbang dan Perbukuan et al., 2021).Apart from that, contextual questions related to students' daily lives where they live are very interesting to activate students in learning and challenge students' mathematical thinking processes (Charmila et al., 2016).There are three cognitive levels in AKM questions, namely knowledge and understanding, application, reasoning.The questions prepared also have various question forms, namely essay, short entries, multiple choice, complex multiple choice, and matching.The information obtained by researchers at the analysis stage is used as a guide in compiling the question grid carried out at the design stage.Preparing a question grid is important because it can help researchers in compiling questions according to the specified criteria (Young et al., 2019).At this stage, the researcher collects materials used for stimulus questions.This material is in the form of photos, information on official websites, news, books, etc. designed to be reading material as a stimulus for questions.
In the question grid, the researcher designed 10 questions in the number domain, 9 questions in the geometry and measurement domain, 10 questions in the algebra domain, and 7 questions in the data and uncertainty domain.Among these questions, there are 11 questions with a personal context, 16 questions with a socio-cultural context, and 9 questions with a scientific context.The researcher planned a total of 11 questions at the knowledge and understanding level, 18 questions at the application level, and 7 questions at the reasoning level.The form of questions used consists of 14 multiple choice questions, 10 complex multiple choice questions, 2 matching questions, 7 short answers, and 3 essay questions.The materials used for stimulus questions were collected by researchers from https://id.weatherspark.com,https://regional.kompas.com,https://www.antarafoto.com,http://klikdinamika.com, https://www.instagram.com,https://jambiekspres.disway.id,https://www.bps.go.id, https://salatigakota.bps.go.id, https://yankes.kemkes.go.id, https://dataku.salatiga.go.id,Google Maps application, and personal documents.
From the designed stimuli, the researcher carried out a development stage by compiling 36 questions guided by a question grid to obtain draft question 1.At this stage, expert validation was also carried out by asking two validators in the field of mathematics education to carry out assessments and provide input regarding draft question 1.The following are the validation results for draft question 1.The assessment from expert validation obtained a score of 4,27 in the very good category.However, there are still things that need to be improved, such as inconsistencies with the framework regarding the number of answer choices and statements in the form of multiple choice, complex multiple choice, and matching questions.Improvements were also made to the questions and answer keys so that draft question 2 was obtained.Note: the number of statements in the Complex Multiple Choice question form is 3 statements, adapting to the AKM framework 14 Note: the number of responses in the matching question form must be greater than the number of statements, according to the AKM framework, so one more response is added.16 Wakil Wali Kota Salatiga Muhammad Haris (kanan) memberikan tempat kompos kepada warga saat peringatan Hari Peduli Sampah Nasional di TPA Ngronggo, Salatiga, Jawa Tengah, Senin (21/2/2022).Kegiatan yang diikuti oleh unsur pemerintah dan berbagai komunitas peduli sampah tersebut untuk mengajak masyarakat agar peduli terhadap sampah dan mengolah sampah secara mandiri (https://www.antarafoto.com/id/view/1607881/hari-peduli-sampahnasional-di-salatiga).Seperti yang kita lihat pada gambar di web Wakil Wali kota Salatiga memberikan tempat kompos yang berbentuk tabung.Lebar tabung 42 cm dan tinggi dari tabung 100 cm.
Note: Because in the photo, the tube image has different diameter sizes between the top, middle and bottom, the word "average" was added.
After going through stage 1 revision, the researcher conducted limited test questions by giving them to two middle school mathematics teachers and five class VIII middle school students.Following are the results of the assessment.Based on the results of the analysis, the average score from the teacher response questionnaire was 4,08.By obtaining this average score, it can be concluded that the questions prepared are categorized as good.Looking at each assessment component, all components can be categorized as good.
Meanwhile, student assessments include construction, language, and time to complete questions.Based on the data from the student response questionnaire, data analysis was carried out with the following results.Based on the results of the analysis, the average score from the student response questionnaire was 4,37.By obtaining this average score, it can be concluded that the questions prepared are categorized as very good.Looking at each assessment component, all components can be categorized as very good.
The teacher's assessment received a score of 4,08 which was categorized as good, while the students received a score of 4,37 in the very good category.However, researchers need to carry out stage 2 revisions based on teacher input.Note: before the revision, the Salatiga City Square was square in shape.However, due to the fact that it is not square, the mention of the location of the square was omitted One instrument that can be used to measure success in achieving learning goals is questions (Sobarningsih et al., 2019;Utomo, 2018).Validation of questions by experts is one effort to obtain quality questions.In this development research, questions were developed to measure numeracy skills for students at junior high school level.Therefore, it is important to ensure that the questions developed are truly appropriate and appropriate for measuring numeracy abilities in students at the junior high school level.Thus, the questions developed are said to be valid if they can measure skills or indicators of numeracy ability based on the AKM framework (An Nabil et al., 2022).Therefore, in developing this question, the step taken after designing the question was to validate draft question 1 with two experts in the field of mathematics education.The assessment from expert validation obtained a score of 4.27 in the very good category.This shows that in general, the questions developed are in accordance with the AKM framework in terms of material and content.Apart from that, this expert assessment also looks at aspects of question construction, language and time in completing the questions.This fulfills the rules for writing good questions, namely by paying attention to material, construction and language (Setiawati & Lapassau, 2022). he results of validation by experts are supported by the results of teacher and student response questionnaires with good and very good results.However, the draft questions produced still require improvement according to the notes of experts and teachers.

CONCLUSION
Based on the results of the research and discussion, it was concluded that the student numeracy questions for SMP/MTs level developed in this research were for use.This is based on the validator's assessment being in the very good category, the teacher's assessment being in the good category, and by the students being in the very good category.Based on the research results, the researcher provides suggestions for continuing the question development process through question testing activities so that the quality of the questions is known, such as validity, reliability, level of difficulty and distinguishing power.Once the quality of the questions is known, teachers can use the questions that have been developed in this research to measure students' numeracy skills at the middle school level, as well as develop numeracy questions with more diverse stimuli, following developments in existing information, as well as discussion trends or topics that are being widely discussed in the community.Apart from that, students can use the questions that have been developed as practice questions to increase their numeracy skills.

Table 1 .
: Category of Expert Assessment

Table 2 .
Learning Progression Numeracy for 8th Grade Students

Table 3 .
Data Analysis of Validation Results

Table 4 .
Revision of Draft Question 1 Note: the number of statements in the Complex Multiple Choice question form is 3 statements, adapting to the AKM framework No Before Revision After Revisions and Explanations Note: the number of statements in the Complex Multiple Choice question form is 3 statements, adapting to the AKM framework 6 Note: the number of answer choices becomes 4 choices, adapting to the AKM framework 3

Table 5 .
Data Analysis of Teacher Assessment Results

Table 6 .
Data Analysis of Student Assessment Results

Table 7 .
Revision of Draft Question 2