The quality of a country’s education system is reflected in the overall academic performance of its students. High-quality education systems typically feature effective curricula, teaching methods, and resources, all within a supportive learning environment. These factors enhance student engagement, comprehension, and, ultimately, lead to higher cognitive achievement.
Student learning assessments play a crucial role in capturing cognitive achievement. These assessments provide valuable insights into students’ comprehension and mastery of a variety of subjects, such as math, science, and reading. The data collected from these assessments can inform curriculum development, teaching strategies, and education policies at a broader level. Therefore, student learning assessments are not only essential for monitoring student progress but also necessary for improving the overall quality of education in a country.
International educational assessments like TIMSS and PISA provide general scores to describe students’ proficiency in specific areas, i.e., science, math, reading. The achievement scores have a significant impact on educational practice and reform in education policies in many countries (OECD, 2013; Woessmann, 2005). The scores of TIMSS and PISA are presented in a unidimensional model which is the ideal design for ranking of participants in a cross-country analysis (OECD, 2013). Studies have also used the scores to explain the development of a country (Woessmann, 2005; Hanushek and Woesmann, 2012) or examined factors that affect achievement scores (Lamb and Fullarton, 2001).
Meanwhile, national educational assessments collect data to inform system-wide performance and diagnose the needs of each school. This data can be analyzed using various methods to estimate proficiency levels, monitor longitudinal performance changes, compare group achievement, and investigate the link between achievement and student background. Each approach can be employed independently or concurrently.
SMERU undertakes a project that analyzes the National Assessment (AN) and the Indonesian Madrasah Competency Assessment (AKMI) for primary and junior secondary school levels.
In general, evidence and knowledge generated from this research is expected to inform future education quality improvement and system transformation efforts.
Specifically, this study aims to answer the following research questions:
- How do students perform in terms of literacy and numeracy? (disaggregated by type of schools (school/madrasah), urban and rural, female and male principals, and other school characteristics)
- How do students perform in terms of acquiring 21st century skills? (disaggregated by type of schools (school/madrasah), urban and rural, female and male principals, and other school characteristics)
- What are some of the underlying determinants affecting performance in literacy, numeracy, and/or 21st century skills? (disaggregated by type of schools (school/madrasah)
- What are the factors and processes affecting learning performance following the implementation of the AN? (This may include but not limited to teaching method, classroom management, lesson planning based on the assessment result, teachers’ characteristics, school leadership, supervision, teacher professional development, accountability, etc.)
- How might we better improve the learning assessment process; what are some of the remaining data gaps? How might we improve accessibility to make it more inclusive?
This study employs a combination of various methodologies depending on the level of accessible data. Acquiring educational assessment data at the student level can offer greater insights into student achievement, especially when data is disaggregated across distinct competency domains. This enables reporting of student achievement by domain performance. To analyze student mastery of attributes, we examine performance across specific groups. For instance, we can compare the scores of the top 20% of performers to those in the bottom 20%.
Another approach to analyze student-level data is to look for connections between competency domain achievement in each competency and background factors such as student and school characteristics. School-level data from assessments, if available, would enable us to link these factors to overall school performance.
To achieve a fine-grained diagnosis of student performance in sub-competency domains, we will employ the cognitive diagnostics method (Leighton and Gier, 2007). Additionally, decomposition techniques will be applied to analyze the correlation between achievement and background factors. This will allow us to identify the factors contributing to the achievement gap and explore how student socio-economic status and school characteristics might have affected it.
Using the quantitative method, we analyze secondary data from two educational assessments: the AN and the AKMI. If we could access the student-level data, we could analyze the mastery of attributes among the top 20% and bottom 20% of Indonesian students. Additionally, we could investigate gender differences in attribute mastery.
Assuming the available data is restricted to the school level, we would conduct a gap analysis using school-specific characteristics and employ decomposition analysis to see which school characteristics significantly affect performance differences between groups.
The qualitative method is employed to explore how different factors or processes affect learning performance following the implementation of the AN. Data is collected from two districts: Kabupaten Magelang in Central Java and Kabupaten Maros in South Sulawesi. We focus on how instruction varies based on distinct teacher and school characteristics. Within each district, we purposively select four schools based on a combination of factors: public/private and secular/religious status, location (rural/urban), and whether the school has adopted the Emancipated Curriculum.
