Investigador Responsable: David Torres Iribrarra
_______________________

The aim of this research project is to investigate the impacts of implementing a computer-based Assessment Management System (abbreviated as AMS) as a tool for professional development (PD) among mathematics teachers in Chile, examining its effects on teacher practices and student learning outcomes. We propose working with mathematics teachers in grades 7 to 10 by integrating an AMS into teachers’ PD with the objective to enhance teaching quality in three significant ways: (i) by improving the way teachers design, select or adapt their assessment activities, (ii) by easing the embedded deployment of high-quality classroom assessments, and (iii) by fostering a diagnostic use of the evidence produced from these assessments. This project seeks to assess both direct effects of the combined AMS and PD on teachers’ classroom assessment practices, and its indirect effect on learning outcomes.

This study is motivated on the one hand by empirical evidence of important shortcomings in Chilean teachers’ performance, a finding of the National Teacher Performance Evaluation: in 2019, 7 out of 10 evaluated teachers did not reach the expected level in this field (Sun, 2022). While on the other hand by, it is prompted by the recommendations made by the OECD report on Chile’s teacher evaluation, which emphasized the need for communities of practice in schools “where teachers can share strategies, observe one another, collaborate on projects, all with the aim of learning from one another” (p.177; Santiago et al., 2013).

This project will focus on improvement of classroom assessment practices and their alignment with curricular objectives in an area and grades that are critical in the context of a comprehensive reactivation of learning (Mineduc, 2023), shown to require increased attention to bridge the gap in learning outcomes that has emerged in the post-pandemic years (Agencia de Calidad de la Educación, 2023).

The theoretical rationale behind this study is rooted in (i) the literature on the role of well-designed classroom assessments and diagnostic interpretations of their results as a key aspect of effective teaching (NRC, 2001) and (ii) the research program on learning progressions, which are descriptions of developmental pathways that can be linked to assessments to support the interpretation of student responses (National Research Council, 2005) in order to promote a beneficial role for assessments in a learning culture (Shepard, 2000).

The AMS to be used in this study would be an adaptation of the Berkeley Assessment Software System (BASS), a system created to manage the design, delivery, and analysis of assessments and reporting of their results (Torres Irribarra et al., 2021). Currently, BASS supports the explicit integration of curriculum and assessments, as well as the creation and delivery of classroom assessments, the rating of student work and the generation of reports and summaries of individual and classroom performance.

We intend to use the AMS to support and connect teacher PD and classroom assessment. The AMS would provide teachers with a repository of assessments and student responses for use in PD and discussion on (i) the quality of assessments and (ii) interpreting student responses to inform their lesson plans. From this repository they could easily select individual tasks to use in their classrooms through printed or electronic versions. As the teachers become familiar with the system, they can design their own assessments, and discuss and improve them in the PD sessions, before adding them to the AMS repository and making them available to other teachers. The AMS+PD approach would thus support a beneficial cycle of teacher training and practice, improving classroom assessments and diagnostic use of student responses.

The research questions of this study will address the effectiveness of using an AMS+PD approach in terms of its direct effects on teacher practices and indirect effects on student outcomes. This study’s four central questions are: To what extent does the use of an AMS as a PD tool produce (1) improvements in the quality of classroom assessments designed by teachers?, (2) improvements in the quality of the interpretation of students’ responses to classroom assessments?, (3) more effective embedding of assessments into teachers’ classroom practices?, and (4) improvements in student learning?

To answer these questions, we propose two field studies, one observational and one experimental. The observational study would begin with a pilot during the project’s first year, working with approximately 12 teachers. The purpose of the pilot would be to adapt the AMS system to the Chilean context, while collecting data on teacher adoption of the system, its usability, and the amount and kinds of additional support that participating teachers and schools require to successfully integrate the AMS into their practices and workflow. The information collected during this study would prepare and streamline the AMS and teacher PD sessions for the second study. After finishing the pilot during the first year, we will follow up with this sample in years 2 and 3. Following up with this group of teachers will allow us to collect information about longer term challenges and effects of the adoption of the AMS+PD program.

The experimental study would take place during the second and third years of the project, and will start with a total group of approximately 36 teachers that, through a random assignment, will be allocated into to the ‘treatment’ (i.e. AMS+PD) and the ‘control’ (i.e. PD only) conditions. The control group will only participate in PD sessions on improvement of classroom assessment practices targeting the same curricular units as the treatment condition, but without discussing nor giving access to the AMS. On year 2 only a third of the teachers will be assigned into the treatment condition, and on year 3 the control group will be split randomly, assigning another third of the teachers into a second treatment group.