Evaluating the Efficacy of a High School Chemistry and Physics
Project-Based Learning Intervention: Crafting Engaging Science Environments
Authors: Barbara Schneider1, Joseph Krajcik1, Jari Lavonen2, Katariina Salmela-Aro2, Christopher Klager1, Quinton Baker1, I-Chien Chen1, Lydia Bradford1, Israel Touitou1, Deborah Peek-Brown1, Rachel Marias Dezendorf1, Sarah Maestrales1
1Michigan State University, 2University of Helsinki, Helsinki, Finland
Abstract
Crafting Engaging Science Environments, a system approach to improving science achievement in high school chemistry and physics, includes learning activities and materials, assessments, and professional learning for teachers. Developed over three years, this intervention concluded with a cluster randomized efficacy trial for a diverse group of over 4,000 students in California and Michigan. This study reports the results from this trial; treatment students on average performed 0.2 standard deviations higher on an independent summative science assessment than the control students. These results are consistent across gender, race, or ethnicity. Mediation analysis shows a possible indirect path between teacher and student reported participation in science practices and the outcome. Exploratory analysis also indicates positive treatment effects for changing college ambitions.
Schneider, B., Krajcik, J., Lavonen, J., Salmela-Aro, K., Klager,C., Baker, Q., Chen, I., Bradford, L., Touitou, I., Peek-Brown, D., Marias Dezendorf, R., & Maestrales, S. (2020) Improving Science Achievement- Is it Possible? Evaluating the Efficacy of High School Chemistry and Physics Project Based Learning Intervention: Crafting Engaging Science Environments. Manuscript in Progress.
Citation: Maestrales, S., Marias Dezendorf, R., Tang, X., Samela-Aro, K., Bartz, K., Juuti, K., Lavonen, J., Krajcik, J., & Schneider, B. (In Press). US and Finnish High School Science Engagement During the Covid-19 Pandemic. International Journal of Psychology.
Abstract
In response to the call for promoting three-dimensional science learning (NRC, 2012), researchers argue for developing assessments that go beyond rote memorization tasks to ones that require deeper understanding and the use of reasoning that can improve science literacy. Such assessment tasks are usually performance-based constructed responses and need technology involvement to ease the burden of scoring placed on teachers. This study responds to this call by examining the use and accuracy of a machine learning text analysis protocol as an alternative to human scoring of constructed response items. The tasks we employed represent multiple dimensions of science learning as articulated in the 2012 NRC report. Using a sample of 26,000 constructed responses taken by 6700 students in chemistry and physics, we trained human raters and compiled a robust training set to develop machine algorithmic models and cross-validate the machine scores. Results show that machine scoring algorithms achieved similar performance to human raters. These results indicate the potential of using machine learning protocols to lessen the load of human scoring while maintaining fidelity and differentiation between different dimensions of learning.
Citation: Maestrales, S., Zhai, X., Touitou, I., Baker, Q., Krajcik, J., & Schneider, B. (2021). Using Machine Learning to Evaluate Multidimensional Assessments of Chemistry and Physics. Journal of Science Education and Technology, 30(2), 239-254
Integrating the light and dark sides of student engagement with person-oriented and situation-specific approaches
Abstract
This study contributes to the research on student engagement in three ways: 1) by combining questionnaire and situational measures of engagement using the Experience Sampling Method (ESM), 2) by applying a demands-resources model to describe the positive and negative aspects of student engagement, and 3) by adopting a person-oriented approach to describe subgroups of students with different profiles of engagement and burnout symptoms. Two studies were conducted: sample one comprised 255 US high school students (45.5% female, 9th – 12th grade), and sample two 188 Finnish comprehensive and high school students (59.6% female, 9th to 10th grade). Latent profile analyses (LPA) of person-level measures of schoolwork engagement and burnout in the US and Finland revealed four profiles: 1) engaged, 2) engaged-exhausted, 3) moderately burned out (risk for burnout) and 4) burned out. These four groups were identified in both samples, but differed in their prevalence. The groups differed in their state experiences of situational demands, resources and engagement at school. Engagement is not wholly an experience of ‘flourishing’: some students experienced elevated levels of both engagement and burnout. Thus, positive and negative aspects of engagement should be studied and modeled together.
Citation: Salmela-Aro, K., Moeller, J., Schneider, B.; Spicer, J., & Lavonen, J.. “Integrating the light and dark sides of student engagement with person-oriented and situation-specific approaches.,” Learning and Instruction., 2016. doi:EARLI doi:10.1016/j.learninstruc.2016.01.001.
Investigating Optimal Learning Moments in U.S. and Finnish Science Classes
Abstract
This study explores how often students are engaged in their science classes and their affective states during these times, using an innovative methodology that records these experiences in situ. Sampling a subset of high schools in the U.S. and Finland, we collected over 7,000 momentary responses from 344 students over the course of a week. We examine engagement within and between students in different environments identifying common social and emotional factors they may be experiencing in their science classes, suggesting the challenges that the U.S. and Finland may encounter when implementing their new science standards (i.e., Next Generation of Science Standards and Finnish National Core Curriculum). We operationalize engagement as situational when students experience high levels of challenge, skill, and interest, which we term as optimal learning moments. Specifically we analyze: (i) the components of optimal learning; (ii) the relationship of optimal learning with other subjective measures; (iii) how optimal learning moments in science classes compare to other academic classes; and (iv) the extent that optimal learning moments predict an individual’s perception of importance to self and future in science classes. Using several multivariate models, results show that when students are challenged in their classes and are appropriately skilled they are more likely to feel confident, successful, and happy during specific science classes as well as in other academic classes. When students experience more times of optimal learning in their science classes they are more likely to report that they perceive science as important to them and their futures. Females, however, report being more stressed in their science classes than males. © 2015 Wiley Periodicals, Inc. J Res Sci Teach 53:400–421, 2016
Citation: Schneider, B., Krajcik, J., Lavonen, J., Salmela-Aro, K., Broda, M., Spicer, J., Bruner, J., Moeller, J., Linnansaari, J., Juuti, K. and Viljaranta, J.. “Investigating optimal learning moments in U.S. and finnish science classes.,” Journal for Research in Science Teaching, v.53, 2016, p. 400. doi:10.1002/tea.21306
Does Anxiety in Science Classrooms Impair Math and Science Motivation? – Gender Differences Beyond the Mean Level
Abstract
This study investigated gender differences in the experience of situational anxiety (referred to as ‘state anxiety’) among a sample of 268 US and 202 Finnish lowerand upper-secondary-school / high-school students (51.0% female; 177 ninthgraders, 218 tenth-graders, 37 eleventh-graders, 38 twelfth-graders, 10 unspecified grade). Three main research questions guided our study: 1) Do male and female students differ in their anxiety during science lessons if in-the-moment state measures are used?; 2) How does anxiety affect motivation in science classes?; and 3) Does the relationship of anxiety to motivation differ by gender We employed the experience sampling method (ESM), a form of time/diary instrument, to assess experiences of anxiety in the moment in which they occur, in different contexts, e.g., in and out of school and in specific science lessons. Males and females did not differ in mean levels of state anxiety with in-the-moment measures, which corroborates previous findings. Females tended to experience less positive affect and intrinsic motivation, and more negative affect and withdrawal motivation in anxious states across all their everyday life experiences. In science lessons, the only consistent finding was that females tended to experience more stress in anxious situations than males. The findings suggest that previously found gender differences in math and science anxiety might be due to biases in the applied measures (see Goetz et al., 2013), which has important theoretical and practical implications for the assessment and interpretation of gender differences in science classrooms.
Citation: Moeller, J., Salmela-Aro, K., Lavonen, J., & Schneider, B.. “Does anxiety in science classrooms impair math and science motivation? – Gender differences beyond the mean level,” International Journal of Gender, Science and Technology, v.7, 2015, p. 229.
Students Making System Models: An Accessible Approach
Abstract
From the large to the small, many phenomena can be conceptualized as sets of interacting parts that form a system with emergent properties. Understanding and thinking about phenomena from a systems perspective is critical in many areas of science, because the world is complex and interconnected in ways that can often be described best using system models. Defining a system and constructing a model of that system allows scientists to focus on variables that are critical to explaining a phenomenon. We expect our students, like real scientists, to explore systems and system modeling as a necessary step in developing scientific literacy. Although scientists often use a systems perspective to frame their studies, and A Framework for K–12 Science Education (NRC 2012) and the Next Generation Science Standards (NGSS Lead States 2013) have identified Systems and System Models as a crosscutting concept, we have found that students do not naturally think in terms of systems.
Citation: Damelin, D., Krajcik, J., McIntyre, C., and Bielik, T.. “Students making system models: An accessible approach.,” Science Scope, v.40, 2017.
The Activity Summary Board: Adding a Visual Reminder to Enhance a Project-Based-Learning Unit
Abstract
Project-based learning (PBL) is an instructional approach to science teaching that supports the Next Generation Science Standards. Here, Touitou et al introduce an additional tool, the activity summary board (ASB). The ASB is a classroom organizational tool that summarizes what students do and figure out in the classroom as they take on the role of scientist or engineer to make sense of the phenomenon or problem.
Citation: Touitou, I., Barry, S., Bielik, T., Schneider, B., & Krajcik, J.. “The Activity Summary Board: Adding a visual reminder to enhance a project-based-learning unit.,” Science Teacher, 2018.
Science Classroom Activities and Student Situational Engagement
Abstract
This study examines the association between student situational engagement and classroom activities in secondary school science classrooms in Finland and the U.S. Situational engagement is conceptualised as times when students feel that a task is interesting and challenging to them and that they have the skills to complete it (see Schneider et al., 2016. Investigating optimal learning moments in U.S. and Finnish science classes. Journal of Research in Science Teaching, 53(3), 400–421. doi:10.1002/tea.21306). Data on situational engagement and classroom activities were obtained using the experience sampling method (ESM) from 247 Finnish students in 13 secondary science classrooms and 281 U.S. students in 18 secondary science classrooms. In both samples, the students tended to be situationally engaged only a small proportion of the time during their science classes. However, the Finnish students were more likely than the U.S. students to report being situationally engaged. To investigate when the students were most likely to report being situationally engaged, hierarchical logistic regression models were employed, which suggested that some classroom activities were associated with higher levels of student situational engagement than others. The Finnish students were more likely to report being situationally engaged when calculating and presenting scientific information. In the U.S., the students were more likely to report being situationally engaged while discussing scientific information and less likely when listening to the teacher. The results suggest that situational engagement is momentary and associated with specific science classroom activities.
Citation: Inkinen, J., Klager, C., Schneider, B., Juuti, K., Krajcik, J., Lavonen, J., & Salmela-Aro, K.. “Science classroom activities and student situational engagement.,” International Journal of Science Education, v.41, 2019, p. 316.
High School Students? Situational Engagement Associated with Perceived Scientific Practices in Designed Science Learning Situations
Abstract
This study seeks to understand how different scientific practices in high school science classrooms are associated with student situational engagement. In this study, situational engagement is conceptualized as the balance between skills, interest, and challenge when the reported experiences are all high. In this study, data on situational engagement were collected using the experience sampling method (ESM) from 142 students in southern Michigan (the United States), resulting 993 ESM responses, and 133 students in southern Finland, resulting 1,351 responses. In both countries, scientific practices related to developing models and constructing explanations were associated with higher student situational engagement than other practices. In southern Finland, using a model was also associated with a high level of student situational engagement. The results indicate that students may experience situational engagement more often in science classrooms that use models than those that do not employ such practices. Thus, scientific practices related to models should be used frequently in science classrooms to situationally engage students while learning science.
Citation: Inkinen, J., Klager, C., Juuti, K., Schneider, B., Salmela‐Aro, K., Krajcik, J., & Lavonen, J. (2020). High school students’ situational engagement associated with scientific practices in designed science learning situations. Science Education, 104(4), 667-692.
