Archived Assessment Report
| Program | PHYS Science Gen Ed |
| Assessment Reporter | [email protected] |
| Theme | Practicing Community |
| Review Year | 2024-2025 - Final Report |
| Learning Outcome (or Gen Ed Essential Skill) | Focus Area |
|---|---|
| 2b. Personal and Social Responsibility: Sustainability and the natural and human worlds | Do at least 80% of students understand the role of physics in protecting the natural world? |
| 2d. Personal and Social Responsibility: Collaboration skills, teamwork and value systems | Do at least 80% of the groups collaborate respectfully? |
| 3b. Quantitative Reasoning: Analysis of Quantitative Arguments | Can at least 70% of the students correctly interpret quantitative information presented by others? |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population or Courses Assessed | Hypothetical Analysis/Target |
|---|---|---|---|
| 2b. Personal and Social Responsibility: Sustainability and the natural and human worlds | Question within "Climate Change" lab. | PHYS 1230L | If at least 80% of students can list at least three ways that we, as a country, can reduce our carbon footprint, then we will know that at least 80% of students understand the role of physics in protecting the natural world. |
| 2d. Personal and Social Responsibility: Collaboration skills, teamwork and value systems | Instructors will observe lab groups as they work to complete a lab activity. Instructors will use a rubric to assess how respectful group members are of one another. | PHYS 1310L | We expect that at least 80% of the groups will collaborate respectfully. |
| 3b. Quantitative Reasoning: Analysis of Quantitative Arguments | Post and Reply discussion activity over the topic of conservation of linear momentum. | PHYS 1115 | If at least 70% of students are able to correctly apply conservation of momentum, then we will know that at least 70% were able to correctly interpret the quantitative information presented by others. |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Results | Reflection on Focus Area | Intepretation of Results |
|---|---|---|---|
| 2b. Personal and Social Responsibility: Sustainability and the natural and human worlds | Students were asked to identify a way that we, as a country, can reduce our carbon footprint and then to identify the top three renewable energies we might use to mitigate climate change. Fifteen of the 15 students assessed demonstrated that they understand the role of physics in protecting the natural world. Fourteen of the students provided appropriate answers to all of the prompts while one student lacked one of the renewable energy sources. | The results do reflect our focus area, although not as directly as we had originally intended. We did not realize that our lab question had been edited since we wrote our assessment plan. Therefore, the lab did not contain a question that was an exact match to our focus area. | The results indicate that students understand the role of physics in protecting the natural world. |
| 2d. Personal and Social Responsibility: Collaboration skills, teamwork and value systems | Three sections of PHYS 1310L were assessed through instructor observation of student group interactions. One hundred percent of the 22 groups assessed earned at least a 3 out of 4 points on a rubric that gauged the respectfulness of their group interactions. Seventy three percent of the groups (16 of 22) earned 4 points. The average score for the groups was 3.73 out of 4 points. | The results support our focus question because instructors observed that 100% of the groups in PHYS 1310L were respectful of one another. The rubric standard for a score of "3" states that the groups were "Mostly respectful of the efforts of fellow group members" and a score of "4" required "All members showed respect for the efforts of fellow group members". | These results indicate that our learners are well prepared to collaborate respectfully with others. |
| 3b. Quantitative Reasoning: Analysis of Quantitative Arguments | The thirteen learners enrolled in PHYS 1115 in Spring 2024 were assessed on their ability to interpret a quantitative description of a physical event and answer questions about it. They were scored using a rubric that evaluated their answers based on the correctness and depth of their solution. The rubric had three levels, assigning 1, 2, or 3 points depending on the quality of the learner's answer. The correct solution was required to earn a '2' or higher. All 13 learners earned a '2' or higher (with 4 learners earning a '2' and 9 earning a '3'). | The results support our original question, indicating that yes, at least 70% of our general education physics students are able to correctly interpret the quantitative information presented by others. | This shows us that our general education physics students are achieving the essential skill of being able to interpret quantitative information presented by others. |
| 2b. Personal and Social Responsibility: Sustainability and the natural and human worlds | |
|---|---|
| Describe the change that was implemented. | We are going to modify rubric to better fit the question asked in the lab. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | No. |
| What data motivated the change? | The data were good but there's always room to improve an assessment. |
| Hypothesis about the effect the change will have? | The results were already very good, we are not sure there is much room for improvement of outcomes. |
| 2d. Personal and Social Responsibility: Collaboration skills, teamwork and value systems | |
|---|---|
| Describe the change that was implemented. | We will edit the rubric to allow for more refinement between performance levels achieved by the lab groups. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | The changes will require applying a slightly different rubric. |
| What data motivated the change? | Interpreting the data with only four levels to the rubric made it hard to pick a threshold for being adequately respectful. Requiring a score of 4/4 seemed too rigid, but allowing a 3/4 was perhaps too lenient. |
| Hypothesis about the effect the change will have? | The improved rubric will help us better differentiate between those groups meeting the criteria of being adequately respectful and those who are not quite there yet. |
| 3b. Quantitative Reasoning: Analysis of Quantitative Arguments | |
|---|---|
| Describe the change that was implemented. | The outcomes were quite good; however, grading a discussion post takes a lot of time and the enrollment for this course recently grew from one section to 7 full sections. We will be changing the assessment to accommodate the larger number of learners and instructors involved. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, we will move from a discussion to a multiple-choice question on the final exam. |
| What data motivated the change? | The large spike in enrollment numbers and the number of new instructors, some of whom do not utilize discussions in their courses. |
| Hypothesis about the effect the change will have? | These changes will still assess the essential skill we're interested in while automating the collection of data and reducing the burden on the instructors. |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population of Courses Assessed |
|---|---|---|
| 2b. Personal and Social Responsibility: Sustainability and the natural and human worlds | Question in our Climate Change lab. | PHYS 1230L |
| 2d. Personal and Social Responsibility: Collaboration skills, teamwork and value systems | New rubric with 5 levels of performance to choose from. | PHYS 1310L |
| 3b. Quantitative Reasoning: Analysis of Quantitative Arguments | Final exam question. | PHYS 1115 |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Second Round Results | Intepretation of Results, Pre- and Post-Change | Follow up questions, possible next steps |
|---|---|---|---|
| 2b. Personal and Social Responsibility: Sustainability and the natural and human worlds | One section of PHYS 1230L was taught and assessed during Spring 2025. Fifteen of the 15 students assessed demonstrated that they understand the role of physics in protecting the natural world. All 15 identified a way that we, as a country, can reduce our carbon footprint (most students named reducing reliance on cars and/or switching to electric cars) and all 15 identified the top three renewable energies we might use to mitigate climate change. | Students seem to understand the role of physics in creating solutions for a sustainable future. Do they understand the tools and theories we used to initially diagnose the problems? | |
| 2d. Personal and Social Responsibility: Collaboration skills, teamwork and value systems | Two sections of PHYS 1310L were assessed through instructor observation of student group interactions. One hundred percent of the 12 groups assessed earned at least a 4 out of 5 points on a rubric that gauged the respectfulness of their group interactions. Sixty-seven percent of the groups (8 of 12) earned the full 5 points. The average score for the groups was 4.7 out of 5 points. | This time around we focused on the collaborating respectfully aspect of our program objective. Next time we could focus on the other half of the rubric, which examines how well they stay on task, to focus on their effectiveness in completing their tasks. | |
| 3b. Quantitative Reasoning: Analysis of Quantitative Arguments | We had some issues with our first time using Brightspace to collect our assessment data. We had six sections run the assessment, but due to issues with settings, etc., only four sections produced data we can analyze. The final exam question over conservation of momentum required that students interpret quantitative information presented by others. We assessed 118 students across four sections, and 85% (100 students) answered the question correctly, which is well above our goal of 70%. This result indicates that these general education physics students are successfully interpreting quantitative information presented by others. | The students performed very well on this assessment and we have no follow-up questions at this time. |
Describe any change in student achievement observed as part of this assessment process, and what led to those changes.
Our students learn about practicing community the most during group work and experimentation. The students themselves do most of the teaching, as they work together and communicate about what is or is not working. Instructors are more like facilitators, there to help guide groups through situations where they may be in disagreement or distracted from the task at hand.
Describe long-term changes in the program(s) that the assessment process led to, and what motivated those changes?
No, we were pleased with the level to which our program embodied "practicing community" and did not feel any program-level changes were warranted.
What did you learn about the teaching and learning of "Practicing Community" in your programs?
Through guidance from the instructors and practicing community, the students learn that discussion and negotiations lead to better comprehension for both themselves and everyone else in the group.
Describe any external factors affecting the program or affecting assessment of the program.
We had a large spike in the enrollment in our PHYS 1115 course as a result of changes to an external degree program's requirements. This necessitated some changes to both how we communicated with the larger number of instructors and how we deployed the assessment for PHYS 1115. Although we were largely successful pivoting to address this sudden change, we did have some issues with the collection of data for this assessment and look forward to correcting those issues next time around.