Archived Assessment Report
| Program | First Responder |
| Assessment Reporter | [email protected] |
| Theme | Practicing Community |
| Review Year | 2024-2025 - Final Report |
| Learning Outcome (or Gen Ed Essential Skill) | Focus Area |
|---|---|
| 1. Affective Domain: Be a role model of exemplary professional behavior including: but not limited to, integrity, empathy, self-motivation, appearance/personal hygiene, self-confidence, communications, time management, teamwork/ diplomacy, respect, patient advocacy, and careful delivery of service. (Professionalism) | Are students consistently meeting the minimum competency of 26/36 points on their Professional Behavior evaluations? |
| 2. Preparedness: the student consistently arrived on time with required materials and was ready to learn. Professional appearance: the student is dressed appropriately and is neat in appearance. No hygiene issues. | Are students consistently arriving for classes on time, professionally groomed, and with all required materials? |
| 3. Initiative: student demonstrates interest in EMS through actions and interactions with evaluator. | During class interactions, do students actions show that they have an interest/ passion for the EMS profession? |
| 4. Conduct: Student interacts with other in a respectful and empathetic manner. Demonstrates respectability and professional ethics. | Do students consistently demonstrate respect, empathy, and professional ethics while interacting with their instructors, classmates and patients? |
| 5. Careful Delivery of Service: Student follows policies, procedures & protocols. Uses appropriate safeguards in the performance of duties | Do students consistently follow written policies and procedures during their classroom or clinical interactions with fellow students, faculty and/ or patients and preceptors? |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population or Courses Assessed | Hypothetical Analysis/Target |
|---|---|---|---|
| 1. Affective Domain: Be a role model of exemplary professional behavior including: but not limited to, integrity, empathy, self-motivation, appearance/personal hygiene, self-confidence, communications, time management, teamwork/ diplomacy, respect, patient advocacy, and careful delivery of service. (Professionalism) | Behavioral rubric used to gain an overall evaluation of the affective domain of our students. Additionally, there are a couple of questions that address their cognitive and psychomotor abilities. | EMS 1091 | We wish to determine if the rubric gives a consistent, accurate evaluation of our student’s ability to be role models of the professional attributes listed. |
| 2. Preparedness: the student consistently arrived on time with required materials and was ready to learn. Professional appearance: the student is dressed appropriately and is neat in appearance. No hygiene issues. | Behavioral rubric used to gain an overall evaluation of the affective domain of our students. Additionally, there are a couple of questions that address their cognitive and psychomotor abilities. | EMS 1091 | We wish to determine if the rubric will give us a consistent and accurate determination of a student’s timeliness, professional grooming standards, and preparedness to learn in classroom setting. |
| 3. Initiative: student demonstrates interest in EMS through actions and interactions with evaluator. | Behavioral rubric used to gain an overall evaluation of the affective domain of our students. Additionally, there are a couple of questions that address their cognitive and psychomotor abilities. | EMS 1091 | We wish to determine if the rubric will provide a consistent, accurate determination of a student’s interest in the EMS profession through their classroom and clinical interactions. |
| 4. Conduct: Student interacts with other in a respectful and empathetic manner. Demonstrates respectability and professional ethics. | Behavioral rubric used to gain an overall evaluation of the affective domain of our students. Additionally, there are a couple of questions that address their cognitive and psychomotor abilities. | EMS 1091 | We wish to determine if the rubric will provide consistent and accurate determination of a student’s ability to interact with fellow students, faculty, preceptors and patients in a respectful and empathetic manner. Additionally, we wish to determine if use of the rubric can give us a consistent, accurate determination of a students use of professional ethics while in class. |
| 5. Careful Delivery of Service: Student follows policies, procedures & protocols. Uses appropriate safeguards in the performance of duties | Behavioral rubric used to gain an overall evaluation of the affective domain of our students. Additionally, there are a couple of questions that address their cognitive and psychomotor abilities. | EMS 1091 | We wish to determine if the assessment tool adequately gives us a consistent, accurate evaluation of the student’s affective domain. Additionally, we wish to evaluate of the cognitive and psychomotor questions are necessary or relevant to the evaluation. |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Results | Reflection on Focus Area | Intepretation of Results |
|---|---|---|---|
| 1. Affective Domain: Be a role model of exemplary professional behavior including: but not limited to, integrity, empathy, self-motivation, appearance/personal hygiene, self-confidence, communications, time management, teamwork/ diplomacy, respect, patient advocacy, and careful delivery of service. (Professionalism) | All students evaluated exceeded the minimum threshold for this assessment. However, the current assessment tool includes all three learning domains and should be modified or replaced to ensure a more specific focus on affective assessment. | The results support that students score well on the rubric. However, due to the rubric also including psychomotor and cognitive assessments, it should be modified or swapped out for a tool that focuses solely on affective performance. | The data shows us that though students perform well when assessed with the current tool, there is room to improve the tool to get a better focus on or students affective performance in classes. |
| 2. Preparedness: the student consistently arrived on time with required materials and was ready to learn. Professional appearance: the student is dressed appropriately and is neat in appearance. No hygiene issues. | All students evaluated exceeded the minimum threshold for this assessment. However, the current assessment tool includes all three learning domains and should be modified or replaced to ensure a more specific focus on affective assessment. | The results support that students score well on the rubric. However, due to the rubric also including psychomotor and cognitive assessments, it should be modified or swapped out for a tool that focuses solely on affective performance. | The data shows us that though students perform well when assessed with the current tool, there is room to improve the tool to get a better focus on or students’ affective performance in classes. |
| 3. Initiative: student demonstrates interest in EMS through actions and interactions with evaluator. | All students evaluated exceeded the minimum threshold for this assessment. However, the current assessment tool includes all three learning domains and should be modified or replaced to ensure a more specific focus on affective assessment. | The results support that students score well on the rubric. However, due to the rubric also including psychomotor and cognitive assessments, it should be modified or swapped out for a tool that focuses solely on affective performance. | The data shows us that though students perform well when assessed with the current tool, there is room to improve the tool to get a better focus on or students’ affective performance in classes. |
| 4. Conduct: Student interacts with other in a respectful and empathetic manner. Demonstrates respectability and professional ethics. | All students evaluated exceeded the minimum threshold for this assessment. However, the current assessment tool includes all three learning domains and should be modified or replaced to ensure a more specific focus on affective assessment. | The results support that students score well on the rubric. However, due to the rubric also including psychomotor and cognitive assessments, it should be modified or swapped out for a tool that focuses solely on affective performance. | The data shows us that though students perform well when assessed with the current tool, there is room to improve the tool to get a better focus on or students’ affective performance in classes. |
| 5. Careful Delivery of Service: Student follows policies, procedures & protocols. Uses appropriate safeguards in the performance of duties | All students evaluated exceeded the minimum threshold for this assessment. However, the current assessment tool includes all three learning domains and should be modified or replaced to ensure a more specific focus on affective assessment. | The results support that students score well on the rubric. However, due to the rubric also including psychomotor and cognitive assessments, it should be modified or swapped out for a tool that focuses solely on affective performance. | The data shows us that though students perform well when assessed with the current tool, there is room to improve the tool to get a better focus on or students’ affective performance in classes. |
| 1. Affective Domain: Be a role model of exemplary professional behavior including: but not limited to, integrity, empathy, self-motivation, appearance/personal hygiene, self-confidence, communications, time management, teamwork/ diplomacy, respect, patient advocacy, and careful delivery of service. (Professionalism) | |
|---|---|
| Describe the change that was implemented. | Though we are currently having good outcomes, we feel that we can get outcomes that better focus on the intended affective domain outcomes with a different rubric. Therefore, we plan to implement a new rubric that focuses solely on affective behaviors. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, a new behavioral (affective) rubric will be implemented. |
| What data motivated the change? | The old rubric included psychomotor and cognitive evaluations that detracted from the affective focus. We feel it best to have an evaluation that solely focuses on affective traits in our affective evaluations. |
| Hypothesis about the effect the change will have? | We expect that by changing to a new affective rubric that solely focuses on affective traits, we feel that we will be able to gain more accurate affective evaluations. |
| 2. Preparedness: the student consistently arrived on time with required materials and was ready to learn. Professional appearance: the student is dressed appropriately and is neat in appearance. No hygiene issues. | |
|---|---|
| Describe the change that was implemented. | Though we are currently having good outcomes, we feel that we can get outcomes that better focus on the intended affective domain outcomes with a different rubric. Therefore, we plan to implement a new rubric that focuses solely on affective behaviors. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, a new behavioral (affective) rubric will be implemented. |
| What data motivated the change? | The old rubric included psychomotor and cognitive evaluations that detracted from the affective focus. We feel it best to have an evaluation that solely focuses on affective traits in our affective evaluations. |
| Hypothesis about the effect the change will have? | We expect that by changing to a new affective rubric that solely focuses on affective traits, we feel that we will be able to gain more accurate affective evaluations. |
| 3. Initiative: student demonstrates interest in EMS through actions and interactions with evaluator. | |
|---|---|
| Describe the change that was implemented. | Though we are currently having good outcomes, we feel that we can get outcomes that better focus on the intended affective domain outcomes with a different rubric. Therefore, we plan to implement a new rubric that focuses solely on affective behaviors. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, a new behavioral (affective) rubric will be implemented. |
| What data motivated the change? | The old rubric included psychomotor and cognitive evaluations that detracted from the affective focus. We feel it best to have an evaluation that solely focuses on affective traits in our affective evaluations. |
| Hypothesis about the effect the change will have? | We expect that by changing to a new affective rubric that solely focuses on affective traits, we feel that we will be able to gain more accurate affective evaluations. |
| 4. Conduct: Student interacts with other in a respectful and empathetic manner. Demonstrates respectability and professional ethics. | |
|---|---|
| Describe the change that was implemented. | Though we are currently having good outcomes, we feel that we can get outcomes that better focus on the intended affective domain outcomes with a different rubric. Therefore, we plan to implement a new rubric that focuses solely on affective behaviors. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, a new behavioral (affective) rubric will be implemented. |
| What data motivated the change? | The old rubric included psychomotor and cognitive evaluations that detracted from the affective focus. We feel it best to have an evaluation that solely focuses on affective traits in our affective evaluations. |
| Hypothesis about the effect the change will have? | We expect that by changing to a new affective rubric that solely focuses on affective traits, we feel that we will be able to gain more accurate affective evaluations. |
| 5. Careful Delivery of Service: Student follows policies, procedures & protocols. Uses appropriate safeguards in the performance of duties | |
|---|---|
| Describe the change that was implemented. | Though we are currently having good outcomes, we feel that we can get outcomes that better focus on the intended affective domain outcomes with a different rubric. Therefore, we plan to implement a new rubric that focuses solely on affective behaviors. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, a new behavioral (affective) rubric will be implemented. |
| What data motivated the change? | The old rubric included psychomotor and cognitive evaluations that detracted from the affective focus. We feel it best to have an evaluation that solely focuses on affective traits in our affective evaluations. |
| Hypothesis about the effect the change will have? | We expect that by changing to a new affective rubric that solely focuses on affective traits, we feel that we will be able to gain more accurate affective evaluations. |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population of Courses Assessed |
|---|---|---|
| 1. Affective Domain: Be a role model of exemplary professional behavior including: but not limited to, integrity, empathy, self-motivation, appearance/personal hygiene, self-confidence, communications, time management, teamwork/ diplomacy, respect, patient advocacy, and careful delivery of service. (Professionalism) | Unlike the old rubric, the new one solely focuses on affective traits, allowing for a clearer evaluation of a student’s affective behaviors. As such, it will our faculty to better coach student toward becoming entry-level competent in the affective domain. | EMS 1001 and EMS 1091 |
| 2. Preparedness: the student consistently arrived on time with required materials and was ready to learn. Professional appearance: the student is dressed appropriately and is neat in appearance. No hygiene issues. | Unlike the old rubric, the new one solely focuses on affective traits, allowing for a clearer evaluation of a student’s affective behaviors. As such, it will our faculty to better coach student toward becoming entry-level competent in the affective domain. | EMS 1001 and EMS 1091 |
| 3. Initiative: student demonstrates interest in EMS through actions and interactions with evaluator. | Unlike the old rubric, the new one solely focuses on affective traits, allowing for a clearer evaluation of a student’s affective behaviors. As such, it will our faculty to better coach student toward becoming entry-level competent in the affective domain. | EMS 1001 and EMS 1091 |
| 4. Conduct: Student interacts with other in a respectful and empathetic manner. Demonstrates respectability and professional ethics. | Unlike the old rubric, the new one solely focuses on affective traits, allowing for a clearer evaluation of a student’s affective behaviors. As such, it will our faculty to better coach student toward becoming entry-level competent in the affective domain. | EMS 1001 and EMS 1091 |
| 5. Careful Delivery of Service: Student follows policies, procedures & protocols. Uses appropriate safeguards in the performance of duties | Unlike the old rubric, the new one solely focuses on affective traits, allowing for a clearer evaluation of a student’s affective behaviors. As such, it will our faculty to better coach student toward becoming entry-level competent in the affective domain. | EMS 1001 and EMS 1091 |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Second Round Results | Intepretation of Results, Pre- and Post-Change | Follow up questions, possible next steps |
|---|---|---|---|
| 1. Affective Domain: Be a role model of exemplary professional behavior including: but not limited to, integrity, empathy, self-motivation, appearance/personal hygiene, self-confidence, communications, time management, teamwork/ diplomacy, respect, patient advocacy, and careful delivery of service. (Professionalism) | As this course has only run twice over the past two years with a total of 17 students, the data is not conclusive but points to the same outcomes seen in the other skill levels. Overall the implementation of the three planned changes led to mostly positive outcomes, but it wasn’t all fruitful. On the positive side, change one – expanding to assess students in all aspects of the program (theory, lab, clinical and internship), allows us to evaluate all students several times in all courses. These focused affective evaluations are giving us the ability to track a student’s behaviors and give frequent feedback/ coaching to allow them to understand where improvements must be made. It also allows us the ability to use the behavioral tracking when students must be placed on a PIP. Additionally, by implementing a new rubric that exclusively focuses on affective traits, we have seen improvement with scoring students and being able to clearly articulate where students need to work on making improvements. We believe that these improved scores allow our faculty to state whether a student is affectively prepared for field work or not and confidently use the evaluations to support our assertions. Finally, our third change is where we see the mixed results. On the one hand, adding a rubric into our Brightspace shells was wonderful in aiding with faculty buy-in, simplicity with completion (no more paper forms that required manual grade entry, scanning and transfer to SharePoint for long term storage), and lower duplication expenses. It has also been easier to review the completed evaluations without needing to search for them in SharePoint. However, there was one dark spot with this change. Unfortunately, we have found that all rubrics must be downloaded individually from Brightspace, and these downloads can only be done when the course shell is still active. This has proven to be much more time consuming that anticipated and we feel that we may need to consider a move from the rubric in Brightspace to a fillable Word or PDF doc | Outcome needs to be reviewed and updated. | |
| 2. Preparedness: the student consistently arrived on time with required materials and was ready to learn. Professional appearance: the student is dressed appropriately and is neat in appearance. No hygiene issues. | The data shows students are arriving to class prepared for the day with little to no problems. It also shows that faculty are consistent in their grading of a student’s preparedness for class. The anomaly seen in round one is felt to have been the narrow focus of the old rubric. By spreading the scoring for this outcome over three categories on the new rubric – self-motivation, accountability, and appearance/ personal hygiene, and moving from a five-point system (0-4 points) to a three-point system (0–2), allows for decreased ambiguity and lower chances of subjectiveness in grading. Giving us improved focus for our student feedback to drive counseling and coaching. Allowing for clearer direction of our student and improved performance in the affective domain. | Outcome needs to be reviewed and updated. | |
| 3. Initiative: student demonstrates interest in EMS through actions and interactions with evaluator. | The data shows that students are interacting properly and effectively showing their interest in becoming EMS professionals. Moving to evaluating this outcome using six data points – integrity, accountability, respect, self-motivation, self-confidence, and communication provided a clearer picture of a student’s overall performance on this outcome that a single point did not historically yield. Additionally, the move from a five-point system (0-4) to a three-point system (0-2) decreases ambiguity and reduces subjectiveness in grading. Giving us improved focus for our student feedback to drive counseling and coaching. This gives us the opportunity to more clearly direct our students on improving their performance in the affective domain. | Outcome need to be reviewed and updated. | |
| 4. Conduct: Student interacts with other in a respectful and empathetic manner. Demonstrates respectability and professional ethics. | The data shows that students routinely conduct themselves with respect, empathy, and professional ethics during class and clinical rotations. As with other outcomes, the implementation of a new rubric required altering which data points were used to score each outcome. In this case, the decision was made to use a total rubric score that assessed every aspect of affective behavior to grade a student. Additionally, the alteration of the scoring system from a five-point (0-4) to a three-point (0-2) diminished the potential for ambiguity and reduced the potential for subjectivity in grading. Thereby keeping the objective lens in our affective evaluations. Ultimately, we have found that with the new rubric spreading the affective behaviors to individually scored items, we are receiving better feedback to drive our student counseling and coaching. This gives us the opportunity to more clearly direct our students on improving their performance in the affective domain. | Outcome needs to be reviewed and updated. | |
| 5. Careful Delivery of Service: Student follows policies, procedures & protocols. Uses appropriate safeguards in the performance of duties | The data shows that most students consistently follow program policies/ procedures and conduct their care in a professional manner during class and clinical rotations. As with other outcomes, the implementation of a new rubric required altering which data points were used to score each outcome. In this case, the decision was made to move from a single data point to the use of eleven criteria – integrity, compassion, accountability, respect, empathy, self-motivation, self-confidence, communication, teamwork/diplomacy, patient advocacy, and cultural competence to grade students on this outcome. Additionally, the alteration of the scoring system from a five-point (0-4) to a three-point (0-2) diminished the potential for ambiguity and reduced the potential for subjectivity in grading. Thereby keeping the objective lens in our affective evaluations. Ultimately, we have found that with the new rubric spreading the affective behaviors to individually scored items, we are receiving better feedback to drive our student counseling and coaching. This gives us the opportunity to more clearly direct our students on improving their performance in the affective domain. | Outcome needs to be reviewed and updated. |
Describe any change in student achievement observed as part of this assessment process, and what led to those changes.
Yes, we found that by increasing the number and focus of affective evaluations, students were more self-aware of their behavior and tended to correct borderline or inappropriate behaviors before an instructor needed to step in. Additionally, the changes gave my team the data needed to focus our coaching of potentially problematic behaviors before they became a problem. Between the improved self-awareness and focused coaching, we were able to salvage students that would have historically exited from training. By retaining these students, we were able to keep them moving toward becoming well rounded EMS professionals and graduating.
Describe long-term changes in the program(s) that the assessment process led to, and what motivated those changes?
Yes, we have made a few lasting changes. We recognized that our previous affective rubric was utterly inadequate in evaluating a student’s affective performance. Therefore, we have adopted and will continue to utilize a new affective domain rubric that allows us to evaluate students on twelve specific affective traits. Giving my team a clear snapshot of the student’s current performance at any given time. Additionally, because some students behave differently in different classes, we noted that we were not getting a full portrayal of the student’s affective performance in the program. Therefore, we implemented affective evaluations into all theory courses. While it may seem excessive, we found that by evaluating a student five times the program, allowed us to take those snapshots and construct a clear picture of the students overall affective behavior and performance in the program.
What did you learn about the teaching and learning of "Practicing Community" in your programs?
Over the past two years, my team has learned several things regarding the universal theme, “Practicing Community” as it relates to our student’s affective performance in the First Responder (EMR) program. First, considering EMS professionals are given significant public trust, we found that the assessment tool we were using was woefully inadequate in evaluating a student’s true affective performance. Therefore, we adopted a new evaluation tool that allows us to focus directly on a student’s affective traits. With the implementation of the new rubric, we are assessing, tracking, and coaching students on twelve affective traits necessary in EMS professionals - integrity, compassion, accountability, respect, empathy, self-motivation, appearance and personal hygiene, self-confidence, communication, teamwork/ diplomacy, patient advocacy, and cultural competence. Next, we noted that some students tended to let their professional behavior diminish in classes where affective evaluations were not performed. Consequently, we determined the need to ensure students were evaluated in both courses (theory and lab). This has assisted in curbing unprofessional behaviors and increased students’ performance in effective communication, accountability, and increased cultural competence. Additionally, we have learned that by streamlining the grading system for the rubric, there is less ambiguity, allowing for the evaluations to remain objective in the First Responder (EMR) program. Finally, we learned that by evaluating a student several time per course, we were able to use those snapshots to build a portrait of the students overall affective performance in the program. Through this evaluation experience, my faculty have learned that we have the power to make positive changes, hold students accountable, and ensure that when a student graduates, we can confidently say that the student is prepared to enter the EMS industry and handle the honor of public trust.
Describe any external factors affecting the program or affecting assessment of the program.
During the past two years, we have not found any external factors that affected our program or the assessment process.