Archived Assessment Report
| Program | POLS Social Behavioral Gen Ed |
| Assessment Reporter | [email protected] |
| Theme | Practicing Community |
| Review Year | 2024-2025 - Final Report |
| Learning Outcome (or Gen Ed Essential Skill) | Focus Area |
|---|---|
| 3a. Personal and Social Responsibility: Intercultural reasoning and intercultural competence | Students focus solely on social media and ignore the concepts of framing and bias in media in general. |
| 3e. Personal and Social Responsibility: Civic discourse, civic knowledge and engagement – local and global | Students are able to summarize the compromises involved in the framing of the U.S. Constitution but are not able to analyze the impact of these compromises on the current political system. |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population or Courses Assessed | Hypothetical Analysis/Target |
|---|---|---|---|
| 3a. Personal and Social Responsibility: Intercultural reasoning and intercultural competence | Short response/essay on the power of the media in American democracy. | POLS 1120 | Students who demonstrate evidence of reading the textbook section on framing and bias in the media are able to comprehensively respond to the question prompt. Our threshold will be 80% and higher compared to any students who scored under 80% on the question prompt on intercultural reasoning. |
| 3e. Personal and Social Responsibility: Civic discourse, civic knowledge and engagement – local and global | Short response/essay on compromises in framing the U.S. Constitution | POLS 1120 | Students who grasp Senate seat malapportionment and the role of slavery in the framing of the Constitution in their short responses are able to explain and support their own positions while recognizing that there may be multiple valid perspectives. |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Results | Reflection on Focus Area | Intepretation of Results |
|---|---|---|---|
| 3a. Personal and Social Responsibility: Intercultural reasoning and intercultural competence | On the criterion Intercultural Competence 3A, 80 percent scored in the highest range of 9-10 out of 10. Fifteen percent of the students scored within the next highest range of 6 to 8 points. Only one student out of 20 scored within the lowest rung (0 to 5 points). The average score was 88 percent. | Contrary to expectations, only one learner out of 20 focused only on social media, ignoring traditional media and framing theory. | Overall, the learners’ demonstration of intercultural reasoning and intercultural competence was very strong. The only student who scored in the lowest rung did so because his response failed to incorporate any of the assigned reading. A fifth of the cohort revised their first submissions; their second drafts were substantially improved. The cultural concepts grasped included framing, priming, cultivation theory, cultural polarization, and echo chambers. As one student observed, the media shapes the “way people think, act, and vote.” The students comprehensively understood how the media contributes to the creation and development of political sub-cultures in American politics. Those students scoring within the 60 to 80 percent range did so because they failed to read the assigned chapter section on cultivation theory, framing, and bias. |
| 3e. Personal and Social Responsibility: Civic discourse, civic knowledge and engagement – local and global | On the criterion Civic Discourse, Knowledge, and Engagement 3E, 80 percent scored in the highest range of 9-10 out of 10. Twenty percent of the students scored within the next highest range of 6 to 8 points. No students out of 21 scored within the lowest rung (0 to 5 points). The average score was 94 percent. | As expected, students are able to summarize the compromises involved in the framing of the U.S. Constitution but were not able to analyze the impact of these compromises on the current political system, particularly the impact of Senate malapportionment on the principle of “one person, one vote.” | There were several exemplary essays on the compromises involved in the framing of the U.S. constitution, a critical component of civic knowledge. Every learner demonstrated close attention to the hour-long podcast episode. While the podcast details the compromise that every state, no matter how small, receives two senators, most learners were not able to extrapolate to the larger question of representation of the whole population. Only one student out of 21 incorrectly inferred that larger states have more power in the Senate. A few learners left out compromises involving the election of the president or Electoral College, as well as the overall power of the institution of the presidency. All learners did not neglect the Three-fifths Compromise, but interpretation of its impact ranged from the Civil War to inequalities of wealth and power. Overall, the hypothesis for 3E held, and students were able to gain a more balanced perspective on the U.S. Constitution. |
| 3a. Personal and Social Responsibility: Intercultural reasoning and intercultural competence | |
|---|---|
| Describe the change that was implemented. | The SAS data was difficult to navigate, 19 different assessments were used to collect data and faculty engagement in the process varied making it difficult to understand where there may be room for improvement. Standardizing assessment tools and grading will be necessary going forward. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, for consistency. Or we can collect data only from a representative sample of courses. |
| What data motivated the change? | No consistent assessment tool or rubric used |
| Hypothesis about the effect the change will have? | More consistent processes will provide useful data |
| 3e. Personal and Social Responsibility: Civic discourse, civic knowledge and engagement – local and global | |
|---|---|
| Describe the change that was implemented. | The SAS data was difficult to navigate, 19 different assessments were used to collect data and faculty engagement in the process varied making it difficult to understand where there may be room for improvement. Standardizing assessment tools and grading will be necessary going forward. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, for consistency. Or we can collect data only from a representative sample of courses. |
| What data motivated the change? | No consistent assessment tool or rubric used |
| Hypothesis about the effect the change will have? | More consistent processes will provide useful data |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population of Courses Assessed |
|---|---|---|
| 3a. Personal and Social Responsibility: Intercultural reasoning and intercultural competence | Written essays | POLS 1120 American National Government |
| 3e. Personal and Social Responsibility: Civic discourse, civic knowledge and engagement – local and global | Written essays | POLS 1120 American National Government |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Second Round Results | Intepretation of Results, Pre- and Post-Change | Follow up questions, possible next steps |
|---|---|---|---|
| 3a. Personal and Social Responsibility: Intercultural reasoning and intercultural competence | On the criterion Intercultural Competence 3A, nearly 70 percent scored in the highest range of 9-10 out of 10. Twenty-three percent of the students scored within the next highest range of 6 to 8 points. Only one student out of 13 scored within the lowest rung (0 to 5 points). The average score was 88 percent. | Instructors should contact students who submit AI-generated products and ask for a brief phone or Zoom interview to verify students’ grasp of the material. Instructors might consider covering only social media, leaving out traditional media from the assigned reading and essay prompt. However, this would omit concepts such as framing theory and media bias from the course curriculum. | |
| 3e. Personal and Social Responsibility: Civic discourse, civic knowledge and engagement – local and global | On the criterion Civic Discourse, Knowledge, and Engagement 3E, 40 percent scored in the highest range of 9-10 out of 10. Forty percent of the students scored within the next highest range of 6 to 8 points. Twenty percent scored within the lowest rung (0 to 5 points). When asked for a revision, one student failed to respond to the email and early alert. The average score was 72 percent. | There is not much improvement needed on civic discourse. Students seem to absorb the material more readily through a podcast that weaves the tale of the framing of the Constitution. Instructors provided an exemplar essay, but it was found that the main problem students have is taking the time to listen to the podcast or read its transcript. These students submit AI-generated essays that do not demonstrate close attention to the details of the assigned material. Instructors might consider calling students who submit AI products and “interviewing” them briefly about what they learned from the material. Instructors might consider creating a separate chart depicting a list of the compromises involved in the framing of the U.S. Constitution for students to review. |
Describe any change in student achievement observed as part of this assessment process, and what led to those changes.
Not enought data to determine at this point
Describe long-term changes in the program(s) that the assessment process led to, and what motivated those changes?
While representing less than a fifth of the cohort assessed, AI-generated submissions by students led to alternate assessment methods such as “make your own podcast” assignments as well as the elimination of essays in favor of discussion boards where peer pressure would incentivize learners to avoid AI-generated work.
Did you see any change in student achievement as part of this assessment process? If so, please describe the changes and what led to them.
Overall, there was no change in student achievement as part of this assessment process. Those students who submitted work consistently scored in the A range on the learning outcomes assessed. Only a few students who submitted work scored in the B range. There continued to be a small segment of each cohort who were not dropped before the census date and who failed to submit any work at all. In short, those students who submitted work did very well. Those students who did poorly failed to submit any work at all
What did you learn about the teaching and learning of "Practicing Community" in your programs?
Both AA Degree and General Education results were strong. For the AA degree, over 80 percent of students were able to incorporate more than one perspective (e.g. source, method, technique, and/or approach) as well as communicate effectively. For the General Education component, over 80 percent of students were able to demonstrate personal and social responsibility through Intercultural reasoning and intercultural competence. They were able to grasp concepts fundamental to media-produced sub-cultures. In addition, most learners showed strong performance in personal and social responsibility through civic discourse, civic knowledge, and engagement – local and global. They absorbed and interpreted the compromises involved in framing the United States Constitution.
The changes introduced—providing exemplar essays, having students submit rough drafts before final drafts—failed to help those few students who failed to submit any work at all. Moreover, instructors are beginning to grapple with the advent and spread of the use of generative artificial intelligence in producing written essays.
Since each instructor used a different assessment tool and rubric for each learning outcome for each course, SAS data was difficult to interpret. Both AA Degree and General Education components hope to streamline and coordinate a common rubric for the next universal theme, Professional Skills.
Describe any external factors affecting the program or affecting assessment of the program.
There were no external factors besides more common use of generative artificial intelligence (AI) such as ChatGPT. During the Spring 2025 and into the Fall 2025 term, some students demonstrated reluctance to share their political viewpoints due to the political climate