Review Assessment Report

Part 1: Contact & Program Identification

Report Year and Contact Information
Academic Year	Modified By	Date Modified
2022-2023	[email protected]	2023-10-25T15:51:43.303Z

School	Name of Program	Courses
CHSS	FUTR Flex Gen Ed	FUTR 1110

Part 2: Program Summary

Provide a high level review of the program to include highlights, successes, challenges, significant changes, and significant resources needed to support the program
Interest in FUTR 1110 seems steady, with one section offered in each term during 22-23. Some students arrive to the class with no awareness or incorrect beliefs about what the subject matter might be, suggesting that increasing knowledge among advisors about the substance of the course may be helpful. We continue to surprise global futures thinkers with the existence of our course, which could reflect an opportunity for further futures programming at CNM. Note: The Fall 22 section of FUTR 1110 had so few students that no useful data was produced. The ensuing report will therefore discuss assessment results for Spring and Summer 2023.

Provide a high level review of the program to include highlights, successes, challenges, significant changes, and significant resources needed to support the program

Interest in FUTR 1110 seems steady, with one section offered in each term during 22-23. Some students arrive to the class with no awareness or incorrect beliefs about what the subject matter might be, suggesting that increasing knowledge among advisors about the substance of the course may be helpful. We continue to surprise global futures thinkers with the existence of our course, which could reflect an opportunity for further futures programming at CNM. Note: The Fall 22 section of FUTR 1110 had so few students that no useful data was produced. The ensuing report will therefore discuss assessment results for Spring and Summer 2023.

Part 3: Data Review

	2020-2021	2021-2022	2022-2023
Annual number of graduate awards is greater than 10	n/a	n/a	n/a
Number of declared majors	n/a	n/a	n/a
Average Class Size	24	26.5	18.7
Annual Average Class withdrawal rate is 30% or below (SAGE 35%)	21%	15%	21%
Annual C-Pass rate for coursework is 60% or above	79%	77%	66%
Average class fill rate at 60% or above capacity within a term or over a year	77%	85%	60%
Graduate Transfer to 4-year Schools	n/a	n/a	n/a
Full-time Faculty Coverage by Section	0%	0%	0%

Summarize how your program met or did not meet the target measures based on the data above
The dip in average class size and fill rate is due to two factors in 22-23: A hybrid course offering in Fall 22 that was low enrolled but allowed to stay on the schedule because of the trial format, and a Summer 23 course that was two-thirds full. These smaller total numbers may be contributing to the lower pass rate. The course has only one instructor who uses an ungrading scheme and C-pass contract grading, so students who complete the term as attendees can meet the criteria for passing, but students who do not respond or otherwise disappear from participation later in the course do not complete the required end-of-term activities needed to pass the class.

Summarize how your program met or did not meet the target measures based on the data above

The dip in average class size and fill rate is due to two factors in 22-23: A hybrid course offering in Fall 22 that was low enrolled but allowed to stay on the schedule because of the trial format, and a Summer 23 course that was two-thirds full. These smaller total numbers may be contributing to the lower pass rate. The course has only one instructor who uses an ungrading scheme and C-pass contract grading, so students who complete the term as attendees can meet the criteria for passing, but students who do not respond or otherwise disappear from participation later in the course do not complete the required end-of-term activities needed to pass the class.

Part 4: Program Learning Outcome Analysis

Learning Outcome	Population or Course(s) Assessed	Description	Summary of Assessment Results
3c. Information and Digital Literacy: Information Structures	FUTR 1110	Other Homework Assignments Class Project	Information structures are assessed in some practice assignments that ask students to gather information and sort it in various ways before performing similar tasks on longer, independent scenarios projects. These can be written or voice-over presentation assignments. In spring 23, about half of completing students (12) performed at a higher competency level on the major assignments, including these scenarios assignments than the competency level across all assignments that were measured. Slightly less than half (10) performed at a lower level. In summer 23, slightly more than about half of completing students (8) performed at a higher competency level on the major assignments, including these scenarios assignments than the competency level across all assignments that were measured. Slightly less than half (5) performed at a lower level. Students in both terms scored higher on the more structured projects with more practice.
3a. Information and Digital Literacy: Authority and Value of Information	FUTR 1110	Class Project Homework Assignments In-class Activities Other	This skill is assessed mostly through students identifying signals or drivers of change through "scanning." This is implied in the scenarios projects, but separately assessed through specific, recurring practice assignments. Note that scanning does not always imply particular judgments about the relative merits of a signal's source, except that weak signals tend not to be widespread or found more often in niche sources. In spring 23, about half of completing students completed a key formative assessment of this skill, which was also related to the technology used in the assessment. In summer 23, after a change to the means of collecting student work in this area, greater than two-thirds of completing students submitted assignments, with the mode of the qualitative assessment of students' competency at 3 out of 4. Most students performed at a consistent level, while a few increased competence, and even fewer decreased.
3b. Information and Digital Literacy: Digital Literacy	FUTR 1110	In-class Activities Homework Assignments Class Project	Students in both assessed sections completed activities in the Google Jam Board environment, as well as submitting assignments in standard MS software such as Word or PowerPoint. In Spring 23, students accessed an Excel spreadsheet via MS Teams, while students in Summer 23 accessed Google Sheets documents. In Spring 23, assignments requiring access to Excel through Teams had consistently low participation rates (near 50 percent) compared with other assignments (60 percent or greater). The Google Jam Board assignments ranged from a low participation rate of 42 percent to 61 percent. In Summer 23, assignments requiring access to Google sheets had completion rates of 83 percent and 94 percent, respectively. Jam Board activities ranged from a low participation rate of 44 percent to 78 percent. Making access to the different digital environments easier seems to have increased participation from spring to summer. Competency of use was not assessed.
1a. Communication: Genre and Medium Awareness, Application, and Versatility	FUTR 1110	In-class Activities Class Project Homework Assignments Other	Competence of genre awareness was not assessed, but students in both classes successfully completed a range of information presentation activities from personal reflection and metacognition of their learning, a simulation diary, and formal forecast or scenarios assignments. No difficulties in following the appropriate format for the style of writing were noticed.
1b. Communication: Strategies for Understanding and Evaluating Messages	FUTR 1110	In-class Activities Class Project Homework Assignments Other	Similar to genre awareness, competence of understanding and evaluating messages is not directly assessed. Students in both sections scan for signals of change across a variety of mediums, from news reports to opinion blogs to scientific or technical papers and distilled appropriate information regardless of source.
1c. Communication: Evaluation and Production of Arguments	FUTR 1110	In-class Activities Class Project Other Homework Assignments	Students demonstrate understanding of plausible scenarios through formative homework assignments and longer scenarios projects. These involve identifying a variety of signals of change and making judgments about trends and their directions, as well as counter trends or stabilizing forces in developing one or more scenarios or forecasts for a future time horizon of at least 10 years out. (In some cases, the time horizon is 20 or 30 years.) For spring 23, the average competence assessment for both scenarios assignments was 2.8. More speculative assignments scored at 3.8 and 4, respectively. For summer 23, the average competency assessment was 3.7 out of 4 for the first, more highly structured scenarios assignment, and 2.75 for the second and less structured assignment. Ending the course with an A correlated highly with a 4 on the first scenarios assignment. More speculative assignments for this group tended to score lower.

Interpretation of Assessment findings
Some interpretation is noted above. In general, students in summer performed better with easier access to some of the tools than the students in spring who had to learn and navigate MS Teams. Further, reconceptualizing the scanning assignments as separate weekly discussion tasks made completing the tasks easier or more obvious versus requiring students to navigate to a different platform to complete the task. This suggests changes in student course navigation and ease of design is important for increasing student performance generally, regardless of competence. Students seemed to do better at assignments with more practice with the structure, though both groups had similar opportunities and the summer group scored meaningfully higher on the more structured summative assessment related to scenarios building. Students had many choices for how to complete the unstructured assignments , and this may have inadvertently undermined students confidence to carry out the task across both sections.

Interpretation of Assessment findings

Some interpretation is noted above. In general, students in summer performed better with easier access to some of the tools than the students in spring who had to learn and navigate MS Teams. Further, reconceptualizing the scanning assignments as separate weekly discussion tasks made completing the tasks easier or more obvious versus requiring students to navigate to a different platform to complete the task. This suggests changes in student course navigation and ease of design is important for increasing student performance generally, regardless of competence. Students seemed to do better at assignments with more practice with the structure, though both groups had similar opportunities and the summer group scored meaningfully higher on the more structured summative assessment related to scenarios building. Students had many choices for how to complete the unstructured assignments , and this may have inadvertently undermined students confidence to carry out the task across both sections.

Part 5: Additional Action Plan in Support of Student Learning (If Appropriate)

Upcoming Year	Changes Planned for the upcoming year	Data Motivating this change
2022-2023	Seek opportunities to simplify how students complete assignments.	Data show more students complete assignments and more successfully when access is easier.
2022-2023	Create clearer explanations for futures techniques that are less formal and create more structured instructions for alternative means of completing scenarios assignments.	Both groups struggled with too much choice and creativity in applying less formal futures techniques.
2022-2023

Please select all of the following that characterize the types of changes described in the above action plan

Back