Archived Assessment Report
| Program | Artificial Intelligence & Machine Learning AAS |
| Assessment Reporter | [email protected] |
| Theme | Practicing Community |
| Review Year | 2024-2025 - Midpoint Report |
| Learning Outcome (or Gen Ed Essential Skill) | Focus Area |
|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | Students are required to complete assignments using natural language processing, computer vision, and data science. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | Students are required to complete AI project cycle worksheets identifying aspects of the machine learning life cycle and stakeholders. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | Students are required to complete assignments implementing common trade applications such as employee attrition, insurance fraud, predictive maintenance and chatbots. |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | Students are required to complete assignments using natural language processing, computer vision, and data science. |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Students are required to complete assignments identifying ethical issues in artificial intelligence using videos, simulations, and worksheets. Students apply ethics best practices. |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population or Courses Assessed | Hypothetical Analysis/Target |
|---|---|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | Quizzes and Modules | AIML2010, AIML2110, AIML2120 | Students are required to complete assignments using natural language processing, computer vision, and data science. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | Quizzes, AI project cycle worksheets | AIML2010, AIML2120, AIML2110 | The threshold will be 75% completion of the assigned work. Students will be assessed on the substance of the worksheets. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | Use appropriate programming languages to implement artificial intelligence (AI) solutions. | AIML21010, AIML2110 | 75% used as a threshold for students completing the programming implementation |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | Students will build chatbots using the Google Dialogflow tool. | AIML2010, AIML2120 | The course will be assessed on the ability of students to create a simple conversational chatbot. A 75% threshold |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Group assignments. | AIML21010, AIML2110, AIML2120 | 75% is the threshold. Students will be assessed on how well they work together on a group exercise dealing with ethical issues. |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Results | Reflection on Focus Area | Intepretation of Results |
|---|---|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | At least 75% of students completed the Natural Language Processing (NLP) exercises, which included working with tools like Google Dialogflow, creating chatbots, and engaging with the Furhat interface. Additionally, the majority of students completed the associated NLP quizzes. In the Computer Vision assignments, students successfully worked on facial recognition tasks. Lastly, in Data Science, students trained models using Jupyter Notebooks. | The results suggest that the focus area of NLP, computer vision, and data science was effectively integrated into the course. The completion rate of at least 75% indicates that the students engaged well with the exercises, which aligns with the hypothesis that students would be able to learn and apply key concepts in these domains. The exercises seemed to resonate with the intended focus areas, supporting the objective of developing both theoretical understanding and practical skills in AI. | The results show that the majority of students successfully completed the assignments in NLP, computer vision, and data science. This indicates that students were able to grasp the concepts and apply them to real-world tools, such as chatbots, facial recognition, and machine learning models. The high engagement suggests that the course content is relevant and achievable, and that students are comfortable using the technologies involved. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | 95% of students who completed the data science assignments also completed the worksheets that detail the project lifecycle of the neural network training process. The discussion assignments, which focus on business, social, and ethical issues surrounding AI, were also successfully completed by all students. This indicates full engagement in both the technical and ethical aspects of the course. | The results strongly support the focus area of applying AI project development and the machine learning lifecycle to address social and business challenges. The fact that students completed the project lifecycle worksheets for every data science assignment shows that they were actively engaged in understanding the entire machine learning process. Additionally, the discussions surrounding business, social, and ethical issues suggest that students were exposed to the broader implications of AI, aligning well with the goal of addressing real-world problems. | This shows us that students are not only mastering the technical skills necessary for machine learning but are also considering the broader implications of AI on business and society. The completion of both technical worksheets and ethical discussions suggests a well-rounded learning experience, where students are gaining both practical and theoretical knowledge. The high engagement in these areas suggests a solid understanding of the machine learning lifecycle and its potential real-world applications. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | All assignments were designed to align with real-world issues and problems, and students engaged with them by training models using actual data samples. This allowed students to work on practical tasks that are directly relevant to AI and machine learning fields. The assignments are structured in a way that students can easily transfer the skills they have gained to real-world employment opportunities. Common trade applications such as employee attrition, insurance fraud, predictive maintenance and chatbots are the names of actual assignments the students are required ot complete. | The results strongly support the focus area of selecting appropriate machine learning and AI programming solutions. The fact that students worked with real-world data and assignments directly related to industry challenges confirms that the course design aligns with the goal of equipping students with relevant, applicable skills. This supports the hypothesis that students can gain skills that are transferable to the workforce by applying AI solutions to real-world problems. | This shows us that the course is effectively bridging the gap between theoretical learning and real-world application. The students not only completed assignments but also acquired skills that can be directly applied in their future careers. The relevance of assignments to actual business and social problems indicates that students are developing practical problem-solving abilities, making them well-prepared for employment in AI-related roles. |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | At least 90% of students completed assignments using the latest state-of-the-art programming languages and tools, primarily Python and Jupyter Notebooks. These assignments were designed to give students hands-on experience with programming languages that are widely used in the AI industry. The high completion rate indicates strong engagement and success in utilizing these technologies. | The results align with the focus area of using appropriate programming languages to implement AI solutions. The students’ engagement with Python and Jupyter Notebooks supports the hypothesis that introducing state-of-the-art tools helps students gain the necessary skills to implement AI solutions. The fact that students successfully completed assignments using these languages reinforces the idea that they are acquiring the relevant technical competencies for AI development. | This shows us that the majority of students are successfully mastering the programming languages and tools needed for AI development. Python, being the primary language used in AI, along with Jupyter Notebooks, provides an effective platform for students to practice and apply machine learning techniques. The high completion rate suggests that students are able to not only understand the theoretical concepts but also effectively implement them in practical scenarios. |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Based on the activities outlined, X% of students participated actively in the discussion assignments and completed the required responses to at least two classmates' posts. Additionally, Y% of students effectively incorporated their understanding of bias, culture, ethics, regulations, and professional expectations into their worksheets and assignments. These participation rates and assignment completions suggest a strong engagement with the topic areas. | The results support the original focus area of encouraging students to critically evaluate the intersections of AI and societal factors. Most students demonstrated comprehension and engagement with the critical areas outlined in the curriculum. However, there may be room to expand the depth of reflection or provide additional guidance for students who only partially addressed these topics in their assignments. | These findings indicate that the majority of students are meeting the learning objectives by actively participating in discussions and completing assignments. However, they also highlight that some students may need additional resources or scaffolding to fully grasp the nuances of ethical, cultural, and regulatory considerations in AI. This suggests that while the curriculum effectively encourages engagement, it may benefit from refinement to support deeper critical thinking and application. |
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | |
|---|---|
| Describe the change that was implemented. | Integrate Prompt Engineering into the Curriculum: Add a module focused on prompt engineering using tools like ChatGPT. .Provide Additional Resources: Create step-by-step video tutorials and visual aids to clarify complex tasks. Establish a Curriculum Update Schedule: Implement regular reviews to incorporate advancements in AI and machine learning tools.. Expand Practical Applications: Introduce case studies and assignments that highlight real-world applications of AI ethics and societal impacts. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | New Assessments for Prompt Engineering: Create quizzes, projects, and practical exercises specifically for the new module on prompt engineering. Rubric Adjustments: Update grading rubrics to account for the expanded focus on real-world applications and ethical considerations in assignments. Feedback Mechanisms: Introduce more detailed feedback processes to guide students through challenging tasks and enhance their learning experience. |
| What data motivated the change? | Completion Rates: At least 75% of students successfully completed assignments in NLP, computer vision, and data science, indicating a solid baseline of engagement and comprehension. Emerging Trends: The growing significance of prompt engineering and conversational AI tools like ChatGPT suggests a need to prepare students for industry demands. Identified Gaps: While students performed well overall, additional resources and clearer guidance could help those who struggle with challenging tasks, ensuring more uniform success. |
| Hypothesis about the effect the change will have? | Improve Understanding and Skills: By adding focused content on prompt engineering and providing more detailed resources, students will gain a deeper understanding of emerging AI tools and concepts. Enhance Engagement: Practical applications and real-world examples will make assignments more relevant and engaging, fostering higher levels of participation and retention. Boost Success Rates: By addressing learning challenges through additional resources, more students will successfully complete assignments and achieve course objectives. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | |
|---|---|
| Describe the change that was implemented. | A Prompt Engineering assignment will be added to the curriculum, which will include practical applications using tools like ChatGPT and DALL-E. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, the changes will require adjustments in assessment tools. New rubrics will be developed to evaluate students’ proficiency in prompt engineering, focusing on creativity, effectiveness, and ethical considerations in using generative AI. Additionally, practical assignments and project-based assessments will be introduced to measure their ability to integrate these tools into the machine learning lifecycle. |
| What data motivated the change? | The emergence of generative AI tools like ChatGPT has created new opportunities for AI applications not considered in the original curriculum. Exposure to these tools is essential for preparing students for the evolving demands of AI-related fields. |
| Hypothesis about the effect the change will have? | Integrating a Prompt Engineering assignment will improve outcomes by equipping students with hands-on experience using state-of-the-art generative AI tools. This will enhance their understanding of interacting with these systems effectively and applying them to real-world problems. By doing so, students will gain practical skills that align with current AI trends, bridging the gap between technical capabilities and real-world applications. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | |
|---|---|
| Describe the change that was implemented. | Additional real-world case studies across diverse industries, including healthcare, business, and social sectors, to provide broader exposure to AI applications. Furthermore, regular discussion assignments will be incorporated to allow students to reflect on their learning and collaborate with peers. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, the changes will require updates to assessment tools. New assignments and rubrics will be developed to evaluate students' ability to analyze and apply knowledge from the added case studies. Participation in discussion sessions will also be assessed based on engagement, critical insights, and contributions to collaborative problem-solving. |
| What data motivated the change? | The results indicate that students are successfully applying their knowledge to real-world problems through assignments aligned with common industry challenges. However, providing a wider range of examples and fostering deeper reflection and collaboration will enhance their practical understanding and adaptability to evolving AI trends. |
| Hypothesis about the effect the change will have? | By adding diverse case studies, students will gain a broader perspective on the applications of AI in various domains, enhancing their ability to solve complex, interdisciplinary problems. Discussion sessions will encourage critical thinking, collaborative learning, and knowledge-sharing, helping students solidify their understanding and apply it effectively in real-world scenarios. |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | |
|---|---|
| Describe the change that was implemented. | Real-world assignments that incorporate emerging libraries, frameworks, and industry-relevant challenges will be incorporated into the currriculm. These assignments will focus on trending areas such as generative AI, reinforcement learning, and large language models. Additionally, periodic reviews will be conducted to update the curriculum with the latest tools and techniques in AI programming. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, the changes will necessitate updates to assessment methods. New rubrics will be designed to evaluate students' proficiency in utilizing advanced tools and libraries. Assignments will include problem-solving in diverse domains and emphasize innovative applications, with assessments measuring implementation skills, creativity, and real-world applicability. |
| What data motivated the change? | The current high completion rates demonstrate students’ ability to engage with Python and Jupyter Notebooks for practical AI applications. However, given the rapid evolution of AI tools and technologies, ensuring that students learn the latest programming methodologies is critical for maintaining their competitiveness in the job market. |
| Hypothesis about the effect the change will have? | Introducing new assignments based on current AI trends will ensure that students are exposed to cutting-edge technologies and industry practices. This will enhance their ability to tackle real-world AI challenges, improve adaptability to new tools, and increase their employability in a fast-changing field. |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | |
|---|---|
| Describe the change that was implemented. | Real-world case studies focusing on AI bias, ethical dilemmas, and cultural and regulatory implications will be incorporated into the curriculum. These will be included in both discussions and assignments. Guest lectures from industry professionals and ethicists will be introduced to provide diverse perspectives. Targeted feedback mechanisms will be implemented to ensure students receive detailed evaluations of their discussion posts and worksheets, promoting deeper critical thinking. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, assessment methods will be updated to include qualitative evaluations of students’ critical thinking in analyzing case studies. Rubrics for discussions and worksheets will be revised to measure depth of understanding and the quality of proposed solutions. Additionally, participation in guest lectures may be assessed through reflective essays or follow-up discussions. |
| What data motivated the change? | Participation rates in discussions and assignments indicate strong engagement but also suggest room for improvement in students’ depth of understanding regarding bias, ethics, and regulatory issues. Incorporating practical examples and expert insights will address these gaps by contextualizing theoretical concepts. |
| Hypothesis about the effect the change will have? | By analyzing real-world case studies and receiving expert insights, students will develop a more nuanced understanding of ethical, cultural, and regulatory challenges in AI. This will enhance their ability to critically evaluate AI applications and propose practical, informed solutions. Feedback on their work will further refine their critical thinking and communication skills. |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population of Courses Assessed |
|---|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | SAS and Brightspace | AIML2010, AIML2110, AIML2120 |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | Brightsapce | AIML2010, AIML2120, AIML2110 |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | Brightspace | AIML21010, AIML2110 |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | Brightsapce | AIML2010, AIML2120 |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Brightspace | AIML21010, AIML2110, AIML2120 |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Second Round Results | Intepretation of Results, Pre- and Post-Change | Follow up questions, possible next steps |
|---|
Describe any change in student achievement observed as part of this assessment process, and what led to those changes.
Describe long-term changes in the program(s) that the assessment process led to, and what motivated those changes?
What did you learn about the teaching and learning of "Practicing Community" in your programs?
Describe any external factors affecting the program or affecting assessment of the program.