Archived Assessment Report
| Program | Artificial Intelligence and Machine Learning Certificate |
| Assessment Reporter | [email protected] |
| Theme | Practicing Community |
| Review Year | 2024-2025 - Final Report |
| Learning Outcome (or Gen Ed Essential Skill) | Focus Area |
|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | Students are required to complete assignments using natural language processing, computer vision, and data science. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | Students are required to complete AI project cycle worksheets identifying aspects of the machine learning life cycle and stakeholders. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | Students are required to complete assignments implementing common trade applications such as employee attrition, insurance fraud, predictive maintenance and chatbots. |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | Students are required to complete assignments using natural language processing, computer vision, and data science. |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Students are required to complete assignments identifying ethical issues in artificial intelligence using videos, simulations, and worksheets. Students apply ethics best practices. |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population or Courses Assessed | Hypothetical Analysis/Target |
|---|---|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | Quizzes, AI project cycle worksheets, Module Assignments | AIML2010, AIML2120, AIML2110 | The threshold will be 75% completion of the assigned work. Students will be assessed on the substance of the worksheet. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | Quizzes, AI project cycle worksheets, Module Assignments | AIML2010, AIML2120, AIML2110 | The threshold will be 75% completion of the assigned work. Students will be assessed on the substance of the module assignments.. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | Quizzes, AI project cycle worksheets, Module Assignments | AIML2010, AIML2120, AIML2110 | The threshold will be 75% completion of the assigned work. Students will be assessed on the substance of the module assignments. |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | Quizzes, AI project cycle worksheets, Module Assignments | AIML2010, AIML2120, AIML2110 | The threshold will be 75% completion of the assigned work. Students will be assessed on the substance of the module assignments. |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Quizzes, AI project cycle worksheets, Module Assignments | AIML2010, AIML2120, AIML2110 | The threshold will be 75% completion of the assigned work. Students will be assessed on the substance of the module assignments. |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Results | Reflection on Focus Area | Intepretation of Results |
|---|---|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | At least 75% of students completed the Natural Language Processing (NLP) exercises, which included working with tools like Google Dialogflow, creating chatbots, and engaging with the Furhat interface. Additionally, the majority of students completed the associated NLP quizzes. In the Computer Vision assignments, students successfully worked on facial recognition tasks. Lastly, in Data Science, students trained models using Jupyter Notebooks. | The results suggest that the focus area of NLP, computer vision, and data science was effectively integrated into the course. The completion rate of at least 75% indicates that the students engaged well with the exercises, which aligns with the hypothesis that students would be able to learn and apply key concepts in these domains. The exercises seemed to resonate with the intended focus areas, supporting the objective of developing both theoretical understanding and practical skills in AI. | The results show that the majority of students successfully completed the assignments in NLP, computer vision, and data science. This indicates that students were able to grasp the concepts and apply them to real-world tools, such as chatbots, facial recognition, and machine learning models. The high engagement suggests that the course content is relevant and achievable and that students are comfortable using the technologies involved. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | 95% of students who completed the data science assignments also completed the worksheets that detail the project lifecycle of the neural network training process. The discussion assignments, which focus on business, social, and ethical issues surrounding AI, were also successfully completed by all students. This indicates full engagement in both the technical and ethical aspects of the course. | The results strongly support the focus area of applying AI project development and the machine learning lifecycle to address social and business challenges. The fact that students completed the project lifecycle worksheets for every data science assignment shows that they were actively engaged in understanding the entire machine learning process. Additionally, the discussions surrounding business, social, and ethical issues suggest that students were exposed to the broader implications of AI, aligning well with the goal of addressing real-world problems. | This shows us that students are not only mastering the technical skills necessary for machine learning but are also considering the broader implications of AI on business and society. The completion of both technical worksheets and ethical discussions suggests a well-rounded learning experience, where students are gaining both practical and theoretical knowledge. The high engagement in these areas suggests a solid understanding of the machine learning lifecycle and its potential real-world applications. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | All assignments were designed to align with real-world issues and problems, and students engaged with them by training models using actual data samples. This allowed students to work on practical tasks that are directly relevant to AI and machine learning fields. The assignments are structured in a way that students can easily transfer the skills they have gained to real-world employment opportunities. Common trade applications such as employee attrition, insurance fraud, predictive maintenance and chatbots are the names of actual assignments the students are required ot complete. | The results strongly support the focus area of selecting appropriate machine learning and AI programming solutions. The fact that students worked with real-world data and assignments directly related to industry challenges confirms that the course design aligns with the goal of equipping students with relevant, applicable skills. This supports the hypothesis that students can gain skills that are transferable to the workforce by applying AI solutions to real-world problems. | This shows us that the course is effectively bridging the gap between theoretical learning and real-world application. The students not only completed assignments but also acquired skills that can be directly applied in their future careers. The relevance of assignments to actual business and social problems indicates that students are developing practical problem-solving abilities, making them well-prepared for employment in AI-related roles. |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | At least 90% of students completed assignments using the latest state-of-the-art programming languages and tools, primarily Python and Jupyter Notebooks. These assignments were designed to give students hands-on experience with programming languages that are widely used in the AI industry. The high completion rate indicates strong engagement and success in utilizing these technologies. | The results align with the focus area of using appropriate programming languages to implement AI solutions. The students’ engagement with Python and Jupyter Notebooks supports the hypothesis that introducing state-of-the-art tools helps students gain the necessary skills to implement AI solutions. The fact that students successfully completed assignments using these languages reinforces the idea that they are acquiring the relevant technical competencies for AI development. | This shows us that the majority of students are successfully mastering the programming languages and tools needed for AI development. Python, being the primary language used in AI, along with Jupyter Notebooks, provides an effective platform for students to practice and apply machine learning techniques. The high completion rate suggests that students are able to not only understand the theoretical concepts but also effectively implement them in practical scenarios. |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Based on the activities outlined, X% of students participated actively in the discussion assignments and completed the required responses to at least two classmates' posts. Additionally, Y% of students effectively incorporated their understanding of bias, culture, ethics, regulations, and professional expectations into their worksheets and assignments. These participation rates and assignment completions suggest a strong engagement with the topic areas. | The results support the original focus area of encouraging students to critically evaluate the intersections of AI and societal factors. Most students demonstrated comprehension and engagement with the critical areas outlined in the curriculum. However, there may be room to expand the depth of reflection or provide additional guidance for students who only partially addressed these topics in their assignments. | These findings indicate that the majority of students are meeting the learning objectives by actively participating in discussions and completing assignments. However, they also highlight that some students may need additional resources or scaffolding to fully grasp the nuances of ethical, cultural, and regulatory considerations in AI. This suggests that while the curriculum effectively encourages engagement, it may benefit from refinement to support deeper critical thinking and application. |
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | |
|---|---|
| Describe the change that was implemented. | Integrate Prompt Engineering into the Curriculum: Add a module focused on prompt engineering using tools like ChatGPT. .Provide Additional Resources: Create step-by-step video tutorials and visual aids to clarify complex tasks. Establish a Curriculum Update Schedule: Implement regular reviews to incorporate advancements in AI and machine learning tools.. Expand Practical Applications: Introduce case studies and assignments that highlight real-world applications of AI ethics and societal impacts. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | New Assessments for Prompt Engineering: Create quizzes, projects, and practical exercises specifically for the new module on prompt engineering. Rubric Adjustments: Update grading rubrics to account for the expanded focus on real-world applications and ethical considerations in assignments. Feedback Mechanisms: Introduce more detailed feedback processes to guide students through challenging tasks and enhance their learning experience. |
| What data motivated the change? | Completion Rates: At least 75% of students successfully completed assignments in NLP, computer vision, and data science, indicating a solid baseline of engagement and comprehension. Emerging Trends: The growing significance of prompt engineering and conversational AI tools like ChatGPT suggests a need to prepare students for industry demands. Identified Gaps: While students performed well overall, additional resources and clearer guidance could help those who struggle with challenging tasks, ensuring more uniform success. |
| Hypothesis about the effect the change will have? | Improve Understanding and Skills: By adding focused content on prompt engineering and providing more detailed resources, students will gain a deeper understanding of emerging AI tools and concepts. Enhance Engagement: Practical applications and real-world examples will make assignments more relevant and engaging, fostering higher levels of participation and retention. Boost Success Rates: By addressing learning challenges through additional resources, more students will successfully complete assignments and achieve course objectives. |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | |
|---|---|
| Describe the change that was implemented. | A Prompt Engineering assignment will be added to the curriculum, which will include practical applications using tools like ChatGPT and DALL-E. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, the changes will require adjustments in assessment tools. New rubrics will be developed to evaluate students’ proficiency in prompt engineering, focusing on creativity, effectiveness, and ethical considerations in using generative AI. Additionally, practical assignments and project-based assessments will be introduced to measure their ability to integrate these tools into the machine learning lifecycle. |
| What data motivated the change? | The emergence of generative AI tools like ChatGPT has created new opportunities for AI applications not considered in the original curriculum. Exposure to these tools is essential for preparing students for the evolving demands of AI-related fields. |
| Hypothesis about the effect the change will have? | Integrating a Prompt Engineering assignment will improve outcomes by equipping students with hands-on experience using state-of-the-art generative AI tools. This will enhance their understanding of interacting with these systems effectively and applying them to real-world problems. By doing so, students will gain practical skills that align with current AI trends, bridging the gap between technical capabilities and real-world applications. |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | |
|---|---|
| Describe the change that was implemented. | Additional real-world case studies across diverse industries, including healthcare, business, and social sectors, to provide broader exposure to AI applications. Furthermore, regular discussion assignments will be incorporated to allow students to reflect on their learning and collaborate with peers. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, the changes will require updates to assessment tools. New assignments and rubrics will be developed to evaluate students' ability to analyze and apply knowledge from the added case studies. Participation in discussion sessions will also be assessed based on engagement, critical insights, and contributions to collaborative problem-solving. |
| What data motivated the change? | The results indicate that students are successfully applying their knowledge to real-world problems through assignments aligned with common industry challenges. However, providing a wider range of examples and fostering deeper reflection and collaboration will enhance their practical understanding and adaptability to evolving AI trends. |
| Hypothesis about the effect the change will have? | By adding diverse case studies, students will gain a broader perspective on the applications of AI in various domains, enhancing their ability to solve complex, interdisciplinary problems. Discussion sessions will encourage critical thinking, collaborative learning, and knowledge-sharing, helping students solidify their understanding and apply it effectively in real-world scenarios. |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | |
|---|---|
| Describe the change that was implemented. | Real-world assignments that incorporate emerging libraries, frameworks, and industry-relevant challenges will be incorporated into the currriculm. These assignments will focus on trending areas such as generative AI, reinforcement learning, and large language models. Additionally, periodic reviews will be conducted to update the curriculum with the latest tools and techniques in AI programming. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, the changes will necessitate updates to assessment methods. New rubrics will be designed to evaluate students' proficiency in utilizing advanced tools and libraries. Assignments will include problem-solving in diverse domains and emphasize innovative applications, with assessments measuring implementation skills, creativity, and real-world applicability. |
| What data motivated the change? | The current high completion rates demonstrate students’ ability to engage with Python and Jupyter Notebooks for practical AI applications. However, given the rapid evolution of AI tools and technologies, ensuring that students learn the latest programming methodologies is critical for maintaining their competitiveness in the job market. |
| Hypothesis about the effect the change will have? | Introducing new assignments based on current AI trends will ensure that students are exposed to cutting-edge technologies and industry practices. This will enhance their ability to tackle real-world AI challenges, improve adaptability to new tools, and increase their employability in a fast-changing field. |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | |
|---|---|
| Describe the change that was implemented. | Real-world case studies focusing on AI bias, ethical dilemmas, and cultural and regulatory implications will be incorporated into the curriculum. These will be included in both discussions and assignments. Guest lectures from industry professionals and ethicists will be introduced to provide diverse perspectives. Targeted feedback mechanisms will be implemented to ensure students receive detailed evaluations of their discussion posts and worksheets, promoting deeper critical thinking. |
| Type of Change |
|
| Change in Assessment Approach or Tools? | Yes, assessment methods will be updated to include qualitative evaluations of students’ critical thinking in analyzing case studies. Rubrics for discussions and worksheets will be revised to measure depth of understanding and the quality of proposed solutions. Additionally, participation in guest lectures may be assessed through reflective essays or follow-up discussions. |
| What data motivated the change? | Participation rates in discussions and assignments indicate strong engagement but also suggest room for improvement in students’ depth of understanding regarding bias, ethics, and regulatory issues. Incorporating practical examples and expert insights will address these gaps by contextualizing theoretical concepts. |
| Hypothesis about the effect the change will have? | By analyzing real-world case studies and receiving expert insights, students will develop a more nuanced understanding of ethical, cultural, and regulatory challenges in AI. This will enhance their ability to critically evaluate AI applications and propose practical, informed solutions. Feedback on their work will further refine their critical thinking and communication skills. |
| Learning Outcome (or Gen Ed Essential Skill) | Description of Assessment Tool | Population of Courses Assessed |
|---|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | Brightspace | AIML2010, AIML2120, AIML2110 |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | Brighspace | AIML2010, AIML2120, AIML2110 |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | Brightspace | AIML2010, AIML2120, AIML2110 |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | Brightspace | AIML2010, AIML2120, AIML2110 |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Brightspace | AIML2010, AIML2120, AIML2110 |
| Learning Outcome (or Gen Ed Essential Skill) | Summary of Second Round Results | Intepretation of Results, Pre- and Post-Change | Follow up questions, possible next steps |
|---|---|---|---|
| 1. Apply common artificial intelligence (AI) concepts and methodologies for analysis and decision making. | In the second round, students continued completing assignments in natural language processing, computer vision, and data science using Brightspace. Most students successfully applied AI concepts and methodologies to analyze and solve problems, including building chatbots with Google Dialogflow, engaging with the Furhat interface, completing facial recognition tasks, and training models in Jupyter Notebooks. New exercises in prompt engineering introduced students to emerging industry trends, and detailed feedback helped guide learning in more complex tasks. Completion rates remained high, demonstrating consistent engagement and the ability to translate theoretical AI concepts into practical solutions. | Future assessments could expand on prompt engineering, AI ethics, and conversational AI applications to reflect industry evolution. Rubrics may be updated to assess the quality of decision-making, reasoning, and innovative application of AI concepts. Additional resources, guided tutorials, and collaborative problem-solving exercises could help students struggling with complex tasks and ensure more uniform success across all course modules. | |
| 2. Apply artificial intelligence (AI) project development and machine learning life cycle to address social and business issues, opportunities, and problems. | In the second round of assessment, students continued to complete AI project cycle worksheets and discussion assignments, now including exercises involving generative AI tools such as ChatGPT. Preliminary results indicate that students are effectively identifying machine learning lifecycle components and considering stakeholders, with added engagement in creative prompt engineering and ethical applications of generative AI. Most students demonstrated competency in integrating these tools into project-based assignments, though variability was observed in the sophistication and ethical framing of their solutions. Overall, the second round shows that students are not only retaining technical skills but also adapting to emerging AI technologies. | How can assessment rubrics be further refined to measure creativity and ethical reasoning in generative AI projects? Should additional professional scenarios or stakeholder analyses be incorporated to simulate real-world AI decision-making? What targeted interventions or resources can help students who show lower proficiency in prompt engineering or ethical analysis? Explore longitudinal tracking of graduates to evaluate how effectively the curriculum prepares them for AI-related career challenges in dynamic environments. | |
| 3. Select appropriate machine learning and artificial intelligence (AI) programming solutions. | In the second round of assessment, students continued to complete assignments implementing real-world AI applications, including employee attrition, insurance fraud detection, predictive maintenance, and chatbots. New case studies and discussion sessions were introduced to encourage deeper engagement, collaboration, and critical reflection. Preliminary observations indicate that students are effectively selecting and applying appropriate AI and machine learning programming solutions to these problems. Most students demonstrated proficiency in analyzing datasets, choosing suitable models, and implementing solutions in a way that mirrors real-world industry practices. Some variability was noted in the sophistication of model optimization and collaborative problem-solving. | How can assessment rubrics be further refined to evaluate critical thinking, collaboration, and model optimization skills more comprehensively? Should additional industry case studies or emerging AI applications be incorporated to keep the curriculum aligned with evolving trends? Are there targeted interventions to support students who demonstrate lower proficiency in applying programming solutions to complex, real-world scenarios? Explore opportunities for peer review and team-based projects to enhance collaborative problem-solving and practical implementation skills. | |
| 4. Use appropriate programming languages to implement artificial intelligence (AI) solutions. | Yes. The assessment highlighted the importance of keeping programming instruction aligned with evolving industry standards. As a result, the certificate program is incorporating additional exercises using advanced AI libraries, such as TensorFlow, PyTorch, and scikit-learn, along with new modules on prompt engineering and conversational AI. Rubrics were updated to evaluate not only coding proficiency but also the strategic use of libraries, code optimization, and innovative problem-solving. The changes were driven by observed student success with Python and Jupyter Notebooks and the need to prepare students for emerging trends in AI implementation. | External factors include rapid advancements in AI programming tools and industry expectations for proficiency in multiple libraries and frameworks. The emergence of conversational AI and prompt engineering influenced curriculum updates, requiring the integration of new technologies and exercises. Additionally, students’ prior exposure to online AI resources and varying levels of programming experience impacted engagement and performance. Remote learning and access to cloud-based AI platforms also affected how students completed projects and interacted with assignments, necessitating flexible assessment methods and additional support resources. | |
| 5. Evaluate issues of bias, culture, environment, ethics, regulations, and professional expectations in the field of artificial intelligence (AI) and machine learning. | Yes. The assessment highlighted the need to integrate ethics, bias, and professional standards more explicitly throughout the program. Long-term changes include the addition of case studies, interactive simulations, and guest lectures focused on AI ethics, regulatory compliance, and professional responsibilities. Rubrics were updated to assess students’ critical thinking, reflective analysis, and application of ethical principles in real-world scenarios. These changes were driven by observed variability in students’ ability to analyze bias and ethical issues in assignments, as well as the growing industry demand for professionals who understand responsible AI practices. | External factors include the rapid evolution of AI technologies and the increasing emphasis on ethical, culturally aware, and environmentally responsible AI practices in industry. The accessibility of AI tools online and varying levels of prior knowledge among students influenced engagement and performance in ethical analysis assignments. Additionally, regulatory updates and industry standards for responsible AI require continuous updates to course content. Remote learning modalities and technology access also impacted collaborative discussions and engagement with simulations, necessitating flexible assessment strategies and additional instructor support. |
Describe any change in student achievement observed as part of this assessment process, and what led to those changes.
External factors include rapid advancements in AI technologies, evolving industry standards, and increased emphasis on ethical and culturally aware AI practices. Students’ prior exposure to AI tools and online resources influenced engagement and performance. Additionally, remote learning and access to cloud-based AI platforms impacted participation in group discussions and simulations, necessitating flexible assessment strategies. Industry trends, such as conversational AI and prompt engineering, also shaped curriculum updates and assessment approaches to ensure students are prepared for current workforce demands.
Describe long-term changes in the program(s) that the assessment process led to, and what motivated those changes?
Yes. Students showed improvements in both technical and professional competencies. Completion rates for assignments in NLP, computer vision, and data science remained high, while the addition of collaborative and reflective exercises enhanced critical thinking and application of ethical principles. Students provided more nuanced analyses in discussions and case studies, demonstrating deeper understanding of bias, cultural considerations, and professional expectations. These changes were driven by interactive assignments, clear feedback, and structured reflection activities integrated into the modules.
What did you learn about the teaching and learning of "Practicing Community" in your programs?
Yes. The assessment highlighted the importance of integrating collaborative learning and ethical reflection throughout the certificate program. Long-term changes include adding more group assignments, peer review activities, and case studies to encourage discussion and knowledge-sharing. Rubrics were updated to assess engagement, critical thinking, and the application of AI concepts in community-based contexts. These changes were prompted by observations that while students were technically competent, structured opportunities for collaboration and reflection improved their understanding of societal, cultural, and ethical implications of AI.
Describe any external factors affecting the program or affecting assessment of the program.
Additional files supporting the assessment include:
Tables summarizing student performance across NLP, computer vision, and data science assignments.
Sample student submissions demonstrating practical application of AI solutions.
Rubrics used for evaluating programming proficiency, ethical reflection, and collaborative assignments.
Visualizations of participation rates in discussions and peer review exercise