2018-2019 — Fullerton — Quality Assurance ePortfolio

Proposal Summary: For AY 18-19, we engaged faculty from a wide variety of disciplines and partnered with the Online Education and Training (OET) department to strengthen and expand our Quality Matters (QM) community. We were able to complete the goals of this project through a combination of QM Workshops (including two Fullerton APPQMR cohort sessions) and campus-based lunch and learn events. Our campus continues to work toward addressing the four areas of campus need identified in 2014 for online teaching, assessment, and quality assurance: 

1. Uneven level of expertise in online/blended pedagogy, design, and delivery
2. Lack of understanding of faculty needs related to effective online/blended teaching
3. Lack of coordination on strategies and choice of tools to ensure the quality of online and blended courses
4. Lack of organization and community among departments to support quality online and blended teaching

Campus Goals for Quality Assurance

• Goal 1: Conduct a study to explore the impact of faculty Quality Matters (QM) expertise development on student learning experiences
• Goal 2: Expand the awareness and implementation of the QM process on campus
• Goal 3: Deepen QM experience for selected faculty 
• Goal 4: Support expansion of QM training and course certification
• Goal 5: Disseminate project success and lessons learned  

Quality Assurance Leads

• Marsha Orr, MSN, RN, Faculty QM Lead
• Sinjini Mitra, Ph.D., SQuAIR Lead
• Shelli Wynants, Ed.D., QMC/Campus Contact

Supporting Campus Partners

• Online Education and Training
• Faculty Support Services (Faculty Development Center, Academic Technology Center, Faculty Affairs and Records)
• Office of Assessment and Educational Effectiveness
• Information Technology
• CSUF Colleges/Departments

Campus Commitment Toward Sustainability of QA Efforts

• Host workshops and events for faculty on online teaching pedagogy and effective use of technology tools
• Disseminate information on Quality Matters to the campus to promote online quality assurance
• Support faculty and campus team members to attend professional development opportunities locally and nationally

Summary of Previous QA  Accomplishments

QM Certified Peer Reviewers

• Chris Street, Ph.D., (2018) Professor, College of Education
• Shana Charles, Ph.D., (2017) Assistant Professor, College of Health and Human Development
• Whitney Youngren, M.A., (2017) College of Humanities and Social Sciences
• Shelli Wynants, Ed.D., (2016) Lecturer, College of Health and Human Development
• Sinjini Mitra, Ph.D., (2016), Associate Professor, Mihaylo College of Business and Economics
• Sudarshan Kurdwarkar, Ph.D., (2016) Associate Professor, College of Engineering and Computer Science
• Barbara Glaeser, Ph.D., (2016) Professor, College of Education
• Rachel McClanahan, DNP, RN, (2015) Associate Professor, College of Health and Human Development
• Suzanne Robertson, Ph.D. RN, (2014) Professor, College of Health and Human Development
• Marsha Orr, MSN, RN (2014) Lecturer, College of Health and Human Development

QM Certified Courses

Dissemination of QA Efforts— Conferences

• Hosted a roundtable discussion for faculty at the Annual University Assessment Forum on May 2, 2019. Title: Ensure Quality of Online Instruction: Designing a new online course or want to improve your existing online course? Join this table to learn about Quality Matters (QM).
• OET offered two Lunch & Learn events for faculty who completed QM training in the 18-19 academic year; the purpose of the events was to support faculty in applying the QM rubric to their courses and inform them of other QM opportunities (e.g., peer reviewer course, course certification). SQuAIR research findings were also shared.  

Development of Campus QA Resources

Creation of two helpful faculty resources:

• Best Practices for Online Teaching Guide
  The department of Online Education and Training (OET) created this guide to provide useful information about online instruction at CSUF to create opportunities for instructors to improve online teaching practices.

  Open Fullerton
  The goal of the OpenFullerton site, created by OET, is to empower faculty in supporting student success by providing them a variety of resources and professional development opportunities to help reduce textbook costs, and by supporting their efforts to explore, cultivate, create, and access high quality Open Educational Resources (OER).

Next Steps for QA Efforts

By the end of AY19-20, CSUF hopes to achieve the following seven deliverables:

1. 40-50 faculty/staff completing the online “Applying the QM Rubric” (APPQMR) workshop
2. 10-12 faculty/staff completing the QM “Peer Reviewer Course” (PRC) online workshop
3. 15-20 faculty completing a QM self-assessment of the essential standards for one of their online/blended courses
4. 10-15 online courses being informally reviewed within the CSUF community
5. 1- 3 courses completing QM certification
6. 3-4 Lunch & Learn workshop events to support faculty application of QM standards in their online/blended courses
7. A research report analyzing study results of QM-expertise on student learning outcomes and experiences

Training Completions

The following table summarizes all of the CSUF faculty and staff Quality Matters training completions that occurred during the 2018/19 academic year.

Additionally, CSUF offers a Teaching Online Certificate Program (TOP) that addresses quality assurance best practices aligned with both the QLT and QM rubric. The following table summarizes all of the CSUF faculty and staff who completed TOP courses during the 2018/19 academic year.

Accessibility/UDL Efforts

1. OET offers an online certificate program for Accessible Instructional Materials (AIM). Our AIM certificate program has 3 sequential modules; each 4-week module is designed to take 5 hours to complete and is a combination of instructional activities, collaborative discussions, and individual application.  In this program, faculty become more aware of the various disabilities of their students and the resultant issues, including legal and regulatory obligations, instructional needs, and campus resources. This program also includes an introduction to Universal Design for Learning and its application in creating accessible documents and captioning instructional videos.
2. OET provides resources for faculty on accessibility and UDL on its website.
3. CSUF purchased Ally for the LMS to help faculty.  

Student Quality Assurance Impact Research

Goal #1: Study the extent of Quality Assurance training at CSUF

Total: 168 faculty/staff

Goal #2: Study the impact of faculty QA trainings on student performance in online courses at CSUF

Student Performance Outcomes – Repeatable (DFW) grades & GPA

• For Fall 2016 and Spring 2017, courses taught by QM-trained faculty had statistically significantly lower DFW rates than those courses which were taught by faculty with no QA-related training (p-values < 0.0001)
• Average course GPA was highest for QM-trained faculty taught courses  

• Repeatable grades (DFW rates) for underrepresented (UR) students were significantly lower for online courses taught by faculty with some type of QA training

Student Survey Data – Quantitative Feedback

• Survey developed by CSU Chancellor’s Office
• 25-item survey aligned with QM and QOLT standards, using a 6-point Likert scale (1: Strongly Disagree; 2: Disagree; 3: Somewhat Disagree; 4: Somewhat Agree; 5: Agree; 6: Strongly Agree)
• Overall, very positive feedback from students regarding the course design
• Limitation: Sample size is small (n = 27 responses)