Quality Assurance for blended and online courses

2019-2020 — Fullerton — Quality Assurance ePortfolio

CSUF Quality Matters Tuffy elephant mascot

Proposal Summary: For AY 19-20, we engaged faculty from a wide variety of disciplines and partnered with the Online Education and Training (OET) department to strengthen and expand our Quality Matters (QM) community. Despite the challenges of COVID 19, we co-hosted a QM training on our campus with the CSU Chancellor’s Office for 23 faculty, had several campus members complete peer review training, and enrolled two OET members in the June 2020 APPQM facilitator training so that CSUF will be able to host future face-to-face sessions.

Our campus continues to work toward addressing the four areas of campus need identified in 2014 for online teaching, assessment, and quality assurance: 

  1. Uneven level of expertise in online/blended pedagogy, design, and delivery
  2. Lack of understanding of faculty needs related to effective online/blended teaching
  3. Lack of coordination on strategies and choice of tools to ensure the quality of online and blended courses
  4. Lack of organization and community among departments to support quality online and blended teaching

Campus QA Goals

Campus Goals for Quality Assurance

  • Goal 1: Expand the awareness and implementation of the QM process on campus 
  • Goal 2: Deepen QA/QM expertise on campus
  • Goal 3: Support expansion of QM course certification
  • Goal 4: Disseminate project success and lessons learned
  • Goal 5: Expand research on the impact of faculty QA/QM expertise on student learning outcomes and experiences

Quality Assurance Leads

  • Shelli Wynants, Ed.D., Faculty QM Lead, QMC/Campus Contact
  • Sinjini Mitra, Ph.D., SQuAIR Lead

Supporting Campus Partners

  • Online Education and Training
  • Faculty Support Services (Faculty Development Center, Academic Technology Center, Faculty Affairs and Records)
  • Office of Assessment and Educational Effectiveness
  • Information Technology
  • CSUF Colleges/Departments

Campus Commitment Toward Sustainability of QA Efforts

  • Host workshops and events for faculty on online teaching pedagogy and effective use of technology tools
  • Disseminate information on Quality Matters to the campus to promote online quality assurance
  • Support faculty and campus team members to attend professional development opportunities locally and nationally

Summary of Previous QA  Accomplishments

QM Certified Peer Reviewers

  • Manal Alatrash, Ph.D. RN, (2019) Professor, College of Health and Human Development
  • Tara Barnhart, Ph.D. NBCT, (2017) Professor, College of Education
  • Shana Charles, Ph.D., (2017) Assistant Professor, College of Health and Human Development
  • Catherine Dinh, MBA, (2015) Instructional Designer, Online Education & Training
  • Susan Gaitan, MA, (2014) Instructional Designer, Online Education & Training
  • Barbara Glaeser, Ph.D., (2016) Professor, College of Education
  • Sudarshan Kurdwarkar, Ph.D., (2016) Associate Professor, College of Engineering and Computer Science
  • Rachel McClanahan, DNP, RN, (2015) Associate Professor, College of Health and Human Development
  • Sinjini Mitra, Ph.D., (2016) Associate Professor, Mihaylo College of Business and Economics
  • Ken Moyer, M.A., (2019) Instructional Designer, Online Education & Training
  • Marsha Orr, MSN, RN (2014) Lecturer, College of Health and Human Development
  • Suzanne Robertson, Ph.D. RN, (2014) Professor, College of Health and Human Development
  • Chris Street, Ph.D., (2018) Professor, College of Education
  • Diana Tyler, MSN, RN (2018) Professor, College of Health and Human Development
  • Shelli Wynants, Ed.D., (2016) Lecturer, College of Health and Human Development
  • Whitney Youngren, M.A., (2017) College of Humanities and Social Sciences

QM Certified Courses

List of 11 CSUF QM certified courses

Dissemination of QA Efforts— Conferences  

  • Shelli Wynants presented CSUF SQuAIR data as part of a panel presentation with other CSU QA leads: Measuring the Impact of QA Efforts on Teaching-Learning Success at the Cal State 2019 Tech Conference in San Diego on July 31, 2019
  • CSU SQuAIR research findings were also shared by Sinjini Mitra in another panel presentation, Impact of QM Professional Development and Course Certification on Teaching Performance and Student Success, at the Quality Matters West Regional Virtual Conference on April 3, 2020 


Development of Campus QA Resources

Creation of new materials to support faculty teaching remotely during COVID 19:

  • Keep Teaching
    Multiple resources were culminated and created for this webpage to support faculty in teaching remotely during a campus closure emergency
  • Shelli’s Blog on Virtual Teaching
    With the campus shift to teaching remotely, this blog was created to share some ideas, best practices, and tips for teaching online from Shelli Wynants, Director of Online Education and Training, and faculty member in Child and Adolescent Studies.

Continued updates and enhancements to helpful faculty resources:

  • Best Practices for Online Teaching Guide 
    The department of Online Education and Training (OET) created this guide to provide useful information about online instruction at CSUF to create opportunities for instructors to improve online teaching practices.

    Open Fullerton
    The goal of the OpenFullerton site, created by OET, is to empower faculty in supporting student success by providing them a variety of resources and professional development opportunities to help reduce textbook costs, and by supporting their efforts to explore, cultivate, create, and access high quality Open Educational Resources (OER).

Next Steps for QA Efforts 

By the end of AY20-21, CSUF hopes to achieve the following five deliverables:

  1. 30 faculty/staff completing the “Applying the QM Rubric” (APPQMR) workshop
  2. At least two CSUF-hosted APPQMR face-to-face workshops taught by CSUF faculty/staff certified as Facilitators
  3. 15 faculty creating a QM deliverable, reflecting application of at least two QM essential standards applied to one of their online/blended courses
  4. At least three OET webinars to support faculty application of QM standards in their online/blended courses
  5. A research report analyzing study results of QM-expertise on student learning outcomes and experiences

Quality Assurance Results

Training Completions

The following table summarizes all of the CSUF faculty and staff Quality Matters training completions that occurred during the 2019/20 academic year.

Training

Number of Completions

Applying the Quality Matters Rubric

37

Peer Reviewer Course

2
Introduction to Teaching Online Using QLT
1
Improving Your Online Course
1


Additionally, CSUF offers a Teaching Online Certificate Program (TOP) that addresses quality assurance best practices aligned with both the QLT and QM rubric. The following table summarizes all of the CSUF faculty and staff who completed TOP courses during the 2018/19 academic year.

Training

Number of Completions

TOP 1 - Introduction to Online Teaching

12

TOP 2 - Designing a Quality Online Course

11

TOP 3 - Engaging Students Online

25


Accessibility/UDL Efforts

  1. OET offers an online certificate program for Accessible Instructional Materials (AIM). Our AIM certificate program has 3 sequential modules; each 4-week module is designed to take 5 hours to complete and is a combination of instructional activities, collaborative discussions, and individual application.  In this program, faculty become more aware of the various disabilities of their students and the resultant issues, including legal and regulatory obligations, instructional needs, and campus resources. This program also includes an introduction to Universal Design for Learning and its application in creating accessible documents and captioning instructional videos.
  2. OET provides resources for faculty on accessibility and UDL on its website.
  3. CSUF purchased Ally for the LMS to help faculty.  

Student Quality Assurance Impact Research

Goal #1: Study the extent of Quality Assurance training at CSUF

Overall Total: 211 Faculty/Staff

Type of Quality Assurance TrainingNumber of Faculty/Staff
APPQMR (QM)161
QOLT13
Peer Reviewer Course (QM)15
Master Reviewer Course (QM)1
Other QM (Re-certification, Rubric, update, facilitator)7
QOLT Reviewer
1
Other (e.g. Accessibility training, IYOC, an Introduction to Online Delivery)13

  

Type of Course Review & CertificationNumber of Courses
QM - Certified Course11
QOLT - Reviewed Courses8
Number of QM Certified Peer Reviewers16

  

Goal #2: Study the impact of faculty QA trainings on student performance in online courses at CSUF

Student Performance Outcomes – Repeatable (DFW) grades 

  • Repeatable grades (DFW rates) for underrepresented (URM) students were significantly lower for online courses taught by faculty with some type of QA training in Fall 2018; no significant differences were found between courses taught by QM-trained faculty and non-QA-trained faculty in Fall 2019 or Spring 2019.
  • DFW rates for first generation students were significantly lower for online courses taught by faculty with some type of QA training in both Fall 2018 and Fall 2019.

Student Survey Data – Quantitative Feedback

  • Survey developed by CSU Chancellor’s Office
  • 25-item survey aligned with QM and QOLT standards, using a 6-point Likert scale (1: Strongly Disagree; 2: Disagree; 3: Somewhat Disagree; 4: Somewhat Agree; 5: Agree; 6: Strongly Agree)
  • Overall, very positive feedback from students regarding the course design
  • Limitation: Sample size is small (n = 66 responses)