Quality Assurance for blended and online courses

2018-2019 — Cal State LA — QA ePortfolio

Proposal Summary:  This year's project plan includes offering a Fully Online Development Program that required face-to-face meetings, instructional designer consultations, and the use of an LMS template. We anticipate that an additional 5 hybrid courses will be ready for QM-managed peer reviews. We would like to formalize our data collection and data analysis efforts by disseminating student mid-course evaluations in courses that were developed through our QA efforts.  Finally, we  requested professional development funds to attend InstructureCon as the campus transitioned from Moodle to Canvas.   

Campus QA Goals

Campus Goal for Quality Assurance

Over the last 3 years, Cal State LA’s Quality Assurance (QA) efforts have focused on increasing hybrid/blended course offerings. Recently, there has been an emerging interest in the creation of fully online courses. This year’s QA efforts seek to continue existing hybrid development efforts, as well as begin developing complementary fully online development efforts, all housed in the campus’ Center for Effective Teaching and Learning (CETL).

  • Offer an Online Development Program for teams of faculty who wish to develop a fully online course for the department
  • Submit an additional 5 courses for QM certification
  • Professional Development and Travel funds to InstructureCon as the campus migrates to the Canvas LMS

Quality Assurance Lead

  • Beverly Bondad-Brown, Ph.D. — Director of Academic Technology

Supporting Campus Partners

Center for Effective Teaching and Learning (CETL) staff:

  • Catherine Haras, Senior Director of CETL
  • Jeff Suarez-Grant, Senior Instructional Designer
  • Owynn Lancaster, Instructional Designer
  • Lauren Brady, Instructional Designer
  • Naat Jairam, Instructional Designer

Campus Commitment Toward Sustainability of QA Efforts

  • Course development stipends were funded by the Office of the Vice Provost, Planning and Budget
  • CETL instructional designers designed, facilitated, and consulted with faculty in all course development efforts

Summary of Previous QA  Accomplishments

In the 2017-2018 year we accomplished the following:

  • 23 faculty participated in CETL's Hybrid Development Program
  • 4 courses received QA certification
  • 30 QA training sessions completed
  • Developed a self-paced "Road to Hybrid" checklist as guidance for faculty and departments on hybrid course development

Dissemination of QA Efforts— Conferences 

The following conferences and presentations have been opportunities to share QA-related information and/or faculty development on QA-related topics, not necessarily to share our QA results:

Quality Assurance Results

Image caption

QA Training Completions
Quality Matters and QLT Training Completions:

Additional QA Team QM Course Completions

  • Beverly Bondad-Brown (QA lead) completed QM's Improving Your Online Course (IYOC)  Online Facilitator Certification
  • Lauren Brady (CETL Instructional Designer) completed QM's APPQMR and PRC course to become a QM peer reviewer
  • Naat Jairam  (CETL Instructional Designer) completed QM's APPQMR and IYOC
  • Owynn Lancaster  (CETL Instructional Designer) completed QM's APPQMR and DYOC

Additional QA-related trainings

  • 40 Cal State LA faculty completed an Association of Colleges and University Educators (ACUE) module on Establishing Powerful Learning Outcomes.
  • 17 Cal State LA faculty completed CETL's Grading Best Practices workshop
  • 15 Cal State LA faculty completed CETL's Transparent Assignment Design workshop

Quality Assurance Results (cont.)

Course Peer Review and Course Certifications

Although we planned to submit up to 5 course for QM review, it has been especially difficult to identify faculty willing to go for QM certification this past year.  Many faculty have preconceived notions of what "peer review" means.  Their context reminds them of the peer review process for academic conferences, and/or manuscript submission.  They believe they will receive scathing "Revise and Resubmit" comments back from the QM peer review process.  Faculty have no idea about the valuable, helpful, and collegial comments they would receive when going through QM peer review.  This is likely because we have never introduced QA programming by starting with the Applying the Quality Matters Rubric (APPQMR) workshop were they would learn about the QM review process.  Thus, as we move forward in our QA efforts, we will be more formally tying the QM rubric standards to our QA efforts, so that faculty understand that their courses are indeed good candidates for QM peer review.  In almost all cases, courses that have gone through CETL programming need minimal adjustments before the formal QM review process.  In the upcoming year, we will continue to encourage our faculty to submit their courses, and develop a stronger informal review so faculty can see their courses are great candidates for full QM peer review.

Student Quality Assurance Impact Research 

Participants in CETL's Hybrid Development Program in Spring 2018 disseminated a student survey in their redeisgned Fall 2018 courses.  Respondents were asked to rate their agreement with a range of items that related to the design and delivery of the course.  The survey was given in 31 course sections, taught by 13 instructors.  (Many instructors were teaching multiple sections of their hybrid course.)  Over 500 students responses were collected. The following graphs show how students rated the course design and delivery:

Development of Campus QA Resources

CETL "Going Online" Checklist

Accessibility/UDL Efforts

As part of our Fully Online Development Program, CETL instructional designers have designed an accessibilty workshop and informal review process.  This training session includes the following:

  • How to check YouTube videos on correct captioning
  • YouTube auto-generated vs. manually added captions
  • Checking Word docs for headings, color contrast, alt text
  • Use of lists and tables in Word
  • How to use the Word Accessibility Checker
  • An activity on checking the accessibiliyt of a PowerPoint presentation
  • Differences between OCR and non-OCR PDFs
In addition, CETL staff informally reviewed all courses created through CETL's Online Development Program.  Videos needing captioning were sent to AutoSync for remediation.

Next Steps for QA Efforts 

In the 2019-2020 academic year, we plan the following:

  • The development of a new Desiging Online Courses (DOC) certificate program housed in CETL.  This program will combine face-to-face workshops with self-paced online modules to walk faculty through desiging a quality fully online course.
  • Development of an informal QM review process to transition faculty toward formal QM course review
  • Help scale development of courses from hybrid to fully online by offering campus QM IYOC workshops facilitated in-house by our QA lead