Quality Assurance for blended and online courses

Enhance Academic Quality in Online Courses at CSU Bakersfield

Project Summary:  The purpose of the Quality Assurance project is to continue offering support and assistance to our faculty in designing and delivering quality instruction to ensure student success. This ePortfolio includes activities throughout the academic school years included Summer Institutes for Online Teaching, and supporting faculty to become Quality Matters certified. 

Campus QA Goals

     CSUB continues to progress in ensuring a high-quality support system for faculty in developing and teaching online and/or hybrid courses. In the spring of 2013 the CSUB Academic Senate approved the Online and Hybrid Instruction Policy. The policy requires all faculty to receive training prior to teaching online or hybrid courses. The Distributed Learning Committee appointed by President Mitchell has proposed a set of requirements to be met prior to teaching online/hybrid courses. These requirements include the demonstration of competencies in using Learning Management Systems through completing training, developing and submitting a course for review by either an internal or external team. 

     The Summer Institute has been conducted by the Faculty Teaching and Learning Center (TLC) since the summer of 2011. Its goal aligns with the University’s overall strategic goals.

     Our goal is to expand our capacity to build high quality online programs. Central to this effort is to expand the scale of the campus quality assurance project.

Background on Quality Assurance Efforts

Our campus became a Quality Matters (QM) subscriber in May of 2012. Currently, three-hundred-fourty-seven faculty and staff have received QM Rubric training; sixteen courses have received QM certification; two-hundred-fourty-seven courses were internally reviewed by the campus QM certified peer reviewers; eight faculty and staff became QM peer reviewers, two faculty and the Director of Instructional Development became a QM course facilitator. Also, our campus currently houses three online QM Facilitators, two face-to-face QM facilitators and three QM Master reviewers.

Our campus is also actively engaged with the CSU system Quality Online Learning and Teaching (QLT) initiative. One course from our campus each year has been selected as the exemplary course by QLT. The Summer Institute has been a successful model for a campus wide training program for faculty. Participation of faculty has consistently increased each year with twenty-six faculty members completed the 2018 Summer Institute. All four of CSUB's academic schools and Community College faculty are well represented in the Institute. Feedback received from attendees has been overwhelmingly positive.

Next Steps for QA Efforts 2019-2020

  • Step 1 - Conduct 9th Year of Summer Institute for Online Teaching (AY 2019-2020) with the expectations on increasing attendance

  • Step 2 - Continue to conduct peer reviews

  • Step 3 - Continue to offer Quality Matters training to faculty members

  • Step 4 - Provide opportunities to share Quality Assurance experiences and efforts with other faculty members and other CSU campuses

  • Step 5 - Provide opportunities for faculty to share student learning outcome from Quality Assurance structured online courses

Faculty Participants Feedback

  • 100% Agreed or Strongly Agreed that the first week of the 2018 Summer Institute was informative
  • 93.55% Agreed or Strongly Agreed that the workshop facilitators were effective in their roles
  • 96.78% Agreed or Strongly Agreed that the staff provided needed and appropriate support
  • 96.77% Agreed or Strongly Agreed that the summer institute provided lots of examples of best practices for online course design
  • 93.54% Agreed or Strongly Agreed that the summer institute provided opportunity for participants to share their online course design experiences with other participants
  • 90.32% Agreed or Strongly Agreed that the summer institute provided the information participants needed to successfully design their online courses according to the Quality Matters standards
  • 100% Agreed or Strongly Agreed that participants feel they understand how applying the Quality Matters standards could improve their online courses
  • 96.67% Agreed or Strongly Agreed that the advanced information about the institute is accurate
  • 93.55% Agreed or Strongly Agreed that the setting for the institute was appropriate for the activities conducted
  • 87.09% felt that the length of the Summer Institute was appropriate given the content provided

Dr. Robert Carlisle & Allison Evans Presenting at Summer Institute 2018Drs. Robert Carlisle & Allison Evans Presenting at SI 2018

Tasks/Resources Completed For 2018 Summer Summer Institute for Online Teaching

  • 2018 Summer Institute for Online Teaching week long training

  • Courses were reviewed during the month of August 2018.

Quality Assurance Lead(s)

Dr. Charlene Hu, Director of Instructional Development is the lead for this project. She is responsible in planning, budgeting and recruitment of participants.

Faculty associate, Dr. Robert Carlisle is also actively involved in the organization, presentation and course review for the Institute. He has been a strong supporter of online instruction among the faculty.

  • Campus contact: Charlene Hu, Director of Instructional Development.

    Email: xhu@csub.edu
    Phone 661-654-2811

  • Faculty Associate: Robert Carlisle, Professor of English.

    Email: rcarlisle@csub.edu
    Phone 661-654-2127

Supporting Campus Partners

  • Interm Provost: Vernon Harper, Ph.D.
  • CIO: Faust Gorham
  • Director: Charlene Hu, Ph.D.
  • Faculty Coordinator of Online Instruction: Robert Carlisle, Ph.D.
  • Administrative Support Coordinator: Christina Shiery
  • Instructional Designers: Anthony Bolaños, Alex Slabey
  • Campus Peer Reviewers:
    • Charlene Hu, Ph. D.
    • Bob Carlisle, Ph. D.
    • Allison Evans Ph. D.
    • Michael Ault, Ph. D.
    • Andreas Gabuer, Ph.D.
    • Yeunjoo Lee, Ph.D.
    • Teresa Fernandez-Ulloa, Ph.D.

Campus Commitment Toward Sustainability of QA Efforts

We are fortunate to have strong support from key admininstration, deans, and faculty governance. Provost office has been providing stipends for professional development for faculty participating in QA activities. 

Summary of Previous QA Accomplishments
CSUB adopted QM in 2012. We are also actively participating in QLT effort. We have been focused on building our campus capacity by training faculty to become peer reviewers and have certified internal review team for course review. 

Dissemination of QA Efforts — Conferences 
Drs. Carlisle and Hu have attended various conferences throughout the years in Quality Assurance.

Online Instruction Research Sharing 

 “Controlling Instructional Variables to Increase Academic Achievement in Online Classes,” at The 19th Annual CSU Teaching and Learning Symposium, San Jose, CA,  October 22, 2016.

Quality Assurance Results

2018/2019 QA Training Results

 Academic Year:2018-19
 Faculty that have taken various QM Trainings56
 Faculty that have taken APPQMR51
 Faculty that have completed APPQMR Facilitator  Recertification
2
 Faculty that completed Master & Reviewer  Recertification3
 Faculty that completed QLT Training1
 Courses internally reviewed28
 QM course recertification completed
4
 QM course recertification pending
2
 Subscriber QM course review
1

2018/2019 Internal Course Reviews

 Academic Year:2018-19
Overall
 School of Arts & Humanities761
 School of Business, Public and  Administration8
32
 Library
0
6
 School of Natural Sciences,  Mathematics, and Engineering
1
29
 School of Social Sciences and  Education
11
116
 Other (Off Campuses)
1
2
 Total28
246

Training Completions

  • For the reported period of July 1, 2018, through June 30, 2019, there are a total of fifty-one faculty members who have completed the APPQMR training.
  • Thirty-one faculty attended the 2019 Summer Institute. Twenty-six faculty completed the 2018 Summer Institute. Twenty faculty completed the 2017 Summer Institute. Twenty faculty completed the 2016 Summer Institute. Twenty-nine faculty completed the 2015 Summer Institute week one.
  • Twelve faculty completed the 2018 Advanced Summer Institute. Thirteen faculty completed the 2017 Advanced Summer Institute. Due to personnel changes, the Advanced Summer Institute will not be offered in 2019.
  • Three-hundred-fourty-seven faculty completed Applying the Quality Matters Rubric Workshop (77 Online/ 268 F2F) since May 2012.
  • Currently CSUB has three Online Quality Matters facilitators, two Face to Face QM Faclitators.
  • Sixteen courses are Quality Matters certified. In the past year one course was QM certified, and three courses were re-certified through QM.
  • Eight faculty and staff members have been certified as Peer Reviewers.

Summer 2019 QA Data:

 Academic Year:2013-14
2014-152015-162016-172017-182018-19
Faculty that have taken various QM Trainings59154235315358420
Faculty that have taken APPQMR49139176240292347
Courses that have received QM certification81214151516
QM peer reviewers4111111108
QM Master reviewers000333
QM Facilitators133333
Courses internally reviewed3070130185217246

*All data pulled from QM website

2017-2018 Faculty Learning Community -  Online Academy

The Online Academy is a Faculty Learning Community (FLC) designed to guide participants in the development and design of an online/hybrid course. This yearlong learning community provided faculty with the knowledge and support necessary to build a complete course and become certified to teach online. Monthly hands-on workshops guided participants through the process of creating a course organized according to the Quality Matters Essential Standards. Upon completion of the academy seven faculty members completed the required trainings. Five participants have successfully completed the internal review and received their certification to teach online. Session topics included Applying the Quality Matters Rubric training, Getting Started with the Quality Matters Template, Objectives and Alignment, Assessments and Alignment, Online Assignments, and Grading and Rubrics.

Tasks/Resources Completed During 2018 Advanced Summer Institute for Online Teaching

  • 2018 Advanced Summer Institute for Online Teaching week long training

  • Twelve faculty completed the 2018 Advanced Summer Institute

Faculty Participants Feedback

  • 100% Agreed or Strongly Agreed that the first week of the 2018 Advanced Summer Institute was informative

  • 100% Agreed or Strongly Agreed that the facilitator was effective in his/her role

  • 100% Agreed or Strongly Agreed that the staff provided needed and appropriate support

  • 100% Agreed or Strongly Agreed that the Advanced Summer Institute provided lots of information and content that met the faculty’s expectations

  • 100% Agreed or Strongly Agreed that the Advanced Summer Institute provided lots of examples of best practices for online course design/teaching

  • 100% Agreed or Strongly Agreed that the Advanced Summer Institute provided opportunity for participants to share their online course design experiences with other participants

  • 100% Agreed or Strongly Agreed that the advanced information about the Advanced Summer Institute is accurate

  • 100% felt that the length of the Advanced Summer Institute was appropriate given the content provided

  • 100% of the faculty members would recommend the Advanced Summer Institute to others

Development of Campus QA Resources

We have developed many related webpages. Following are a few examples:

Student Quality Assurance Impact Research 

  1. Built on collection and analyses of survey data from 103 students in online and hybrid courses taught by 10 instructors in a pilot study (Wang & Hu, 2017), further progress has been made in SQuAIR to expand the survey data collection from 256 students in Fall, 2018 and # students in Spring, 2019 to track the impact of QA on student learning experiences from online instruction;
  2. A new faculty perspective survey was developed and implemented in AY 2018-2019.  The first round of data gathering generated responses on 40 questions from 11 online instructors.  
  3. IRB approval was sought through multiple approaches, including an implementation meeting with the Institutional Research, Planning, and Assessment (IRPA) office. As a result, IRPA agreed to handle part of the local data collection for SQuAIR and the IRB support was granted to SQuAIR research.  The protocol establishment represents a new institutional progress beyond the pilot project.

It seemed important to emphasize the unique situation of CSUB (80% faculty received QA).  The report may briefly describe the data SQuAIR required across the system and put more attention to adjust the comparison between face-to-face and online platforms.

Accessibility/UDL Efforts

We have been using the QM standard 8 to support course material accessibility in online/hybrid courses. Faculty are encouraged to use a syllabus that is ADA compliant and make sure that their PDF, PPTs and video/audio files are accessible to diverse student population. We provided various workshops to ensure faculty are aware of the challenges in this area.

We are in the process of looking into Ally and hope to adopt it in the near future.