Quality Assurance for blended and online courses

Enhance Academic Quality in Online Courses at CSU Bakersfield

Project Summary

The purpose of the Quality Assurance project is to continue offering support and assistance to our faculty in designing and delivering quality instruction to ensure student success. This ePortfolio includes activities throughout the academic school years included Summer Institutes for Online Teaching and supporting faculty to become Quality Matters certified.

Campus QA Goals

CSUB continues to progress in ensuring a high-quality support system for faculty in developing and teaching online and/or hybrid courses. In the spring of 2013, the CSUB Academic Senate approved the Online and Hybrid Instruction Policy. The policy requires all faculty to receive training prior to teaching online or hybrid courses. The Distributed Learning Committee appointed by President Mitchell has proposed a set of requirements to be met prior to teaching online/hybrid courses. These requirements include the demonstration of competencies in using Learning Management Systems through completing training, developing, and submitting a course for review by either an internal or external team. 

     The Summer Institute has been conducted by the Faculty Teaching and Learning Center (TLC) since the summer of 2011. Its goal aligns with the University’s overall strategic goals.

     Our goal is to expand our capacity to build high quality online programs. Central to this effort is to expand the scale of the campus quality assurance project.

Background on Quality Assurance Efforts

Our campus became a Quality Matters (QM) subscriber in May of 2012. Currently, four-hundred-twenty two faculty and staff have received QM Rubric training; sixteen courses have received QM certification; two-hundred-seventy nine courses were internally reviewed by the campus QM certified peer reviewers; eight faculty became QM peer reviewers, two faculty and the Director of Instructional Development became a QM course facilitator. Also, our campus currently houses two online QM Facilitators, two face-to-face QM facilitators and three QM Master reviewers.

Our campus is also actively engaged with the CSU system Quality Online Learning and Teaching (QLT) initiative. One course from our campus each year has been selected as the exemplary course by QLT. The Summer Institute has been a successful model for a campus wide training program for faculty. Participation of faculty has consistently increased each year with twenty-six faculty members completed the 2018 Summer Institute. All four of CSUB's academic schools and Community College faculty are well represented in the Institute. Feedback received from attendees has been overwhelmingly positive.

Next Steps for QA Efforts 2020-2021

  • Step 1 - Conduct 10th Year of Summer Institute for Online Teaching (AY 2020-2021) with the expectations on increasing attendance and/or Conduct a Quality Assurance Professional Development every semester.
  • Step 2 - Continue to conduct peer reviews
  • Step 3 - Continue to offer Quality Matters training to faculty members
  • Step 4 - Provide opportunities to share Quality Assurance experiences and efforts with other faculty members and other CSU campuses
  • Step 5 - Provide opportunities for faculty to share student learning outcome from Quality Assurance structured online courses

Faculty Participants Feedback

  • 100% Agreed or Strongly Agreed that the first week of the 2019 Summer Institute was informative
  • 95.84% Agreed or Strongly Agreed that the workshop facilitators were effective in their roles
  • 100% Agreed or Strongly Agreed that the staff provided needed and appropriate support
  • 95.83% Agreed or Strongly Agreed that the summer institute provided lots of examples of best practices for online course design
  • 91.66% Agreed or Strongly Agreed that the summer institute provided opportunity for participants to share their online course design experiences with other participants
  • 100% Agreed or Strongly Agreed that the summer institute provided the information participants needed to successfully design their online courses according to the Quality Matters standards
  • 95.83% Agreed or Strongly Agreed that participants feel they understand how applying the Quality Matters standards could improve their online courses
  • 83.33% Agreed or Strongly Agreed that the advanced information about the institute is accurate
  • 100% Agreed or Strongly Agreed that the setting for the institute was appropriate for the activities conducted
  • 86.96% felt that the length of the Summer Institute was appropriate given the content provided

Dr. Robert Carlisle & Allison Evans Presenting at Summer Institute 2018

Drs. Robert Carlisle & Allison Evans Presenting at SI 2018

Tasks/Resources Completed For 2019 Summer Institute for Online Teaching

  • 2019 Summer Institute for Online Teaching weeklong training
  • Courses were reviewed by Peer Reviewers during the month of August 2019.

Quality Assurance Lead(s)

Dr. Charlene Hu*, Director of Instructional Development is the lead for this project. She was responsible in planning, budgeting, and recruitment of participants.

Faculty associate, Dr. Robert Carlisle was also actively involved in the organization, presentation, and course reviews for the Institute. He has been a strong supporter of online instruction among the faculty.

  • Campus contact: Charlene Hu*, Director of Instructional Development.

Email: xhu@csub.edu
 Phone 661-654-2811

  • Faculty Associate: Robert Carlisle, Professor of English.

Email: rcarlisle@csub.edu
 Phone 661-654-2127

*Dr. Charlene Hu has taken another position on the CSU Bakersfield campus and will no longer be involved in the Quality Assurance Program or a Peer Reviewer (as of January 17, 2020).

Supporting Campus Partners

  • Provost: Vernon Harper, Ph.D.
  • CIO: Faust Gorham
  • Faculty Teaching & Learning Center Director: Rebecca Weller, Ph.D.
  • Faculty Coordinator of Online Instruction: Robert Carlisle, Ph.D.
  • Administrative Support Coordinator: Christina Shiery
  • Instructional Designer: Alex Slabey
  • Campus Peer Reviewers:
    • Charlene Hu, Ph. D.*
    • Bob Carlisle, Ph. D.
    • Allison Evans Ph. D.
    • Michael Ault, Ph. D.
    • Andreas Gebauer, Ph.D.
    • Yeunjoo Lee, Ph.D.
    • Teresa Fernandez-Ulloa, Ph.D.
    • Di Wu, Ph.D.
    • Yong (Nathan) Li, Ph.D.
    • Marselline Diller, Ph.D.

*Dr. Charlene Hu has taken another position on the CSU Bakersfield campus and will no longer be involved in the Quality Assurance Program or a Peer Reviewer (as of January 17, 2020).

Campus Commitment Toward Sustainability of QA Efforts

We are fortunate to have strong support from key administration, deans, and faculty governance. Provost office has been providing stipends for professional development for faculty participating in QA activities.

Summary of Previous QA Accomplishments

CSUB adopted QM in 2012. We are also actively participating in QLT effort. We have been focused on building our campus capacity by training faculty to become peer reviewers and have certified internal review team for course review.

Dissemination of QA Efforts — Conferences

Drs. Carlisle and Hu have attended various conferences throughout the years in Quality Assurance. Dr. Jianjun Wang has presented SQuAIR data for our campus and the Chancellor’s Office for a few years.

Online Instruction Research Sharing

  • Charlene Hu, PhD
    • Cultural Difference Research Summary CHu- PDF
      • cultural_difference_research_summary_chu_copy1.pdf
    • The SQuAIR pilot study
      • https://files.eric.ed.gov/fulltext/ED583873.pdf)
  • Robert Carlisle, PhD
    • “Assessing and Comparing Academic Gains of Students in Traditional and Online Classes,” with Jess F. Deegan at The Fourth Annual QM Conference on Quality Assurance in Online Learning, Tucson, Arizona, October 06, 2012. 
      • qm_conf_assessing_oct_2012_3.pptx

 “Controlling Instructional Variables to Increase Academic Achievement in Online Classes,” at The 19th Annual CSU Teaching and Learning Symposium, San Jose, CA,  October 22, 2016. 

  • qm_conf_oct_2012_3.pptx

Quality Assurance Results

2019/2020 QA Training Results

 Academic Year:

2019-20

 Faculty that have taken various QM Trainings

16

 Faculty that have taken APPQMR

13

 Faculty that completed Peer Reviewer training

2

 Faculty that completed Master & Reviewer Recertification

1

 Courses internally reviewed

30

 QM course recertification completed

2

2019/2020 Internal Course Reviews

 Academic Year:

2019-20

Overall

 School of Arts & Humanities

3

64

 School of Business, Public   and       Administration

8

39

 Library

0

7

 School of Natural Sciences, Mathematics, and Engineering

4

35

 School of Social Sciences and Education

15

120

 Other (Off Campuses)

0

4

 Total

30

269

Training Completions

  • For the reported period of July 1, 2019, through June 30, 2020, there are a total of thirteen faculty members who have completed the APPQMR training. Due to the Corona Virus (Covid-19) our office had to cancel our March 7, 2020 APPQMR training.
  • Twenty-five faculty completed the 2019 Summer Institute. Twenty-six faculty completed the 2018 Summer Institute. Twenty faculty completed the 2017 Summer Institute. Twenty faculty completed the 2016 Summer Institute. Twenty-nine faculty completed the 2015 Summer Institute week one.
  • Twelve faculty completed the 2018 Advanced Summer Institute. Thirteen faculty completed the 2017 Advanced Summer Institute. Due to personnel changes, the Advanced Summer Institute will not be offered in 2019.
  • Four hundred twenty-two faculty completed Applying the Quality Matters Rubric Workshop (77 Online/ 345 F2F) since May 2012.
  • Currently CSUB has two Online Quality Matters facilitators, two Face to Face QM Facilitators.
  • Sixteen courses are Quality Matters certified. In the past year one course was QM certified, and three courses were re-certified through QM.
  • Nine faculty and staff members have been certified as Peer Reviewers.

Quality Assurance Data:

 Academic Year:

13-14

14-15

15-16

16-17

17-18

18-19

19-20

Faculty that have taken various QM Trainings

59

154

235

315

358

420

422

Faculty that have taken APPQMR

49

139

176

240

292

347

355

Courses that have received QM certification

8

12

14

15

15

16

16

QM peer reviewers

4

11

11

11

10

8

9

QM Master reviewers

0

0

0

3

3

3

3

QM Facilitators

1

3

3

3

3

3

3

Courses internally reviewed

30

70

130

185

217

246

269

2017-2018 Faculty Learning Community - Online Academy

The Online Academy is a Faculty Learning Community (FLC) designed to guide participants in the development and design of an online/hybrid course. This yearlong learning community provided faculty with the knowledge and support necessary to build a complete course and become certified to teach online. Monthly hands-on workshops guided participants through the process of creating a course organized according to the Quality Matters Essential Standards. Upon completion of the academy seven faculty members completed the required trainings. Five participants have successfully completed the internal review and received their certification to teach online. Session topics included Applying the Quality Matters Rubric training, Getting Started with the Quality Matters Template, Objectives and Alignment, Assessments and Alignment, Online Assignments, and Grading and Rubrics.

Tasks/Resources Completed During 2018 Advanced Summer Institute for Online Teaching

  • 2018 Advanced Summer Institute for Online Teaching weeklong training
  • Twelve faculty completed the 2018 Advanced Summer Institute

Faculty Participants Feedback

  • 100% Agreed or Strongly Agreed that the first week of the 2018 Advanced Summer Institute was informative
  • 100% Agreed or Strongly Agreed that the facilitator was effective in his/her role
  • 100% Agreed or Strongly Agreed that the staff provided needed and appropriate support
  • 100% Agreed or Strongly Agreed that the Advanced Summer Institute provided lots of information and content that met the faculty’s expectations
  • 100% Agreed or Strongly Agreed that the Advanced Summer Institute provided lots of examples of best practices for online course design/teaching
  • 100% Agreed or Strongly Agreed that the Advanced Summer Institute provided opportunity for participants to share their online course design experiences with other participants
  • 100% Agreed or Strongly Agreed that the advanced information about the Advanced Summer Institute is accurate
  • 100% felt that the length of the Advanced Summer Institute was appropriate given the content provided
  • 100% of the faculty members would recommend the Advanced Summer Institute to others

Development of Campus QA Resources

We have developed many related webpages. Following are a few examples:

  • Online learning policy and requirement: https://www.csub.edu/tlc/online_instruction/index.html
  • Faculty showcase: https://www.csub.edu/tlc/programs_initiatives/quality_matters/faculty_showcase/index.html
  • Related Workshops: https://www.csub.edu/tlc/teaching_pedagogy/index.html

Student Quality Assurance Impact Research

  1. Built on collection and analyses of survey data from 103 students in online and hybrid courses taught by 10 instructors in a pilot study (Wang & Hu, 2017), further progress has been made in SQuAIR to expand the survey data collection from 256 students in Fall, 2018 and # students in Spring, 2019 to track the impact of QA on student learning experiences from online instruction;
  2. A new faculty perspective survey was developed and implemented in AY 2018-2019.  The first round of data gathering generated responses on 40 questions from 11 online instructors.  
  3. IRB approval was sought through multiple approaches, including an implementation meeting with the Institutional Research, Planning, and Assessment (IRPA) office. As a result, IRPA agreed to handle part of the local data collection for SQuAIR and the IRB support was granted to SQuAIR research.  The protocol establishment represents a new institutional progress beyond the pilot project.

It seemed important to emphasize the unique situation of CSUB (80% faculty received QA).  The report may briefly describe the data SQuAIR required across the system and put more attention to adjust the comparison between face-to-face and online platforms.

Accessibility/UDL Efforts

We have been using the QM standard 8 to support course material accessibility in online/hybrid courses. Faculty are encouraged to use a syllabus that is ADA compliant and make sure that their PDF, PPTs and video/audio files are accessible to diverse student population. We provided various workshops to ensure faculty are aware of the challenges in this area.

We are in the process of looking into Ally and hope to adopt it in the near future.

*Dr. Charlene Hu has taken another position on the CSU Bakersfield campus and will no longer be involved in the Quality Assurance Program or a Peer Reviewer (as of January 17, 2020.