2019-2020 — Cal State East Bay— Establishing a Culture of Quality Assurance

Purposal Summary: Cal State East Bay proposed to increase the number of faculty trained in QA and the number of QM certified courses. Implementation of this proposal involved one-on-one Instructional Designer-Faculty relationships and a concentrated effort to provide training. Assessment of this proposal included a qualitative study of faculty feedback in preparation/design, teaching, and students’ learning. Other indicators of student success such as affordability of course materials will also be tracked.

Campus QA Goals

Campus Goals for Quality Assurance

The main focus of our proposal included:

  • Increase the number of faculty who have completed QM training courses;
  • Increase the number of faculty who have completed the course re/design of their online or hybrid courses;
  • Continue to improve accessibility for online learners with disabilities;
  • Maintain support for professional development of faculty participating in the Quality Assurance programming.

With support from the Online Campus, CSUEB’s course design strategy utilizes Instructional Designers / eLearning Specialists by pairing them by program and faculty to provide the following services: offering OER resources, designing quality courses that meet QM standards, and incorporating the latest innovation and design pedagogy. The Development of these and other course resources will rely partially on QA funding and the rest will be covered by CSUEB matching funds.

We met our proposal goals the following ways:

  • Faculty have taken 39 QA trainings;
  • We offered 9 QM workshops as well as 36 webinars for faculty who are interested in increasing quality in their course;
  • 21 courses received QM certification which is more than the goal of 14 that we had in our proposal;
  • The SQUAIR faculty associate completed a qualitative analysis of how QA trainings have helped with their online course design and has benefited students and their teaching;
  • We offered 5 accessibility workshops for faculty to learn how to make their documents accessible for all learners.
  • For the first time we trained new faculty in the APPQMR as new part of New Faculty Orientation and will continue this in 20-21.

Quality Assurance Lead(s) 

  • Jessica Weiss, Director of Faculty Development

  • Roger Wen, Senior Director of the Online Campus

Supporting Campus Partners

  • Provost and Associate Provost, Academic Affairs

  • The Online Campus Instructional Designer, eLearning Specialists, and LMS Administrators/User Support Consultants

  • Office of Faculty Development/Faculty Center for Excellence in Teaching

  • Media and Technology Services

  • Library

Campus Commitment Toward Sustainability of QA Efforts

  • The Development of courses that offer OER resources, meet QM standards and incorporate latest innovation and design pedagogy was partially funded by QA funding and the rest was covered by CSUEB matching funds.
  • CSUEB matched funding to pay for the additional IYOC/APPQMR facilitation fees, APPQMR/IYOC trainings and other workshops in QA and design-related topics.
  • Workshop panels comprised of  recent and former grantees with certified courses (delete: will) disseminated information about the QA program and the opportunities for other faculty members as well as introducing more faculty to the QM essential elements by sharing about their courses.

Summary of Previous QA  Accomplishments

Since Fall 2015 East Bay has engaged in a number of efforts focused on QLT, QM, and Quality Assurance programming, training, and support that contributed to the success of our 2019-20 efforts. Prior Accomplishments Include:

  • Total 143 courses (21 certified in 19-20) certified as high quality courses by QM and QLT;
  • In Fall, the Online Campus received the 2019 Quality Matters National "Making a Difference for Students" Higher Education for our prior QA efforts  (see acceptance video below);
  • Assessment results: The combined 2016-2017 and 2017-2018 efforts are paying off. When we compared courses taught by QA-participating faculty between Summer 2014 and Summer 2017, the percentage of DWFs in online courses taught by faculty who participated in QA activities [9.60%] is significantly lower than that in courses taught by faculty who have not participated in QA activities [10.47%]. [F(1, 3261)=4.162, p=.041]. We anticipate that the courses taught by participating faculty in the remainder of 2017-2018 as well as 2018-2019 will show similar declines;
  • 2017-2018 Online and Hybrid Course Quality Transformation Grant Program (QA);
  • 2016-2017 Online and Hybrid Course Quality Transformation Grant Program (QA);
  • 2015-2016 Online and Hybrid Course Quality Transformation Grant Program (QA);
  • 2015-2016 Revising Your Online Courses for Semester Conversion Faculty Learning Community;
  • 2014-2015 Online and Hybrid Course Quality Transformation Grant Program (QA);
  • 2014-2015 Teaching with Technology Faculty Learning Community;
  • 2014: Summer Institute: Journey to Excellence in Online Education (QOLT introduced);
  • 2015-2016 QOLT Campus Nominee, Kevin Kaatz, Department of History;
  • 2015 College Coordinators for Quality Assurance Faculty Learning Community (QM-based);
  • 2014-2015 QOLT Awardee, Kevin Kaatz, Department of History;
  • The current SQUIAR Analysis is still in progress.

CSUEB 2019 Recipient of the Quality Matters National "Making a Difference for Students" Higher Education award, acceptance video.

Dissemination of QA Efforts— Conferences

List the conference(s) attended, who attended, and/or the link to the presentation — National, Regional, State or Local:

  • "CSUEB Data and Analysis on QM Impact to Students' Learning" at Quality Matters West Regional Conference, Dr. Erik Kong, Link to presentation;
  • Cal State 2019 Tech Conference, Dr. Roger Wen;
  • OLC Innovate Conference 2019, Monica Munoz, Link to presentation.

Quality Assurance Results

Training Completions

The following table summarizes all Cal State East Bay faculty and staff that have taken QA training during the 2019-2020 academic year.

Training
Number of Completions
Applying the Quality Matters Rubric
22
Applying the Quality Matters Rubric Face-to-Face and Online Facilitator1
Improving Your Online Course
10
Improving Your Online Course Online & Face-to-Face Facilitator
2
Peer Reviewer2
Master Reviewer & Recertification
2

Course Peer Review and Course Certifications

Each faculty member is paired with an eLearning Specialist/Instructional Designer and they work together to get the course ready for review. The eLearning Specialist/Instructional Designer conducts a self-review of all courses before they are submitted to QM.

At Cal State East Bay, there are currently 14 eligible QM Peer & Master Reviewers:

  1. Amy June Rowley
  2. Bill Hopkinson
  3. Chinglih Jan
  4. Georgie Ziff
  5. Isaiah Lankham
  6. James Mitchell
  7. James Thomas
  8. Keelin Jardin
  9. Kimberly Kim
  10. Michelle Rippy
  11. Michael Greathouse
  12. Andrew Wong (Master Reviewer)
  13. Kevin Kaatz (Master Reviewer)
  14. Roger Wen (Master Reviewer)

The table below shows the 21 Quality Matters Certified courses for 2019-2020 Academic Year.

Course ID
Course NameInstructor
MGMT460
Healthcare Operations (04/2020/QM)
Balaraman Rajan
HIST102
World History II: 16th Century C.E.-Present (02/2020/QM)
Benjamin Klein
ACCT433Federal Taxation for Partnerships (01/2020/QM)
John Tan
REC610
Strategic Leadership & Ethics (01/2020/QM)Heather Vilhauer
ENSC240
Environmental Biology (12/2019/QM)

Kelly Decker
ENSC241Environmental Biology Lab (12/2019/QM)Kelly Decker
ENGL200
College Writing II (10/2019/QM)
Sartaz Aziz
STAT101Statistics in Everyday Life (10/2019/QM)

Oylum Ispir
HSC445
HIV/AIDS in a Global Context (10/2019/QM)Nidhi Khosla
CMGT660Cost Accounting, Control and Reporting in Construtction (10/2019/QM)Reza Akhvian
PHIL312
Bioethics (09/2019/QM)
Anndretta Wilson
PHIL100Workshop in Critical Thinking (09/2019/QM)Craig Derksen
PHIL100Workshop in Critical Thinking (09/2019/QM)
Susi Ferrarello
OTL682
Design and Implement User Interfaces for Online Instruction (09/2019/QM)Bill Hopkinson
HDEV380Lifespan Human Development (09/2019/QM)
Lisa Handwerker
STAT697
Issues in Statistics (09/2019/QM)Isaiah Lankham
CMGT410
Engineering Graphics for Construction Management (09/2019/QM)
Fadi Castronovo
MGMT404
Staffing and Talent Management (08/2019/QM)
Rajan Selvarajan
ENGL200
College Writing II (08/2019/QM)
Sally Baxter
CRJ380Forensic Science (07/2019/QM)
Michelle Rippy
TED521
Content Literacy/English Language Learners (07/2019/QM)
James Mitchell


CSU East Bay's 2019 Back to the Bay - Faculty participating in the faculty panel for how they made their course accessible for students. 

Student Quality Assurance Impact Research

Dr. Kevin Kaatz from Cal State East Bay conducted the study and wrote a white paper for the 2018-2019 SQUAIR project. The research question (as approved by the IRB) was:

“How has our quality assurance training (either through QM, QOLT, or OLC) helped with your course (online, hybrid, and/or in-class) in the following areas ( Preparation/Design, Teaching, and Students’ Learning)? Please be specific as possible, provide examples, and/or provide students’ feedback.”

This question was sent out to 198 faculty that had taken some type of QA training. 26 faculty responded, giving us a response rate of 13%. 96% of the faculty stated that the training they received was very helpful/helpful. Only one person (4%) was not sure if it was useful. None of the faculty stated that it was unhelpful. The faculty comments were coded into a Positive category with 6 sub-categories. The 6 sub categories were course organization, alignment, accessibility, benefits to students, further use of rubrics and eLearning Specialists.

  • Course Organization: 11 faculty mentioned this and was the most popular response to the research question, “How did our QA training help with...your courses…”;
  • Alignment: 6 faculty mentioned this as being helpful in designing online courses;
  • Accessibility: 4 faculty mentioned this category. With the QM rubric now requiring accessibility to be met, faculty have to build this into their course;
  • Student Benefits: 4 faculty mentioned that students benefited from making courses more engaging and students recognized that the course was more organized than other online courses;
  • The Use of Other Rubrics: 2 faculty mentioned that they now use rubrics as part of their online course due to QA training;
  • eLearning Specialists: 2 faculty mentioned this category and one stated “It has been useful to work with the QM team and have another set of eyes to have my course met National QM standards”;
  • Eric Kong presented at the 2020 West Regional Conference.

Overall, faculty found the QA training helpful and a few commented that the QM rubric should be made available to students and that the QA training should be offered to those who teach in-class courses. These results were shared at the SQUAIR Focus Group 9/10/19, at the Cal State 2019 Tech Conference. 

Development of Campus QA Resources

List of Resources Developed:

  1. Blackboard Exemplary Course Examples
  2. Blackboard Orientation Course
  3. Acquired a copy of iSpring
  4. Additional SoftChalk licenses
  5. Additional GI2025 funding to support QA effort
  6. Regular faculty workshops on course design related topics

List of Resource Links available to faculty

Accessibility/UDL Efforts

  • Launched ALLY in all Blackboard courses;
  • Purchased ABBYY and Grackle docs to help assist in increasing the accessibility of document in Blackboard;
  • Offer Accessibility training workshops to campus community in an effort to establish an inclusive campus community.

Next Steps for QA Efforts

  • Host Virtual Summer Institute serving over 140 faculty members in three sections;
  • Host two sessions of IYOC;
  • Train new faculty Aug 7th on APPQMR;
  • Identify and certify 20 or more online or hybrid courses in 2020-2021;
  • Recruit and support between 25 - 45 faculty will complete APPQMR or IYOC;
  • Seek additional funding to elevate the progress.