banner

Sacramento State Quality Assurance

The Quality Assurance program (QA) at Sacramento State is supported through the Center for Teaching and Learning (CTL) and in partnership with Information Resources & Technology (IRT). The CTL provides professional development in the form of a Learning Community program, a Summer Teaching Institute, New Faculty Orientation, and minigrant programs that fund pedagogy enhancement, learning analytics, assessment, and technology projects. Quality Matters™ (QM™) essential standards such as Accessibility and Alignment are intentionally integrated into each of these programs. 

Program Goals:

  • Increase the number of faculty who are QA trained as practitioners and mentors.
  • Use action research and data analytics to measure student learning, improve student success, and contribute to the Scholarship of Teaching and Learning (SoTL) in online learning environments.
  • Establish an institutional support structure for the development, redesign, review, and QM certification of online and hybrid courses.


 Video: "Supporting Active Online Learning at Sacramento State"
[Video created by Joseph Crenshaw, graduate student in the Master's in Educational Technology (iMET) program]



Student-Centered Activities to Meet Goals

In addition to the funding from the Chancellor's Office, we received additional funding from Academic Affairs to develop and implement our QA activities. We emphasized student success as the focus in our QA efforts and partnered with IRT to achieve our goals. 

QA TrainingsApproximately 100 faculty completed QM or QLT courses in 2019-2020 AY; significantly increasing our overall rate of QA trained faculty.

Screenshot from CSU QA website showing QA training completions by campus - updated May 27, 2020. Sacramento State had a total of 341 training completions.

Faculty Mentoring: Thirty faculty participated in intensive Faculty Learning Communities (FLCs) for hybrid and online teaching (fall and spring) and received individualized support from QA mentors. The FLCs had a positive impact on faculty's understanding of QM standards and their ability to implement these standards in their courses.

Fall 2019 FLC participants and mentors

Results from a post-FLC survey (Fall 2019): 
After completing the online/hybrid teaching FLC,

  • 79% reported that they would "very likely" implement at least one QM standard in their course.
  • 86% reported that they would "very likely" implement one or more UDL and accessibility practices for their course.
  • 79% reported that they would "very likely" implement cognitive, social, or teaching presence in their course.
  • 50% reported that they are "very comfortable" with developing measurable objectives for their course.
  • 64% reported that they would "very likely" complete a course map of alignment for their course.

Sample comments from faculty participants:

"For me, I appreciated the depth and the breadth of the content. The material was delivered in digestible dosage(s) with time provided during face to face meetings to apply the information to aspects of my course. I appreciated the emphasis on not biting off more than I could chew so that I could incorporate the principles of best teaching practice (ie, QM standards) for the online model I had envisioned to meet my course needs."

"All the FLC objectives were helpful. I did not know any of them until taking this course. Thank you for opportunity to grow my teaching skills."

"I found the environment of gaining experience with modules and course map alignment with learning objectives extremely helpful. I simply could not go far enough in learning what was necessary to overhaul my course in the time allotted, so that is no fault of the course. I thought it was very supportive to create an environment where people had to respond to each other."

FLC poster by Brian Moore (Fall 2019 participant)

Course Reviews:  Internal and formal course review processes were formalized and made available through the QA webpage. Professional development funding was awarded to faculty who had successful reviews.

Marketing flyer for internal and formal course reviews 

Over 20 faculty participated in QM self-reviews in the 19-20 AY. We significantly increased the number of QM certified courses from 2 to 5 (250% increase) and have 5 additional courses in the pipeline for QM certification. 
As a result of the FLCs and peer mentorship, we expect an increase in courses being ready for QM certification; the increase in peer and master reviewers will help us meet this demand.


QM Certified Courses:

COMS 106: Digital Media CreationDr. Diego Bonilla
NURS 170: Foundations in Evidence-Based Nursing Practice
Dr. Michelle Dang
NURS 173: Theoretical Foundations for Leadership and Management
Dr. Tara Sharpp
SWRK 126: Theories of Criminal Behavior (6 weeks version)
Dr. Santos Torres, Jr.
SWRK 126: Theories of Criminal Behavior (16 weeks version)Dr. Santos Torres, Jr.


Accessibility and UDL 

We incorporated Accessibility and UDL content in our FLCs and Summer Institute and provided templates of accessibility language and support services for faculty to use in their syllabi. We also developed a process with IRT to have courses go through a comprehensive accessibility check as part of our internal review process. 

One of our QA team members is an accessibility expert who provides direct support to faculty and consultation for every course undergoing review. 

Sample FLC Canvas page with UDL and Accessibility learning objectives 

QM Canvas Template

We created a Canvas Course Template that incorporated QM Standards and made it available through Canvas Commons. We are using the template as part of faculty trainings to help faculty (re)design their courses using best practices. The template was also offered as a resource to over 700 faculty in our summer professional development program, June-July 2020. As of June 24, 2020, the template has been downloaded over 250 times.

Template image as seen in Commons

QA Leadership & Team 
Go Hornets!

(left to right)
Michelle Dang, faculty co-lead and mentor
Mark Rodriguez, faculty co-lead and mentor
Corinne Rowland, mentor and accessibility & instructional technology consultant
Debra Welkley, faculty mentor
Bronwyn Fields, faculty mentor
Deborah George, mentor and instructional designer
Tara Sharpp, faculty mentor

CTL Leadership

Lynn Tashiro, Director

"Novice to Expert" Approach



We took a 3-tiered approach to meet the needs of faculty at various levels of readiness and to increase institutional experts for online teaching. We continue to recruit faculty to complete QA trainings while offering more advanced trainings and FLCs for faculty who want to enhance their skills even further and serve as mentors. 

For 2019-2020, we have increased the number of faculty & staff certified as QM peer reviewers, master reviewers, and facilitators. We currently have 10 master reviewers (20% increase), 13 peer reviewers (15% increase), and 5 certified QM facilitators (40% increase). 

List of faculty and staff with QM certifications, updated May 2020

CourseMatch

As part of our efforts to support the "Finish in Four" initiative at Sacramento State, we have created opportunities for faculty to create online courses, particularly GE and bottleneck courses, that are eligible for CourseMatch and the CSU Fully Online program. In partnership with the College of Continuing Education, we have provided intensive support to 23 faculty over the past two years to redesign their courses to be fully online. In addition, we have provided support to many other faculty through our FLCs, highlighting standards that would make their courses eligible for CourseMatch. We are pleased to see a significant increase in courses selected for CourseMatch over the past several semesters. 

SemesterNumber of courses accepted for CourseMatch
Fall 20186
Spring 20197
Fall 201913
Spring 202015
Fall 202023


Support for Remote Teaching

During the transition to remote teaching Spring-Summer 2020, QA leadership and mentors provided support in the following ways:

  • Zoom tutoring sessions for faculty
  • One-on-one faculty support
  • Consultation with IRT on faculty needs
  • Recruitment of faculty for QA trainings 
  • Support in the development of the summer curriculum for online/remote teaching workshops
  • Serve as facilitators for summer online/remote teaching workshops
  • Serve as facilitator for QM courses through the Chancellor's Office
  • Assisted in the development of a campus-wide study about the transition response to remote teaching

Research

SQuAIR

Our analysis for the CSU Student Quality Assurance Impact Research (SQuAIR) has been an extensive undertaking with a review of over 64,000 data points (PI: Farshid Zabihian). To help guide our analysis, we asked the following questions:

  • Does QA training have an impact on student success?
  • Does QA formal course certification improve student success? 
  • Do students perceive certified courses differently than noncertified?
  • Does success differ by student group?
  • Does class size affect student success?
  • Does course length affect DFW?

The original dataset contained almost 71,000 data points. Once we eliminated data with missing information such as grades, the number of data points was reduced to 64,450. When we compared failing rates of students in courses before and after QA trainings, we found a very slight increase in the failing rate of students in the after group, which was not statistically significant. In terms of student groups and the difference between before and after QA trainings, we had mixed results with some groups having higher failure rates in courses where faculty had QA trainings and some groups had lower failure rates. 
There are many confounding variables that need to be considered, and we plan to consider these variables in our future analyses. First, we did not conduct same group analysis. The students in the "before" group were not the same students for the "after" group. There may have been other contextual factors that influenced our findings such as possible changes in the course design and delivery, more than one instructor teaching the same course, courses being offered in a different modality, and a change in the LMS (from Blackboard to Canvas). 

We plan to address our other research questions in the future using different methodologies (e.g., qualitative methods) and adding other outcome variables (e.g., student satisfaction) to assess QA impact on student success. We believe that the analysis completed thus far has provided us with important descriptive information that will allow us to dig deeper in future studies and understand how our QA efforts have made impact.

SQuAIR Report 1
SQuAIR Report 2

Faculty Surveys

In a recent campus-wide survey related to COVID-19 (May 2020), faculty who completed QA trainings reported that they felt more prepared to teach remotely, and the vast majority of faculty surveyed (whether they completed QA trainings or not) plan to complete professional development for online teaching in the future, suggesting an increased need for trainings and QA support (co-PIs: Sharyn Gardner & Tara Sharpp).

Responses about preparedness to teach remotely (n = 95). The majority surveyed felt QM and QLT courses were moderately, very, or extremely effective in preparing them to teach remotely.

Responses about likelihood of future professional development activities for online teaching (n = 383). The majority surveyed were "extremely likely" to complete professional development activities related to online teaching in the future.
Our QA program permitted graduate students in the iMET program to conduct a mixed methods study about faculty perceptions of professional development related to online and hybrid teaching and learning (n = 45). Results revealed several key recommendations as listed below (co-PIs: Jennifer Hobbs & Joseph Crenshaw):

  • Create stand-alone or asynchronous professional learning opportunities
  • Provide an online model for teaching and exemplar online courses
  • Develop and posting a database of best practices
  • Create and implementing targeted marketing
  • Develop a comparison of QM standards and other accreditation standards
  • Involve department leaders in QM
  • Create a means of dispelling misconceptions about QM
  • Conduct research that explores how quality assurance practices affect the conversion of in person courses to fully online courses

Future funding will help us continue our efforts in examining the effects of QA trainings on student outcomes such as GPA, using campus-wide data, and conducting more focused research on faculty perceptions related to QA trainings and support and student satisfaction. 

Dissemination  

We were selected as podium presenters at the annual QM Western Regional Conference that occurred in April 2020. The conference was changed to a virtual setting due to COVID-19. 

Conference Abstract:

From Novice to Expert: Strategies for Faculty Support and Collaboration*

An online educational goal of the CSU system is to increase online access to coursework. As a result, there has been a significant increase in the number of online courses offered at CSU campuses. This presentation describes the quality assurance (QA) efforts at Sacramento State to establish systematic support of faculty and staff in designing and delivering high quality online and hybrid courses and developing infrastructures to sustain and institutionalize support. Sacramento State's QA program is supported through The Center for Teaching and Learning & Information Resources & Technology in a collaborative effort.

The overarching goals of the QA program are to: 1. Increase the number of faculty who are QA trained as practitioners and mentors; 2. Establish an institutional support structure for the development, redesign, review, and QM certification of online and hybrid courses; and 3. Use action research and data analytics to measure student learning, improve student success, and contribute to the Scholarship of Teaching and Learning in online learning environments.

Using a logic model and QM training completion data, the interprofessional QA team at Sacramento State determined critical inputs and outputs to achieve the aforementioned goals. A 3-tiered approach was implemented to ensure that the needs of novice, moderately-experienced, and highly-experienced faculty are addressed, including addressing accessibility requirements.

Conference Presenters: Michelle Dang, Mark Rodriguez, Corinne Rowland, and Deborah George

Title Slide of QM Regional Conference West 2020
*The phrase "From Novice to Expert" was borrowed from Patricia Benner's "From Novice to Expert" nursing theory.