Sacramento State Quality Assurance

The Quality Assurance program (QA) at Sacramento State is supported through the Center for Teaching and Learning (CTL) and in partnership with Information Resources & Technology (IRT). The CTL provides professional development in the form of a Learning Community program, a Summer Teaching Institute, New Faculty Orientation, and minigrant programs that fund pedagogy enhancement, learning analytics, assessment, and technology projects. Quality Matters™ (QM™) essential standards such as Accessibility and Alignment are intentionally integrated into each of these programs. 

Program Goals:

• Increase the number of faculty who are QA trained as practitioners and mentors.
• Use action research and data analytics to measure student learning, improve student success, and contribute to the Scholarship of Teaching and Learning (SoTL) in online learning environments.
• Establish an institutional support structure for the development, redesign, review, and QM certification of online and hybrid courses.

 Video: "Supporting Active Online Learning at Sacramento State"
[Video created by Joseph Crenshaw, graduate student in the Master's in Educational Technology (iMET) program]

We emphasize student success as the focus in our QA efforts. 

QA Training Data:  

• 66 faculty completed QM or QLT courses in 2020-2021 AY. As of June 18, 2021, our overall completion rate of QA trained faculty was 392, an increase of 15% from the previous academic year. 

Faculty Learning Communities (FLC) Data: 

Results from a post-FLC survey (Spring 2021)
After completing the Improving Your Online Course (IYOC) FLC,

• 100% reported that their knowledge of course design best practices was somewhat or very improved as a result of the FLC.
• 100% would "very likely" implement at least one QM standard in their course.
• 75% reported that they "very comfortable" with developing measurable objectives.
• 50% reported that they would "very likely" create a map of alignment for their course.
• 88% reported that they would "very likely" implement active learning in their course.
• 50% reported that they would "very likely" implement one more UDL and accessibility practices in their course.

Sample reflection statements from faculty participants:

"Of the improvements I made, I think that making all of the readings accessible will have the biggest impact on my students. Currently my course contains many readings that are older and were uploaded as images of the journal article, which makes them inaccessible to some and more onerous for others. By making the documents accessible for screen readers, it will benefit all of my students, who will be able to search for key terms in the articles and thus make reading the articles easier."

"The main (and biggest) improvement I am planning to make for my next course offering is to create assignments that more closely fulfill my course learning objectives."

"Of all the improvements I plan to make, the most impactful one would be to create measurable module level objectives. It will help me self-check if every module I deliver to students are in line with the overarching course objectives."

"...this workshop helped highlight the importance of rubrics for all assignments, not just exams or larger assignments. This really helps the student know exactly what to do to get the grade they want each and every time."

"As with any course that we are teaching, there is always room for improvement. This IYOC course has made visible areas where growth can occur to improve the course for students as well as Instructor. Through my experience in this IYOC course, I’ve committed to an improvement plan focusing on course alignment."

SQuAIR

In Spring 2021, we conducted a survey of students who were enrolled in courses from faculty who had participated in our FLCs for online teaching.

Survey questions addressed QM essential standards. For example:

• The instructor provided clear and detailed instructions for how to begin accessing all course components, such as syllabus, course calendar, and assignments. (QM SRS 1.1)

  The course description included the purpose and format (e.g. fully online, blended; schedule/calendar with specific dates/times) of the course, as well as any applicable prerequisite knowledge (e.g., prerequisite course). (QM SRS 1.2)

  I understood how the learning activities (including the assignments and ungraded activities) helped me achieve the learning objectives each week. For example, I understood how a discussion forum could help me prepare to develop a paper on a topic. (QM SRS 5.1)

Participant characteristics: 78% female; 19% male; 22% Asian; 8% African-American; 32% Latinx; 32% White; almost 100% were undergraduates.

Results (N = 78)

The majority of participants strongly agreed or agreed with statements regarding their courses meeting QM essential standards 1.1, 1.2, 2.1, 2.4, 3.1, 3.2, 3.3, 4.1, 4.2, 4.3, 7.1, 7.3, 8.1, 8.2. & 8.3. See selected graphs below.

Mixed agreement was noted for QM essential standards 5.2, 5.3, & 6.1.

Sample statements reflecting challenges with online learning.

"Learning online is challenging for me how to navigate through Canvas. To be honest, I would prefer in person learning became I can interact with other students and get feedback right away."

"Group projects were difficult for the fully online format."                             "Make it feel more in person. Let students interact, no necessarily about the class." 

Sample positive statements about online learning. 

"I prefer online learning and hope there is still a lot of online options in the coming semesters."

"I like the asynchronous format for online learning."

"It is great & very convenient for those working full time and taking classes online.

"Before this semester, I took one or two courses online. The rest were on-campus courses. I was nervous about taking and managing everything online. My instructor is very supportive of my online learning experience."

Conclusions:

• The majority of students agreed that their courses met most QM essential standards, including standard 8.3 for  accessibility. 
• Most students felt that they were able to obtain help for academic and technology.
• Area to consider in future professional developmental trainings: 
  • Communications with students regarding feedback and grading.
  • Tools and learning activities that enhance student engagement and interactions. 
• A variety of course modalities should be offered in the future as some students preferred in-person instructions while others want to continue with online learning.

QA Leadership & Team 
Go Hornets!

(left to right)
Michelle Dang, faculty co-lead and mentor
Mark Rodriguez, faculty co-lead and mentor
Corinne Rowland, mentor and accessibility & instructional technology consultant
Debra Welkley, faculty mentor
Bronwyn Fields, faculty mentor
Deborah George, mentor and instructional designer
Tara Sharpp, faculty mentor

CTL Leadership

Lynn Tashiro, Director

We use a 3-tiered approach to meet the needs of faculty at various levels of readiness and increase the number of institutional experts for online teaching. 

On-going Training and Support: For 2020-21, we continued to offer QA trainings through the Chancellor's office and FLCs for online teaching and learning. We also incorporated the IYOC workshop as part of our FLCs in fall 2020 and spring 2021. 

For summer 2021, we added the Applying the Quality Matters Rubric (APPQMR) and Peer Reviewer Course (PRC) to our professional development offerings. Almost 100 faculty signed up for these QM FLCs. 

Peer Reviews: We continued to promote our internal peer review process to help faculty prepare their courses for QM certification. QA team members served as reviewers and mentors. We used data from our internal review process to identify patterns of "not met" standards; this strategy helped us proactively address these standards in subsequent reviews. For example, we noticed that a few faculty had challenges meeting SRS 2.4. As a result, we provided mentorship to help faculty meet this standard.

Since we have implemented this review process in 2019-20 AY, all courses that have met standards from our internal review process have subsequently attained course certifications.  

In 2020-2021, three (3) Sac State courses received QM certifications. Three (3) additional courses have completed internal reviews and are currently undergoing formal reviews (summer 2021). 

Preparing Leaders and Mentors: We increased the number of QM certified facilitators. We currently have 2 IYOC and 2 APPQMR  facilitators for online workshops on our QA Team. We have offered the PRC workshop for summer 2021 to increase our capacity of peer reviewers. Twelve (12) faculty have signed up to complete the PRC. 

To further strengthen our relationship with IRT, A QA team member currently serves as an Academic Technology faculty fellow, providing consultation and assisting with academic technology support. 

Course Reviews:  As noted, our Internal and formal course review processes provide mentorship and support for course certifications. We highlighted QM standards such as alignment and accessibility in our FLCs to increase awareness about essential components and promote successful reviews. Professional development funding was awarded to faculty who had met standards for internal and formal reviews. Since Fall 2019, we have increased the number of certified courses from 2 to 8. 

QM Certified Courses: 

During the continued transition to remote teaching for the 20-21 AY, QA leadership and mentors continued to provide support in the following ways:

• Tutoring sessions for faculty
• One-on-one faculty support
• Consultation with IRT on faculty needs
• Recruitment of faculty for QA trainings 
• Support in the development of the summer curriculum for online/remote teaching workshops
• Serve as facilitators and mentors for faculty learning communities, summer teaching institute, and summer course compression program.

As part of our efforts to support the "Finish in Four" initiative at Sacramento State, we have created opportunities for faculty to create online courses, particularly GE and bottleneck courses, that are eligible for CourseMatch and the CSU Fully Online program. In partnership with the College of Continuing Education, we have provided intensive support to 43 faculty over the past three years to redesign their courses to be fully online and compressed for summer. In addition, we have provided support to many other faculty through our FLCs, highlighting standards that would make their courses eligible for CourseMatch. We are pleased to see a significant increase in courses selected for CourseMatch over the past several years. 

We collaborated with nursing faculty to assess their students' perspectives and technology needs as a result of remote teaching. Our study has been accepted to two national conferences - Sigma Theta Tau International Biennial Convention, November 2021, and Nurse Educator's Conference, July 2021.

Conference Abstract: Challenges and Successes of Undergraduate Nursing Students in Online Learning During The COVID-19 pandemic

Authors: Christie Smart, Lyndsay Anderson, & Michelle Dang

Study's Aim: The purpose of this study was to understand factors that supported remote learning, engagement and retention during the transition as a result of the pandemic, and to identify barriers that impacted students’ learning in order to address these barriers as continued distanced learning in our nursing programs is anticipated throughout the pandemic.  

Methods: All enrolled undergraduate students from three different program tracks were invited to participate in an anonymous online survey. The survey included items related to facilitators and barriers to remote learning, instructor readiness and accessibility issues, as well as non-academic factors such as financial aid, internet accessibility, outside employment, family responsibilities, housing, food insecurity, and access to health services. Question types were multiple-choice, Likert-type scale, and open-ended.  

Results: A total of 168 individual responses from students in all three of our programs. Results revealed that most students experienced challenges as a result of the sudden transition to remote learning such as skills of faculty in online teaching, distractions at home and family responsibilities, barriers to accessing campus resources, and fear of not being able to complete their degrees. There were notable differences among students in the three different programs. Students in the traditional pre-licensure program who require clinical hours in acute care settings cited the inability to complete their clinical hours as a significant source of stress and a salient barrier to completing their degrees. Students in this program, which was designed for mostly on-campus instruction prior to COVID-19, reported significantly more technological challenges, higher stress levels, and reduced engagement with faculty and peers. In contrast, students in the other two programs, which were designed as hybrid, reported minimal technological challenges and a positive level of engagement with faculty and peers even after the transition.

Implications: Findings from the current study suggest that there is a need for training related to online teaching and technology for faculty who teach primarily face to face. The results also revealed opportunities for improvement in online teaching (such as issues related to accessibility) and how faculty can better support students throughout the pandemic. With ongoing University changes and new social distancing requirements, the data gained from these students is valuable to adapting nursing programs so they remain accessible to students from the home setting and provide quality education until it is safe to return to the classroom. The sudden transition to remote learning due to COVID-19 permits us to reflect on our current practices and consider how we can be better prepared for the future.   

1. We created a Canvas Course Template that incorporated QM Standards and made it available through Canvas Commons. We are using the template as part of faculty trainings to help faculty (re)design their courses using best practices. As of June 18, 2021, the template has been downloaded 870 times. We plan to update this course template in the next academic year. 

2. To help faculty create alignment for their courses, we created a course alignment template called the "Transformation Plan". The plan has separate tabs for each course component such as objectives, assessments, and learning materials. The data from each course component are automatically transferred to an alignment worksheet that permits faculty to check for alignment across all course components and make changes as needed.

We piloted the template in our summer online course compression program and received very positive feedback from faculty. 

"My favorite document was the Transformation Plan spreadsheet." 

"the transformation plan was excellent and provides a tool that I will continue to use and evaluate semester to semester in my current and future courses."

"I love that the course objectives map to activities and it helps to clarify why assignments are included in the class. I'm going to use it to revamp my existing courses too."

Image of the learning objectives tab in the Transformation Plan