2018-2019 — Fullerton — Quality Assurance ePortfolio
2018-2019 — Fullerton — Quality Assurance ePortfolio
Proposal Summary: For AY 18-19, we engaged faculty from a wide variety of disciplines and partnered with the Online Education and Training (OET) department to strengthen and expand our Quality Matters (QM) community. We were able to complete the goals of this project through a combination of QM Workshops (including two Fullerton APPQMR cohort sessions) and campus-based lunch and learn events. Our campus continues to work toward addressing the four areas of campus need identified in 2014 for online teaching, assessment, and quality assurance:
- Uneven level of expertise in online/blended pedagogy, design, and delivery
- Lack of understanding of faculty needs related to effective online/blended teaching
- Lack of coordination on strategies and choice of tools to ensure the quality of online and blended courses
- Lack of organization and community among departments to support quality online and blended teaching
Campus Goals for Quality Assurance
- Goal 1: Conduct a study to explore the impact of faculty Quality Matters (QM) expertise development on student learning experiences
- Goal 2: Expand the awareness and implementation of the QM process on campus
- Goal 3: Deepen QM experience for selected faculty
- Goal 4: Support expansion of QM training and course certification
- Goal 5: Disseminate project success and lessons learned
Quality Assurance Leads
- Marsha Orr, MSN, RN, Faculty QM Lead
- Sinjini Mitra, Ph.D., SQuAIR Lead
- Shelli Wynants, Ed.D., QMC/Campus Contact
Supporting Campus Partners
- Online Education and Training
- Faculty Support Services (Faculty Development Center, Academic Technology Center, Faculty Affairs and Records)
- Office of Assessment and Educational Effectiveness
- Information Technology
- CSUF Colleges/Departments
Campus Commitment Toward Sustainability of QA Efforts
- Host workshops and events for faculty on online teaching pedagogy and effective use of technology tools
- Disseminate information on Quality Matters to the campus to promote online quality assurance
- Support faculty and campus team members to attend professional development opportunities locally and nationally
Summary of Previous QA Accomplishments
QM Certified Peer Reviewers
- Chris Street, Ph.D., (2018) Professor, College of Education
- Shana Charles, Ph.D., (2017) Assistant Professor, College of Health and Human Development
- Whitney Youngren, M.A., (2017) College of Humanities and Social Sciences
- Shelli Wynants, Ed.D., (2016) Lecturer, College of Health and Human Development
- Sinjini Mitra, Ph.D., (2016), Associate Professor, Mihaylo College of Business and Economics
- Sudarshan Kurdwarkar, Ph.D., (2016) Associate Professor, College of Engineering and Computer Science
- Barbara Glaeser, Ph.D., (2016) Professor, College of Education
- Rachel McClanahan, DNP, RN, (2015) Associate Professor, College of Health and Human Development
- Suzanne Robertson, Ph.D. RN, (2014) Professor, College of Health and Human Development
- Marsha Orr, MSN, RN (2014) Lecturer, College of Health and Human Development
QM Certified Courses
Dissemination of QA Efforts— Conferences
- Hosted a roundtable discussion for faculty at the Annual University Assessment Forum on May 2, 2019. Title: Ensure Quality of Online Instruction: Designing a new online course or want to improve your existing online course? Join this table to learn about Quality Matters (QM).
- OET offered two Lunch & Learn events for faculty who completed QM training in the 18-19 academic year; the purpose of the events was to support faculty in applying the QM rubric to their courses and inform them of other QM opportunities (e.g., peer reviewer course, course certification). SQuAIR research findings were also shared.
Development of Campus QA Resources
Creation of two helpful faculty resources:
- Best Practices for Online Teaching Guide
The department of Online Education and Training (OET) created this guide to provide useful information about online instruction at CSUF to create opportunities for instructors to improve online teaching practices.Open Fullerton
The goal of the OpenFullerton site, created by OET, is to empower faculty in supporting student success by providing them a variety of resources and professional development opportunities to help reduce textbook costs, and by supporting their efforts to explore, cultivate, create, and access high quality Open Educational Resources (OER).
Next Steps for QA Efforts
By the end of AY19-20, CSUF hopes to achieve the following seven deliverables:
- 40-50 faculty/staff completing the online “Applying the QM Rubric” (APPQMR) workshop
- 10-12 faculty/staff completing the QM “Peer Reviewer Course” (PRC) online workshop
- 15-20 faculty completing a QM self-assessment of the essential standards for one of their online/blended courses
- 10-15 online courses being informally reviewed within the CSUF community
- 1- 3 courses completing QM certification
- 3-4 Lunch & Learn workshop events to support faculty application of QM standards in their online/blended courses
- A research report analyzing study results of QM-expertise on student learning outcomes and experiences
Training Completions
The following table summarizes all of the CSUF faculty and staff Quality Matters training completions that occurred during the 2018/19 academic year.
Training | Number of Completions |
Applying the Quality Matters Rubric | 30 |
Improving Your Online Course | 1 |
Introduction to Teaching Online Using QLT | 1 |
Orienting Your Online Learners | 1 |
Addressing Accessibility and Usability | 1 |
Peer Reviewer Course | 0 |
Peer Reviewer Recertification | 0 |
Master Reviewer Recertification | 1 |
APPQMR Facilitator for Online Course Recertification | 1 |
Additionally, CSUF offers a Teaching Online Certificate Program (TOP) that addresses quality assurance best practices aligned with both the QLT and QM rubric. The following table summarizes all of the CSUF faculty and staff who completed TOP courses during the 2018/19 academic year.
Training | Number of Completions |
TOP 1 - Introduction to Online Teaching | 26 |
TOP 2 - Designing a Quality Online Course | 20 |
TOP 3 - Engaging Students Online | 16 |
TOP 4 - Assessment in an Online Environment | 8 |
Accessibility/UDL Efforts
- OET offers an online certificate program for Accessible Instructional Materials (AIM). Our AIM certificate program has 3 sequential modules; each 4-week module is designed to take 5 hours to complete and is a combination of instructional activities, collaborative discussions, and individual application. In this program, faculty become more aware of the various disabilities of their students and the resultant issues, including legal and regulatory obligations, instructional needs, and campus resources. This program also includes an introduction to Universal Design for Learning and its application in creating accessible documents and captioning instructional videos.
- OET provides resources for faculty on accessibility and UDL on its website.
- CSUF purchased Ally for the LMS to help faculty.
Student Quality Assurance Impact Research
Goal #1: Study the extent of Quality Assurance training at CSUF
Total: 168 faculty/staff
Type of Quality Assurance Training | Number of Faculty/Staff |
---|---|
APPQMR (QM) | 98 |
QOLT | 9 |
Peer Reviewer Course (QM) | 12 |
Master Reviewer Course (QM) | 1 |
QOLT Reviewer | 1 |
Other | 13 |
Type of Course Review & Certification | Number of Courses |
Informal (campus-level) | 8 |
QM certification reviews | 12 |
QM certified | 11 |
QOLT | 8 |
Goal #2: Study the impact of faculty QA trainings on student performance in online courses at CSUF
Student Performance Outcomes – Repeatable (DFW) grades & GPA
- For Fall 2016 and Spring 2017, courses taught by QM-trained faculty had statistically significantly lower DFW rates than those courses which were taught by faculty with no QA-related training (p-values < 0.0001)
- Average course GPA was highest for QM-trained faculty taught courses
Semester | QM-Trained (DFW) | Non-QA-Trained (DFW) | QM-Trained (GPA) | Non-QA-Trained (GPA) |
Fall 2016 | 9% | 12% | 3.22 | 3.05 |
Spring 2017 | 8% | 11% | 3.08 | 3.01 |
- Repeatable grades (DFW rates) for underrepresented (UR) students were significantly lower for online courses taught by faculty with some type of QA training
Semester | QM-Trained (UR DFW) | Non-QA-Trained (UR DWF) |
Fall 2016 | 7.93% | 11.51% |
Spring 2017 | 6.65% | 10.13% |
Student Survey Data – Quantitative Feedback
- Survey developed by CSU Chancellor’s Office
- 25-item survey aligned with QM and QOLT standards, using a 6-point Likert scale (1: Strongly Disagree; 2: Disagree; 3: Somewhat Disagree; 4: Somewhat Agree; 5: Agree; 6: Strongly Agree)
- Overall, very positive feedback from students regarding the course design
- Limitation: Sample size is small (n = 27 responses)
Course Design Characteristic | Mean Rating (out of 6) |
Accessibility and Universal Design | 4.62 |
Learner Support and Resources | 4.69 |
Technology for Teaching and Learning | 4.60 |
Facilitation and Instruction | 4.72 |
Student Interaction and Community | 4.72 |
Instructional Materials and Resources Utilized | 4.72 |
Assessment of Student Learning | 4.77 |
Course Overview and Introduction | 4.85 |