Quality Assurance for blended and online courses

2019/2020 — CSU Stanislaus — Quality Learning and Teaching - Faculty Learning Community (QLT-FLC)

Proposal Summary: At the conclusion of the 2019/20 QLT Faculty Learning Community project, we were able to (1) increase understanding of quality online teaching and learning, especially as connected to the elements of the QLT rubric; (2) assist faculty in developing or revising wholly online courses to meet QLT expectations; (3) improve faculty ability to self-assess and peer-evaluate online courses; (4) Continue to develop a peer mentorship program where previous QLT participants supported new members of the 2019/20 Faculty Learning Community.

Campus QA Goals

Campus Need for Quality Assurance 

  • Continued understanding of the components of a quality online (or hybrid) educational experience
  • Continued familiarity with the QLT evaluation tool and the self- and peer-evaluation process
  • Continued understanding of faculty needs with regard to online (and hybrid) teaching
  • Stronger infrastructure/support for faculty teaching online (and hybrid) courses and students taking such courses

Proposal Goals Based on Need

  • Increase understanding of quality online teaching and learning, especially as connected to the elements of the QLT rubric
  • Assist faculty in developing or revising wholly online courses to meet QLT expectations;
  • Improve faculty ability to self-assess and peer-evaluate online courses
  • Train a cohort of QLT Peer Mentors; and develop an ongoing program to promote peer-review for online courses
  • Develop an ongoing program to promote peer-review for online courses
  • Help department chairs (and others) better support and determine the quality of online teaching by their faculty.
  • Review QLT participant courses by certified CSU peer reviewers so that the courses can be added to the CSU Quality Assurance Resource Repository
  • Evaluate the student outcomes of online courses developed through previous QLT-FLCs in alignment with our local graduation initiatives and institutional student success plan.

Quality Assurance Lead(s)

  • Daniel Soodjinda, Assistant Professor, Liberal Studies

Supporting Campus Partners

  • Shradha Tibrewal, Director, Faculty Center for Excellence in Teaching and Learning
  • Glenn Pillsbury, Instructional Designer, OIT/Learning Services

Campus Commitment Toward Sustainability of QA Efforts

  • Support faculty in their efforts to develop quality online and hybrid courses via QLT faculty learning communities, faculty professional development opportunities, and one-on-one support
  • Develop a system of accountability for online and hybrid course development that is aligned to quality assurance measures such as the QOLT or QM evaluation tools 

Summary of Previous QA  Accomplishments

Prior to the introduction of QLT at CSU Stanislaus

  • Regular workshops on pedagogy, course planning, and technology tools delivered by the Faculty Center for Excellence in Teaching and Learning and OIT staff each semester.
  • Spring 2010 Online Teaching Bootcamp: Faculty were invited to attend a three-day series of workshops that introduced significant pedagogical and technological components to successful online teaching.
  • Spring 2011 Faculty Roundtable on Cheating in Online Courses: A select group of instructors were invited to present their experiences and ideas related to the question of cheating in online courses.  Discussion explored the tension between pedagogical and technological solutions (e.g., ProctorU).

QLT Introduction at CSU Stanislaus

  • Spring 2013 QLT workshop and competition: Faculty were invited to attend a three-day series of workshops that introduced each section of QLT along with significant resources and tools related to them.  The participants each self-assessed one of their courses that was in the process of being designed or converted to an online format. 
  • 2013-2014 QLT competition completion - one instructor was selected as a system finalist for that year's competition.
  • Continued QLT efforts at CSU Stanislaus
    • 2014 QLT workshop:  In May, 2014 a workshop was held providing an overview of the QLT evaluation tool, which discussed the similarities and differences between online teaching and learning and face-to-face teaching and learning, and considered the types of supports faculty may need for course development and facilitation. 
    • In Summer 2014 we hosted the "Supporting Quality Online Learning and Teaching Faculty Learning Community." 
      • Ten faculty members chose to participate in the FLC, representing the disciplines of anthropology, art, business administration, communication studies, criminal justice, kinesiology, psychology, public administration, and theatre. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Fall 2014, some were also engaging in transformation of summer 2014 courses.
      • Nine of the ten FLC members were able to complete course redesign, and engage in self and peer evaluation. Upon completion of course design/redesign, FLC members identified between 46 and 54 objectives as "relevant" for consideration in the evaluation of their course (since most courses had not yet been taught, objectives related to the facilitation of the course were typically not able to be assessed). Self- and Peer-evaluations indicated 87-100% of relevant objectives were evaluated as "meeting" or "exceeding" expectations on the QLT evaluation tool.
    • 2015 QLT workshops:  In August, 2015 a set of workshops were held for incoming participants of the 2015/2016 QLT Faculty Learning Community (FLC).  The workshops provided an overview of the QLT evaluation tool, which discussed the similarities and differences between online teaching and learning and face-to-face teaching and learning, and considered the types of supports faculty may need for course development and facilitation. 
    • During the 2015-2016 academic year we established a QLT-FLC that consisted of 7 faculty members representing the disciplines of Education, Child Development, English, Social Work, Nursing, and Biology. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Fall 2015, some were also engaging in transformation of summer 2016 courses. 
      • In an effort to help support the 2015/2016 QLT-FLC during 2015-2016 academic year, we also developed a QLT peer mentor program that consisted of 7 faculty members who were assigned to QLT-FLC participants in a effort to support them in developing an online course.
    • During the 2016-2017 academic year we established a QLT-FLC that consisted of 7 faculty members representing the disciplines of psychology, nursing, art history, kinesiology, political science, and English. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Fall 2017, some were also engaging in transformation of summer 2017 courses. 
      • In addition to supporting the facilitating the 2016/17 QLT-FLC, we also further developed the QLT-FLC mentor program that consisted of 3 faculty members who were assigned to QLT-FLC participants in an effort to support them in developing an online course. 
    • During the 2017-2018 academic year we established a QLT-FLC that consisted of 6 faculty members representing the disciplines of psychology, nursing, art history, kinesiology, political science, and English. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Spring 2018 or Fall 2018, some were also engaging in transformation of summer 2018 courses. 
      • In addition to supporting the facilitating the 2017/18 QLT-FLC, we also further developed the QLT-FLC mentor program that consisted of 2 faculty members who were assigned to QLT-FLC participants in an effort to support them in developing an online course. 
    • During the 2018-19 academic year we established a QLT-FLC that consisted of 6 faculty members representing the disciplines of advanced studies in educational technology, English, kinesiology, psychology, and sociology. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Spring 2019 or Fall 2018, some were also engaging in transformation of summer 2019 courses. 
    • During the 2019-20 academic year we established a QLT-FLC that consisted of 8 faculty members representing the disciplines of Teacher Education, Sociology, Communication Studies, Accounting/Finance, Computer Science, and Psychology. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Spring 2020 or Fall 2020, some were also engaging in transformation of summer 2020 courses.

 

Dissemination of QA Efforts— Conferences 

  • 2019 - Directors of Educational Technology / California Higher Education (DET/CHE) Annual Conference, "Letting the Elephant Out of the Closet! Building a Culture of Quality through QLT Course Reviews and Certificaiton." Ashley Skylar, Megan Eberhardt-Alstot, and Daniel Soodjinda, A link to the presentation can be found HERE
  • 2015 - Loyola Center for Teaching Excellence, "Maintaining Quality and Rigor within Digital Learning Environments."  Daniel Soodjinda
  • 2014 - National Conference on Race in Higher Educaiton (NCORE), "What is the Future of Higher Education: Moocs, Mandates, and Mayhem." Daniel Soodjinda

Quality Assurance Results

Training Completions

The following table represents all QLT-FLC faculty who have completed QA trainings for 2018-2019

Course2014 - 20192019 - 2020Total
Introduction to teaching online using QLT181634
Reviewing courses using the QLT instrument10111
QM teaching online certificate150
1151
Total 178
196


Course Peer Review and Course Certifications

Fall 2019/Spring 2020 Faculty Learning Community

This year we had an 88% completion rate.  We believe that the high completion rate is attributed to more formalized accountability "check-ins" that occurred every 4 weeks via synchronous online meetings, and additional accountability checks completed by the QLT-FLC mentors. 

Sessions were held throughout the academic year, and incorporated face-to-face and online components. Participants were introduced to the QLT instrument, obtained knowledge and access to resources in support of curricular redesign and received training to self- and peer-evaluate courses. Faculty development funds were awarded to six faculty participants who developed a course meeting expectations in at least 75% of the relevant objectives indicated in the QLT rubric. These faculty members also completed and received a peer-evaluation.

Development of Campus QA Resources

Additional resources that were examined during this particular FLC are detailed below:

  • Resources were provided to faculty about Universal Design at our face-to-face meetings and through a self-directed Blackboard Course.  For access to the self-directed course (via CSU Stanislaus Blackboard), please contact Bob Koehler at bkoehler@csustan.edu.
  • An online workshop that highlighted a self-developed QLT course development framework and recommendation matrix.  The matrix was developed to support faculty as they develop their online courses by breaking up the lengthy and cumbersome QLT rubric into different areas that faculty can focus on.  The areas explored include: 
    • Pre-planning:  This is the first thing that faculty should first think about when developing an online course. Faculty don't necessarily have to be detailed here, but should at least have some type of framework for the items listed in this area before they start building the course on their chosen platform.
    • Syllabus development:  Once faculty complete step 1 (pre-planning), a syllabus can then be developed that explains the instructor's policies, course's expectations, assignments , grading, etc. to the items students will find in the online course. This can, and should, be revised after the course has been developed to make sure that things are clearly aligned.
    • Course build (primary): At this stage, faculty can then begin to build their online course by considering the more important elements outlined.  The elements are somewhat in order of importance, but faculty should view all elements before  they begin to build their course so that the course's flow and alignment between objectives/materials/assessments makes sense. 
    • Course build (secondary):  After a framework for the course is built in point 3 above, faculty can then go back and add in these additional elements to supplement what was already developed. 
    • Course delivery/maintenance:  Faculty should be aware of the elements listed in this column, but don't have to address it until the course has begun and students are actively participating. 
  • A presentation about Google Docs suite of applications was provided to faculty participants. Resources relating to the presentation can be found by utilizing the two links below. A discussion about Google Hangouts took place, and a resource was provided relating to Google Hangouts for lecture capture: http://www.eduhacker.net/technology/google-hangouts-lecture-capture.html
    • Google Docs support page:  https://support.google.com/docs/answer/49008?hl=en 
    • Google Apps in Classrooms and Schools:  http://bit.ly/1dEl7lB
  • A presentation was provided to faculty participants about best practices in supporting students with disabilities using the principles of Universal Design. A link is provided here to the presentation's resources:  https://www.csustan.edu/accessible-technology-initiative. A discussion relating to the transformation of Powerpoint or Keynote presentations to video resources took place, and the discussion's resources can be found here:  http://bit.ly/1KTQiHe
    • Additional resources about adding subtitles to online presentations can be found here:  https://support.google.com/youtube/answer/2734796?hl=en
  • A presentation was provided to faculty participants about best practices involving Voice Thread in an online course. 

Participating Faculty/Team Feedback 

QLT-FLC Participants reported that the accountability "check-ins" helped keep them on track. During these online meetings, faculty shared out the work they completed in their online courses at different points in the program and learned about an additional topic of interest that they were surveyed on.  Because of this feedback, we plan to continue the formalized check-ins during the 2020-2021 QLT-FLC program, along with the mentorship program.

Accessibility/UDL Efforts

A presentation was provided to faculty participants about best practices in supporting students with disabilities using the principles of Universal Design. A link is provided here to the presentation's resources.

Next Steps for QA Efforts 

Building upon the work that has already been completed with QLT, we intend to engage in several activities to further support high-quality online teaching and learning and to promote sustainability of the self- and peer-review process. The Faculty Learning Community model has proven effective as a means to support quality online course design on our campus. Participation to date shows increased faculty interest in participating in Faculty Learning Communities during the summer, when scheduling is less cumbersome. For this reason, the campus team is initiating a QLT-FLC for 2020/21 that is more in-depth.  Steps for the 2020/21 QLT program include the following:

  • Continued expansion of campus webpages that provide comprehensive information about quality online teaching and learning, and the support services and resources available to faculty and students.
  • A panel discussion to share current practices in online and hybrid teaching on our campus with those considering teaching in an online or hybrid format. The panel discussion will feature faculty who have utilized QLT to design and evaluate their online courses.
  • FLC: A Faculty Learning Community to Design or Improve an Online Course for approximately six faculty members scheduled to teach a wholly online course in Fall 2020 or Spring 2021. Faculty may either be new to online teaching or desiring to revise an online course. Sessions will be held at the end of Summer 2020, through Spring 2021 incorporating face-to-face and online components. Participants will be introduced to the QLT instrument, obtain knowledge and access to resources in support of curricular redesign in alignment with the QLT objectives, and training to engage in self- and peer-evaluation of online courses. Faculty development funds will provided to those who, based on self- and peer-evaluation, have developed a course meeting expectations in at least 75% of the relevant objectives indicated in the QLT rubric, and have engaged in the peer evaluation of at least one course.  Twenty hours of gatherings (plus voluntary work time) are expected.
  • FLC-M: A QLT Peer Mentor Faculty Learning Community for approximately three faculty members wishing to become designated QLT peer mentors on our campus. These will be faculty who have participated in QLT self- and peer-review through past FLCs. Participants will attend FLC activities, complete the Applying the QLT Instrument online course offered through the Chancellor's Office, and perform peer-evaluation of three online courses–including a debriefing follow-up meeting between Peer Mentors and faculty with courses being reviewed. Twenty-five of online work plus three to five hours of face-to-face meetings are anticipated.
  • To identify a funding source to compensate trained QLT-M Peer Mentors for future peer review work at the request of faculty members teaching in the online format.
  • Support for faculty engagement in system-wide Quality Assurance trainings. The University will fund up to two rental cars per event to support groups of faculty, staff, and administrators who wish to attend local area or CO sponsored workshops or conferences relating to online teaching and learning.
  • One workshop targeted to department chairs, administrators, and members of faculty personnel committees (for lecturers, tenure and promotion, post-tenure review) regarding supports needed for online instructors and means to determine the quality of online teaching and achievement of student learning outcomes.
  • Formalize a local QLT assessment process, where our experienced QLT-M assessors can evaluate online courses for a stipend
  • Have four courses formally reviewed by certified CSU peer reviewers so that the courses can be added to the CSU Quality Assurance Resource Repository
  • Evaluate the student outcomes of online courses developed through previous QLT-FLCs in alignment with our local graduation initiatives and institutional student success plan.