banner

QA Results

Exploring the Canvas LMS

Andrea Saltzman Martin and Mark Laumakis, two instructors and lead facilitators of the SDSU Course Design Institute (CDI) assessed their Canvas course designs in Fall 2019. They used the SQuAIR student survey, a departmental end of course survey and the Student Assessment of Learning Gains (SALG) survey. Response rates for the two courses averaged 25% with the majority of students responding positively.Both came away not only with immediate insight on improvements to their courses but valuable recommendations for faculty moving their courses online.  

  • Canvas Affordances such as the To Do lists and Progress Indicators help students stay on task and navigate the course.
  • Changing the course home page each week so students “are dropped into the current module” keeps students focused and on task. 

 Ideal Course Templates

With the decision to transition to Canvas in May 2020, faculty began opting in to using the LMS and an initial course template was provided which includes:

Front page of Course Template

Accessible Syllabus Template

Example of Course Accessibility Report

Course Planning Grid

Evaluation Results of Canvas Pilot

In May 2020 after two cohorts of faculty and students had tested Canvas, Maureen Guarcello and Marcie Bober-Michel, evaluation leads, presented a 17-page report entitled: Capturing the Canvas Experience: Spring 2020 Pilot Survey Synopsis.

Results of the data collected to that point (including outcomes involving 7,479 students, 100 faculty, and 168 course sections at the heart of the Fall 2019 / Spring 2020 Pilot) suggested that an array of campus-wide needs (e.g., mobile readiness) are not currently supported by the Blackboard LMS.  Most notably, the COVID-19 pandemic underscores the lack of flexibility and functionality needed to provide the learning experiences and interactions both students and faculty expect and deserve. These gaps are equally perceived by faculty and campus leaders as they work to keep students informed and engaged.  With the upcoming virtual Summer Session (and potential Fall 2020 continuation should the pandemic fail to resolve as anticipated), it was recommended that the LMS transition begin immediately. 

Of the pilot participants, 35% of faculty teaches online or hybrid courses, with 55% of students attending online or hybrid courses. 

Overall, faculty and students preferred Canvas, giving it a Learning Management System “grade.”

Faculty assigned Canvas a 3.4 GPA or B and Blackboard a 2.0 or C-. Students (report cards on the right) assigned Canvas a 3.5 GPA or B+, and Blackboard a 2.9 or C+.

Implications for Quality Assurance

Students can’t completely distinguish between “standard” Canvas functionality and functions available to them because their professors made various course design choices. Canvas-savvy students point more specifically to a lack of faculty “readiness.”  

No matter the modality by which it is offered, faculty training/orientation must focus on the many Canvas functions to which students expect access.  Students must also be oriented to such functions, in particular those that they themselves must activate (e.g., providing  secondary contact information).


In terms of course design, faculty must be oriented to workarounds for features/functions that require newer mobile devices to ensure all students have access to site content.


Additionally, several scaled items directed to students focused on the ease or difficulty of locating specific site elements (syllabus, assignments, etc.); most earned mean ratings between 2.0 and 2.5, on a 1 to 5 scale of “very easy” to “very difficult.”  Many students used the open-ended questions to elaborate--often noting confusion over the way their faculty chose to organize the class or specific aspects of it.  Sound transition/implementation strategies must include the criticality of student input before a course gets underway.

Investigating the Quality Matters instrument 

With the Provost’s interest in moving to Quality Matters, we created a QA Resource Team. The team, four Faculty Fellows and five instructional designers participated in Quality Matters training in February and March. The entire team is certified in Improving your Online Course  with three members certified in Applying the Quality Matters Rubric. In addition, we conducted a cost analysis to inform a proposal to the President’s Budget Advisory Committee for funding in 2021. 

QM training of key faculty and instructional designers has forged more explicit collaborations on QA issues with campus partners including World Campus, the Center for Teaching and Learning and college and departmental leaders as the interest in online courses increases. The Course Design Institute and the Flexible Course Design Summer Institute are critical vehicles for online course design training and QA.

Training Completion

Quality Matters
Total
Participants
Faculty/Instructional Designers

Improving Your Online Course 

9
4/5

Applying the Quality Matters Rubric 

3
1/2