Quality Assurance for blended and online courses

2018-2019  Cal Poly Pomona: Hybrid MBA Courses and Quality Matters

Proposal Summary: This project supports Cal Poly Pomona’s (CPP) College of Business Administration (CBA) collaboration with the Faculty Center for Professional Development and eLearning to build MBA core courses as Quality Matters (QM) certified hybrid courses. The project’s overall purpose was to bring together a community of educators to collaborate in developing and sharing best practices for hybrid courses.   Nine faculty members, all teaching or planning to teach 6 core MBA courses, participated.

Campus QA Goals

Campus Goal for Quality Assurance

This project for the “MBA-Hybrid” supports a College of Business Administration (CBA) collaboration with the Faculty Center for Professional Development and eLearning to build MBA core courses as Quality Matters (QM) certified hybrid courses. We brought together a community of educators to collaborate in developing and sharing best practices which will apply to undergraduate as well as graduate students.

Semester conversion at CPP has proliferated courses, adding demand for classrooms. Hybrid courses will help with classroom demand and increase access to students.  The eLearning and the Faculty Center’s goal for quality assurance—that every faculty member who works with us learns about and adheres to quality assurance principles—are aligned with the CBA’s priority on blended and online courses.  

Nine faculty members, all teaching or planning to teach the 6 core MBA courses, participated:

  • GBA 6150 Leadership and Ethics in Organizations
  • GBA 6080 Accounting and Managerial Decisions
  • GBA 6520 Marketing Management Applications
  • GBA 6730 Strategic Information Technology
  • GBA 6050 Data Analytics and Modeling
  • GBA 6870 Management Strategies

These core courses reflect the intention to measure educational effectiveness and other performance metrics on the whole MBA student body.

Although the graduation rate of our MBA students is not a problem, this project does address two strategies in Cal Poly Pomona’s local Student Success Plan: “Success in low completion rate courses,” particularly focusing on pedagogical training, and “Digital learning to engage students and expand success.”  eLearning’s efforts are specifically mentioned in the latter strategy.  The faculty involved in this project teach low-completion-rate undergraduate courses in addition to the MBA courses and will be able to apply what they learn in this targeted effort to their undergraduate responsibilities.

Quality Assurance Lead(s)

Victoria Bhavsar, Director of the Faculty Center for Professional Development and the eLearning team

Supporting Campus Partners

Instructional Designers and Multimedia Developers:  April Dawn, Bo Choi, Erick Zelaya, Eric Davis, and Richard Feldman

Campus Commitment Toward Sustainability of QA Efforts

  • We have created a Quality Matters template for Blackboard
  • We offer QM to every faculty member who requests instructional design consultation
  • We offer regular QM and QM-related workshops

Summary of Previous QA  Accomplishments

The 2016-2017 “DOLCE 3.0” QA project resulted in complete transformation of 9 courses including 5 difficult quantitative courses, and 4 non-quantitative but key courses.  The 2017-2018 project, DOLCE 3.1, produced meaningful changes such as:

  • Attention paid to alignment in all classes
  • Clarification of learning objectives and relationship of learning objectives to activities
  • Clarification of planned response time and feedback
  • Adoption of Cal Poly Pomona’s QM Blackboard template
  • Re-organization of course materials into clear, logically structured modules

This project was more like 2016-2017 DOLCE 3.0, in which a group of faculty worked intensively for a shorter period of time with significant one-on-one support.  

Previous to DOLCE 3.0, our approach to QM centered on informal, “CPP-QM” reviews, along with stipends for course revision.  We provided QM training and stipends for faculty to perform reviews.  Semester conversion, scheduled for Fall 2018, proved to be a significant barrier:  repeated personal outreach and stipend availability failed to engage more than 2-3 faculty in reviewing because faculty simply did not have time due to semester conversion.  Also, courses reviewed in the quarter format would need to be re-reviewed as semester courses.

Current Dissemination of QA Efforts

  • We held a 2-hour Intro to QM workshop on Friday March 22, 2019, following a large all-campus workshop “Online Teaching at Its Best” by Linda Nilson.  Seventeen faculty participated. 
  • One of the Hybrid MBA faculty, Kristen Schiele, presented a paper, "Improving Student User Experience (UX) with Quality Matters" at the April 25-26, 2019 Quality Matters conference in New York City
  • The QA lead Victoria Bhavsar consulted with the College of Education & Integrative Studies on appropriate summative evaluation of online courses.  QM was NOT recommended as an approach for summative evaluation, but the College was encouraged to support faculty in adopting the QM framework.

Quality Assurance Results

Course Peer Review and Course Certifications

  • All faculty teaching in the hybrid MBA program completed QM’s “Design Your Blended Course” in Spring 2018.  
  • The plan was for faculty to create the online portions of their hybrid courses by August 23 whether their course would be taught in Fall 2018 or Spring 2019.  Each faculty member had an eLearning instructional designer assigned for enhanced support and to maintain progress.  
  • In July 2018, eLearning and the Faculty Center facilitated a 4-day retreat, “Complete the Build,” based on the QM framework.  The agenda included several hours each day for faculty to work on their courses, with limited presentations about the updated QM Rubric 6th edition, accessibility, and active learning in online courses.  Faculty were encouraged to seek and provide collegial feedback as they progressed. 
  • Three hybrid MBA courses were taught in Fall 2018 as scheduled, with two more taught in Spring 2019 as scheduled.  
  • Informal, campus level reviews were completed for GBA 6080 Accounting and Managerial Decisions, GBA 6520 Marketing Management Applications, GBA 6730 Strategic Information Technology, and GBA 6870 Management Strategies
  • Due to the support of the Dean of the College of Business Administration, 19 Business faculty members completed QM trainings in 2018-2019.  These included Designing Your Blended Course (11), Designing Your Online Course (1), Creating Presence in Your Online Course (1), Improving Your Online Course (3), and Applying the QM Rubric (3).

Student Quality Assurance Impact Research 

We did not participate in SQuAIR because SQuAIR focuses on fully online courses.  This project did not include any fully online courses, and in fact there are no fully online courses in the MBA program at this time. 

Accessibility/UDL Efforts

We have significant accessibility efforts at Cal Poly Pomona, including frequent faculty trainings such as sessions at new tenure-track and lecturer orientations.  We include including accessibility as a fundamental consideration in every instructional design and multimedia development consultation.  We are in a slow roll-out of the Blackboard Ally tool, with approximately 60 faculty and more than 250 individual courses with Ally enabled from Spring 2018 to Spring 2019.

Next Steps for QA Efforts 

We will identify a faculty “eLearning Accessibility Champion” in each college to serve as a representative in their college for accessibility and quality assurance. The goal is to promote awareness of inclusivity, best practices, and continuous improvement.  The proposal aligns with CSU Quality Assurance goals to:

  • Apply principles of Universal Design for Learning and accessibility to online courses toward greater success of all students
  • Create a network of faculty, staff, and administrators informed of the various quality assurance factors, tools, and resources that enable effective online teaching and learning.