Champions for Accessibility and Inclusivity (CHAI) 2024-2025

Assessing our work and strengthening our community:

The Center for the Advancement of Faculty Excellence (CAFE) has completed its 6th year in support of the Cal Poly Pomona (CPP) Champions for Accessibility and Inclusivity (CHAI) project. The driving force of CHAI is creating excellent learning experiences for each student. We believe that the joy of learning, the freedom to grow, and the benefits of education are basic human rights at a university. CHAI embodies the values that accessibility and high-quality instruction are fundamental tools in making these rights a reality. 

CHAI serves as a model in CPP’s long-standing commitment to accessibility, creating a faculty-to-faculty approach of promoting awareness of best practices and a focus on continuous improvement, with direct help from a team of expert instructional designers.  CHAI creates a network of empathy, both towards students who benefit from Universal Design for Learning and accessible course materials, and towards faculty who provide those materials. We recognize accessibility heroes and inspire others to join in. 

In the 2024-2025 year, the objectives of CHAI were to:  

• Provide “faculty reciprocal peer coaching” (FRPC) for course improvement, using the QLT rubric as an improvement framework 
• Perform informal “Espresso” QLT reviews  
• Systematically assess faculty reciprocal peer coaching and Espresso reviews
• Additionally, progress on evaluating the impact of QM/QLT training on student outcomes 

We align with the California State University Online Course Services (CSU-OCS) goals to:  

• Apply principles of Universal Design for Learning, accessibility, and equity in online courses toward greater success of all students, including those with disabilities. 
• Create a network of faculty, staff, and administrators informed of the various quality assurance factors, tools, and resources that enable effective online teaching and learning.

CAFE assists faculty in making course materials accessible. Beyond that, our goal is that every faculty member who works with us is introduced to quality assurance principles, receives support for using quality assurance resources for any course format, and is introduced and supported to use best practices for Universal Design for Learning, accessibility, and inclusivity in all instructional materials.

Our 2024-2025 project extended the reach of our 2023-2024 project, which had differed from previous CHAI projects in that instead of focusing on training faculty, we investigated the results of previous work.  In 2023-2024 we began to assess the impact of OCS trainings on teaching practices and student success.  In 2024-2025 we continued and expanded this work, and offered an effective “faculty reciprocal peer coaching” (FRPC) program that connected faculty with one another for course quality improvement.

Faculty peer-to-peer coaching involves semester-long or yearlong, structured partnerships between two or sometimes three faculty who offer collegial support, ideas, and constructive critique in the implementation of selected teaching strategies (Rainville et al., 2024).  Faculty engage in training to provide effective peer-to-peer coaching and to select teaching strategies. They then visit and observe class meetings or otherwise review course materials two to three times per semester, offering mutual feedback based on agreed-upon questions of practice.  Pre-visit and post-visit conversations are included. Peer coaching is completely formative, i.e., non-evaluative.

We planned to offer FRPC to 12 faculty teaching sections of 6 courses identified by campus leadership for intensive DFW and equity gap remediation. We planned to use the QLT rubric as a foundational asset to guide faculty in their work together. 

• Fall 2024: 24 faculty completed FRPC.  9 non-STEM faculty were supported by the Quality Assurance program, while 15 STEM faculty were supported by an NSF grant for STEM education, CPP INVESTS (Intentional Venture Engaging STEM Students, Grant No. 2122567). 
• Spring 2025: 16 faculty completed FRPC, with 10 non-STEM supported by Quality Assurance and 6 supported by CPP INVESTS.

While we more than achieved our participation goal, unfortunately only one faculty member from our “target” courses participated in FRPC despite direct, personal, repeated, individual outreach.

Upon consideration we elected not to do a “deep dive” into the QLT rubric for FRPC training, as originally plan. The premise of FRPC is that instructors take the lead in their own experience. We determined that using the QLT rubric would be too prescriptive and also off-putting for the coaching role.  We developed a rubric for classroom observation that drew heavily from Area 5 “Facilitation & Instruction” of the QLT rubric. Instead, we offered “Espresso” reviews, described below, to all FRPC faculty. 

Individual Course Equity Portals were available for all participants except for the few brand-new faculty who did not yet have equity portal.  Participants were encouraged to look at their equity portals to think about teaching strategies to try.  We also created a stable Canvas course site with resources and information that faculty could refer back to as they proceeded in peer coaching.

In future, we plan to prepare the Canvas site for public access.

Rainville, K.N., D.G. Title, and C.G. Desrochers. (2024).  Peer coaching in higher education: Faculty coaching partnerships to support rigorous and engaging classrooms. Charlotte, NC: Information Age Publishing. 

At CPP, full QM and QLT reviews have not been productive, being far too time-consuming and not valued in the faculty reward system.  “Espresso” reviews are fast and focused, addressing only the core QLT standards, plus one extra standard in QLT Section 5 (Facilitation and Instruction) focusing on inclusion and belonging that the faculty identify.  Two instructional designers or trained faculty reviewers divide up the core standards and spend no more than two hours reviewing the course.  Each standard receives only one review. Standards are marked as 1 – Needs major improvement; 2 – Needs minor improvement; 3 – Meets expectations.  Feedback is provided for strengths and areas to work on. Reviewers meet synchronously with the faculty member to share and talk through findings. Faculty can revise and request a new review if they want.

We provided 17 Espresso reviews in the 2024-2025 year, up from 10 in 2023-2024, thereby meeting our goal of increasing the number of reviews. Faculty received an “Explorer” level badge for completing the initial review.

Class sizes at CPP will increase with the emerging budget crisis. Quality assurance principles and strategies can improve the quality and manageability of larger courses regardless of format.  Leveraging faculty’s learning about online and hybrid courses will be of great help, as this comment exemplifies:

“[Following my Espresso review], I got less than 10 emails from my 30 students about an assignment, compared to last semester when I received over 80 emails! WAY more organized is WAY less work for me.”

In addition, “Accessibility Challenge 2026” comes to a head in April 2026 when changes to the Americans with Disabilities Act (ADA) Title II regulations will significantly strengthen enforcement, affecting all digital instructional materials at Cal Poly Pomona. Larger courses present more risk.

Our goals for the 2025-2026 CHAI program are:

1. Support faculty teaching larger courses who have not previously had QLT training to engage in training; provide Espresso reviews following completion 
2. Engage and provide resources for Champions for Accessibility & Inclusivity (CHAI) faculty from each College at CPP. 
3. Other faculty development: 5 short zoom-based workshops open to all faculty in August 2025; January 2026, 2-day in-person working retreat for faculty to improve their larger courses. 
4. Directly assess student response to salient QLT strategies.

Faculty Lead: Maha Ghosn, International Business & Marketing

SQUaiR Fellow: Maha Ghosn, International Business & Marketing

Supporting Campus Partners:

• Director of the Center for the Advancement of Faculty Excellence Victoria Bhavsar (emeritus)
• Instructional Designers April Dawn (emeritus), Toni Davis, & Donya Rahimi
• Administrative Analyst Carla Tetreault

The OCS faculty lead/SQuAIR fellow and CAFE team attained approval for an IRB protocol for “umbrella” coverage of CAFE programs. Although unplanned, this activity supported the assessment described below, and will facilitate future Quality Assurance data collection. 

Peer coaching

Funded by CPP INVESTS, qualitative researchers probed into the faculty experience of peer coaching, and are currently evaluating survey response data concerning the student experience in these classes.  This freed our SQuAIR faculty fellow to do other work such as the umbrella protocol. 

Faculty unanimously reported significant pedagogic learning and growth and found the program to be an extremely effective means of transformation of teaching practices. Most focused on classroom dynamics and student engagement from different groups. Faculty enjoyed being able to connect with other faculty (particularly lecturers) and found this relationship beneficial at both a social and professional levels. They all hoped that the program would continue to be offered every semester and indicated their interest in continuing in the program. The Canvas site was useful, but only when faculty were reminded and encouraged to look at the resources. 

Despite the benefit that faculty participants experienced, most faculty expressed unwillingness to participate without stipends.  One person candidly admitted that they would not have participated without the stipend as they must prioritize paid work no matter how valuable the unpaid work might be. 

Espresso reviews

Fall 2024 Espresso review faculty provided short testimonials for the January 2025 San Diego Lilly Conference poster.  Faculty feedback at that time was 100% positive in terms of reaction or satisfaction (Level 1 in the Kirkpatrick framework), with strong evidence for learning and change (Levels 2 and 3).  

SQuAIR Faculty Fellow Maha Ghosn and CAFE team member Donya Rahimi reviewed results for selected QLT standards from 12 completed Espresso reviews to see if there were patterns to indicate common quality issues across multiple courses. The goal was to identify opportunities for on-point trainings and support beyond the grant period. Based on a very small sample, we tentatively conclude that an emphasis on clarity of learning outcomes will best serve.

In approximately May 2025, we implemented a survey for all faculty who had done Espresso reviews in 2023-2024 or 2024-2025. Of 27 faculty, we received 10 survey responses. Results were presented in detail at the June 2025 SQuAIR meeting. While faculty did not make many changes or significant changes to their courses in progress, most had intentions to make improvements for their next semester; we have not followed up to see if those intentions were implemented.

Satisfaction with the Espresso review experience was not as strong as faculty reported previously.  One faculty member had a very negative experience that likely resulted from a lack of clear communication about the nature and purpose of the review, as well as the need for CAFE reviewers to strengthen and maintain their expertise and to put ample time in to complete a review.  Generally, though, satisfaction was strong.  For future reviews, it would be very helpful to pair reviews of Canvas course sites with class visits for a fuller picture of the course.

Impact of QLT training on student outcomes

We have begun to assess the impact of extensively offered QM/QLT training in 2020 and 2021 on student grade outcomes.

Based on the knowledge that merely participating in one training had no effect on student outcomes, we limited analysis to those faculty who participated in two or three trainings. This resulted in a sample of 26 faculty (of 250 who participated in 2020 and 2021). Upon review, we further constrained analysis to faculty from only one college that represented 20 of the 26. We identified Spring 2022 courses taught by these faculty for which more than one section was offered. We added one faculty member from a different college who taught a course characterized by historically concerning DFW rates and several sections per semester.

We then pulled grade distributions for all faculty and all sections of the courses.  In two cases, the QLT/QM trained faculty member taught all sections of the course; we used their Spring 2021 grade distributions as a comparison. Using Microsoft CoPilot, we ran chi-square frequency analysis for several courses.

This figure presents results for one course that had the most sections available for analysis. While most student grades were promotable (i.e. C- or better) for each course section, clearly there are differences among instructors that are both important and statistically significant (p<0.005). In this case, Instructor 1 and Instructor 2 had engaged in two or three QM/QLT trainings each. Instructor 3 had engaged in a lot of previous professional development. 

Without knowing the specific situation of each class including factors such as grading schema and difficulty of course material and tests, we cannot conclusively state that professional development including extensive QM/QLT training positively impacts student outcomes.  But, this figure is encouraging.