Evaluating Instructor Strategy and Student Learning Through Digital Accessibility Course Enhancements

Claire Kearney-Volpe, Devorah Kletenik, Kate Sonka, Deborah Sturm, Amy Hurst

ABSTRACT

University students graduating and entering into technology design and development fields are underprepared to support digital accessibility due to a lack of awareness and training. Teach Access is a consortium of 10 industry partners, 5 advocacy groups, and 20 university partners working to address this issue. In an attempt to bridge the gap between what is taught to students and the increasing demand from industry, the initiative described here was aimed at awarding instructor grants to support the development of accessibility modules in tech-related courses. In our study we surveyed student attitudes toward accessibility pre- and post-instruction of these modules, as well as, instructor strategy.  We found that across all courses, student confidence in accessibility-related concepts increased.

The largest increases were found in student confidence in defining the Americans with Disabilities Act (ADA) and the Web Content Accessibility Guidelines (WCAG). Our work makes the following contributions: 1) A detailed description of how accessibility was integrated into 18 different university and college courses 2) Instructional delivery methods found to be effective by participating instructors 3) Insights for resource materials development.

Author Keywords Accessibility; Higher Education; ACM Classification Keywords •Human-Centered Computing →Accessibility; •Social and professional topics →Computing education;

INTRODUCTION

Technology is an integral part of participation in culture, education, and the workplace, and should be usable by everyone, regardless of ability. While there has been innovation and progress in a variety of applications, standards, and regulations – accessibility is still not systemic in the development of new and emerging technologies. Knowledge of accessible development is limited to a handful of domain experts, but to reach the goal of making technology accessible to everyone, it needs to become a mainstream skill. One of the greatest challenges to making accessible technology more ubiquitous is a lack of awareness and understanding of basic accessibility issues, concepts, and best practices. It is not uncommon that a company with thousands of technical positions (i.e. engineers, quality assurance, designers, product managers, etc.) employs only a few accessibility experts. According to a recent survey conducted by the Partnership on Employment and Accessible Technology, “60% [of tech companies] said it was difficult or very difficult for their organization to find job candidates with accessibility skills”. [[30]]

In response to the need for technology employees with a user-centered focus and understanding of accessibility best practices, the industry-higher education consortium Teach Access was formed. As part of an initiative to stimulate teaching and learning of digital accessibility best practices, and in line with research demonstrating the effectiveness of teaching accessibility-related modules [[24] , [27] , [30] ], the consortium implemented an incentivized program for instructors interested in teaching accessibility.

To address the lack of education around accessibility and to better understand issues in its design, we have addressed the following research questions:

  • R1. What learning objectives, instructional methods and resources do instructors use?
  • R2. What do instructors think supports learning?
  • R3. What resource materials and delivery methods support student learning?
  • R4. What learning objectives around accessibility influence outcomes?

In this paper, we describe related works teaching accessibility in computer science courses and surveying other university curricula. We provide an overview of the history and design of the Teach Access organization and programs, introduce the recipients of the 2018 Teach Access Curriculum Development Awards and the courses they taught. We then describe our methods to assess the impact of this program through student surveys administered before (and after) instruction in accessibility, and instructor surveys that were administered after the course. We present results from Likert-scale and open-ended feedback from students and instructors on their experience teaching accessibility course enhancements and reflect on lessons learned and recommendations for future efforts to teach accessibility.

RELATED WORK

A growing awareness of the importance of creating accessible technology for people with disabilities has led to efforts to educate the next generation of software developers in accessibility. A number of initiatives have been aimed at introducing accessibility to students,

both stressing the importance of and providing the skills necessary to create accessible content. In this section, we provide a brief discussion of the questions of “how, who, why and what” of teaching accessibility.

TRENDS IN TEACHING ACCESSIBILITY

Ko et al. suggest “three basic strategies for including accessibility and disability in courses: change a lecture, add a lecture, and add a new course.” [[17] ]. In the domain of changing and adding lectures, accessibility has been incorporated into courses about web design [[14] , [35] , [42] ], HCI [[18] , [21] , [22] , [26] 26, [31] ], design thinking [[36] , [37] ], software engineering [[24] ], introductory programming [[8] ], mobile app development [[10] ] and student capstone projects [[4] ,[11] ].

In the “add a course” domain, accessibility is also taught in stand-alone courses, including a course in adaptive technology [[23] ], assistive technologies and universal design [41], accessible computing [[6] ], usability and accessibility [[16] ], and accessibility and innovation [[27] ]. 

A general education course on accessibility for non-CS majors has also been created [[18] ]. Two MOOCs on accessibility have reached over 10,000 students [[13] ].

Some accessibility efforts have gone far beyond adding a single course. At University of Dundee, accessibility topics are incorporated into a number of courses, including programming, data structures and algorithms, and HCI courses, throughout the four-year curriculum [[41] ]. The Oslo and Akershus University College of Applied Sciences developed a Master’s Program in Universal Design of ICT [[7] ]. Bohman provides an in-depth analysis of the integration of accessibility into three university graduate programs in the US in his thesis [[3] ].

WHO TEACHES ACCESSIBILITY?

In assessing the current state of accessibility education, Putnam et al. conducted interviews with 18 instructors who teach accessibility. The interviews revealed common themes, such as the importance of teaching students to design for diverse audiences. The need to increase student empathy for people with disabilities was stressed, in particular through the use of simulations or interactions with people with disabilities, videos, field trips, or collaborative projects. Challenges reported included a lack of awareness of the importance of accessibility, lack of appropriate textbooks, and the difficulty of engaging students [[33] , [34] ].

A more wide-scale survey was conducted on 1,857 faculty members, of which 375 reported teaching about accessibility. An analysis of the survey responses indicated that instructors who teach accessibility are overwhelming likely to be female, experts in HCI, and have family members, friends or acquaintances with disabilities. Similar to the interviews of Putnam et al., the survey also revealed that the most critical barrier to teaching accessibility is the lack of instructor knowledge and resources about accessibility, and the need to have course-specific accessibility resources [[38] ].

ASSESSING ACCESSIBILITY EDUCATION

In educating students about accessibility, common goals are to increase students’ awareness of people with disabilities and knowledge of how to make technology more accessible. A number of assessments have focused on measuring how well accessibility interventions have accomplished those goals. For example, an analysis of student attitudes both before and after taking an HCI course that included an accessibility module showed an increase in student ratings of the importance of broadening the range of technology users [[32] ]. A series of studies conducted at Rochester Institute of Technology investigated the impact of various accessibility initiatives. In the first, end users with visual impairments served as external stakeholders for student projects. Students’ responses to exam questions indicated increased awareness of accessibility, compared to students the previous semester who had not worked with a person with disabilities [[24] ]. In the second, students participated in an HCI course that included a week of lectures about accessibility.

Differences in scores between pre- and post-instruction indicated increased awareness and (self-reported) knowledge of accessibility following the course [[29] ]. In the third, a combined approach was used in which all students in an HCI course received a week’s worth of lectures about accessibility; additionally, a subset of the students also worked on projects that included a person with disabilities as an external stakeholder.

The subset of students who had first-hand interactions with a person with disabilities was termed the Exposure group; the students who did not were the No Exposure. As in the previous study, pre- and post-instruction scores indicated increases in awareness and (self-reported) knowledge of accessibility for all students as a result of the lectures. Students in the Exposure group, however, also demonstrated increased levels of sympathy towards people with disabilities, as measured using the IDP scale (e.g. [[12] ]). No such increases were evident for the students in the No Exposure group. The authors point out that 1/7 of the student teams in the No Exposure groups (compared to 1/14 of the teams in the Exposure groups) acknowledged that their projects did not accommodate people with disabilities but chose not to address the problem; the authors conclude that accessibility knowledge alone “may not be enough to motivate students to address accessibility barriers” [[25] ].

ACCESSIBILITY LEARNING OUTCOMES

In the responses to the survey of Shinohara et al., the most common learning objective reported by instructors who teach accessibility was “Understand technology barriers faced by people with disabilities” [[38] ]. This dovetails with the discussion of Putnam et al. of the instructional goal of cultivating empathy for those with disabilities [[33] ]. However, a survey of 197 developers found gaps in developers’ knowledge of accessibility that suggested that there are areas “where developers struggle to empathize with accessibility issues and subsequently design interactions for this demographic,” as well as a “lack of understanding in how a person with disability uses technology” which “impacts on how technology interactions are designed” [[9] ]. A driving goal, therefore, of research in accessibility education is to find ways to best cultivate an understanding of the challenges and technology interactions of people with disabilities. One of the themes of the literature on accessibility education is the lack of instructor knowledge and resources. Clearly, many instructors seek guidance on what they should do to accomplish that goal. Existing resources for instructors include the AccessComputing [[5] , [17] ] Knowledge Base [[1] ], which includes case studies and practices for instructors.

An attempt to combat lack of instructor knowledge and resources was initiated by Kawas et al., who pioneered “micro” professional development: “a personalized, integrated, and low-commitment approach to teaching accessibility.”

The authors created mappings from specific CS learning objectives (in topics as diverse as computer vision, algorithms, introductory programming and networks) to accessibility learning objectives, and are working to create teaching materials that reinforce these mappings [[15] ]. Finally, an analysis of the educational literature about accessibility found that “the field lacks the pedagogic culture necessary to support widespread excellence in teaching and learning,” due to lack of a formal curriculum, an approach to teaching accessibility that is too narrowly focused and insufficiently inclusive to the diverse set of CS students, and a “lack of debate, investigation and evaluation regarding how accessibility is taught and learned,” with “insights in this field [that] tend to be based in individual accounts more than detailed pedagogic research” [[20] ]. We call for accessibility advocates to build a pedagogic culture by developing resource materials and instructional methods that are evidence-informed. In this paper, we provide evidence that instructional methods and resource materials contribute to student learning outcomes in digital accessibility. Based on these findings, we make design recommendations for materials and instructional methods.

TEACH ACCESS BACKGROUND

In April 2016, the Teach Access kickoff meeting took place at Yahoo! and brought together more than 40 individuals from leading tech companies and universities. The focus of that meeting was to establish goals and tangible projects to develop ways to engage students in fields such as design, computer sciences, and human-computer interaction to be better prepared to enter the workforce and create future technologies that are truly inclusive. On the industry side, this included directors, developers, and project managers of accessibility teams in companies such as Yahoo!, Facebook, Google, Microsoft, Adobe, among others. University attendees included faculty, academic support staff, and other who work with accessibility/disability on their respective campuses. Through two days of discussions and brainstorming it became clear that in order to address the need of industry to be able to hire recent graduates with basic technical accessibility knowledge, a true collaboration would need to emerge between industry and academia. In order to establish the means to create this sustained collaboration, six distinct task forces emerged out of that initial planning meeting that were each comprised of industry and academic members. Over the past three years, several programs and projects have come out of the work of these task forces. First, the Teach Access Tutorial is a set of best practices for making mobile apps and websites accessible. The Evidence Packet that can be circulated among higher levels of leadership at universities to support the need to infuse accessibility into curricula. Work with organizations like the Accreditation Board for Engineering and Technology (ABET), the Association for Computing Machinery (ACM), and the National Association of Schools of Art and Design (NASAD) to include in their accreditation language an emphasis on the need to understand accessibility basics. And finally, the Teach Access Study Away: Silicon Valley program that brings up to 30 undergraduate students and faculty from a variety of universities to Silicon Valley for an immersive week of study where students learn about the accessibility landscape at several Teach Access technology companies. Feedback from faculty in the kickoff meeting showed that many wanted to teach the basics of accessibility, but more curricular materials were needed to be able to do this. While industry could provide guidance on the types of skills they look for, they did not have the background in pedagogy and an understanding in how to create university curriculal.

As such, in 2018 the Teach Access Curriculum Development Awards were established to foster innovative ways of incorporating accessibility into existing courses.

Funding was secured from two large private foundation donors to award up to 20 instructors with $5,000 stipends to develop curricular material to teach accessibility, such as modules, presentations, exercises, or curriculum enhancements or changes that introduce the fundamental concepts and skills of accessibility design and development into existing courses. “Instructor” was broadly defined to include full-time, part-time, adjunct, or instructional staff at US-based institutions of higher education as a way to attract a variety of applicants. The inaugural round of funded proposals spanned the 2018-19 academic year. Twenty-nine instructors applied for twenty available awards. The seven-member Teach Access selection committee assessed each proposal on the following criteria: • Relevance of the course: How relevant to the goals of Teach Access is the proposed course, based on the provided course title and course info? • Relevance of the applicant’s background/expertise: How relevant is the applicant’s background and subject matter to the goal of these grants? • Likelihood of success based on previous experience: Regardless of how experienced the applicant is with the subject matter, does his/her statement regarding previous experience indicate a likelihood of success? • Strength of internal promotion plan: Is the applicant’s institutional promotion plan likely to expand the knowledge and understanding of accessibility in the applicant’s institution? • Strength of external dissemination plan: How strong is the applicant’s external dissemination plan? • Understanding of the goals of the grant (from written proposal): Does the applicant’s written proposal demonstrate an understanding of the goals of the grant? • Likelihood of expanding knowledge (from written proposal): Does the proposal show a likelihood of expanding students’ knowledge of accessibility in a substantive and sustainable manner? Conditions of the award acceptance included 1) Instructors sharing their course materials that would be posted the Teach Access website, 2) Evidence that instructors presented their new course components to other instructors and administrators at their institution, 3) Administering and sharing results from pre- and post-instruction surveys of students to measure the impact of the curriculum, and 4) Write a summary report about their experiences teaching and developing this content. Through the Teach Access Curriculum Development Awards, several hundred undergraduates across the United States have been exposed to digital accessibility training in Computing, Art, Psych and Education Departments. In the rest of this paper we investigate the impact of our efforts.

METHODOLOGY TO EVALUATE IMPACT OF 2018-2019 COHORT

This paper reports on an effort to assess accessibility instruction strategies in design, web, computer science, education, and human computer interaction courses during the fall 2018 and spring 2019 semesters. 

Teach Access surveyed student attitudes toward accessibility before and after accessibility-related course enhancements and granted us this data. We then gathered instructor feedback about their delivery methods, learning objectives, and resources used.

PARTICIPANTS

The first included approximately 400 students enrolled in either undergraduate level (21 courses) or graduate level (9 courses) courses that were taught by the awardees in the second group. Many of these courses were hybrid allowing both graduate and undergraduate students to enroll, and we do not have exact enrollment numbers. The second group included 12 instructors recruited via email from a mailing list of 19 Curriculum Development Awardees.

The 12 instructors are a mix of Computing and Non-Computing (Art, Psychology, and Education) department professors teaching CS or related courses (including one project that funded instructor training) in 16 different colleges from across the United States. A list of the courses included in this work is provided in Table 1. All participation in this study was voluntary; students and instructors were not compensated.

Any Questions?

(234) 301-3513

1234 Divi St. #1000, San Francisco, CA 95123

info@divitherapy.com

Skip to content