Introduction

Software developers who are blind or visually impaired find it difficult to use productivity-enhancing IDEs, like Visual Studio or Visual Studio Code. CodeTalk is a Visual Studio extension that was previously released to help address these problems by providing accessibility improvements when writing and debugging source code. A core challenge when developing while blind or visually impaired is that these development environments often have highly-complex UI that presents large volumes of related content and this content is often dynamically updated to convey application state. Source code content is structurally interrelated, creating significant impediments to efficient navigation, and dynamic updates can either show continuously updated state or provide context that a user action is required.

VSCodeTalk offers a set of features that address the following categories of usage scenarios i) glanceability, ii) navigability, and iii) real-time alertability. Glanceability is the ability to quickly look at a source code file and ascertain information about the overall code structure. Navigability is the ability to quickly move around the code to the desired code segment. Real-time alertability is the ability to receive notifications about the application state at the desired time. The VSCodeTalk features implemented in this project help improve these use-cases.

Please watch the following video for additional information about VSCodeTalk, including a demo of some of the most useful features.

The VSCodeTalk project was inspired by the CodeTalk project that is documented at https://www.microsoft.com/en-us/research/project/codetalk/. That project includes a Visual Studio extension that implements many of the ideas discussed in the accompanying research paper https://www.microsoft.com/en-us/research/uploads/prod/2018/04/CodeTalkCHI2018CameraReady.pdf.

The issues that people with visual impairments encounter when using software development environments are of interest to software tools vendors. The Visual Studio Code team has implemented several concepts similar to VSCodeTalk features that help improve the default product's accessibility. One example is the Code Outline feature that overlaps with the Function List and Code Summary features in some regards. Another example is the debugger which has built-in voice notifications for several breakpoint events, such as narrating the words 'stopped' or 'continue'.

Solution

VSCodeTalk project helps improve several accessibility use-cases for visually impaired users. There is a Visual Studio Code extension available that can be manually installed from a VSIX extension file. The source code and binary files for this extension are available from the following links.

The accessibility challenges of i) glanceability, ii) navigability, and iii) real-time alertability have been addressed through the following VSCodeTalk features.

The VSCodeTalk extension can be installed in Visual Studio Code through the Extension Manager. The extension details for all implemented commands, settings, and views are viewable in VS Code as shown below.

The VSCodeTalk extension readme page

Validation

Concept Validation

The CodeTalk feature set was based on research that involved close collaboration with the BLV community. VSCodeTalk is largely a direct port of these features so we have some validation that the underlying concepts are sound. A significant aspect of the validation that we did was to ensure that features were implemented correctly and that the user experience passed accessibility testing scenarios.

Screen Reader Compatibility Validation

The accessibility testings procedures focused on screen-reader audio output and keyboard-only navigation scenarios. During screen-reader validation, we would systematically go through each feature with the NVDA software enabled. The spoken text would be compared with the displayed content to confirm that the output is correct and that sufficient text was read to provide source code context.

Keyboard-only Navigation Validation

The keyboard navigation validation scenarios were mostly focused on ensuring the cursor focus could easily be moved back-and-forth between the editor and the VSCodeTalk feature view windows. The content in the views, such as the current context or the function list, has keyboard command handlers to refocus back to the editor on the ENTER keypress.

Language Support Validation

We tried our extension with code written in multiple languages including:

This helped ensure that our extension was not coupled to a single language, but could be language-agnostic.

User Validation

We met with Venkatesh during office hours to walk through the extension that was built and gathered feedback.
We also shared an early private build of the extension for validation. Feedback included:

Learnings and Future Work

There were many learning opportunities while implementing the VSCodeTalk project. A major learning came from trying to understand the software development process for people for visual impairments. Software development tools display a large amount of information simultaneously. And there is a lot of context that must be maintained about the current cursor position in order to understand what modifications are required in a source file. While implementing and testing the VSCodeTalk feature set, we tried to mimic these user workflows. This exercise was a great way to build a better understanding of some of the challenges for BLV users when using software development tools.

The next steps for the VSCodeTalk project is to try to merge the Visual Studio Code extension source code upstream to the https://github.com/Microsoft/CodeTalk repo. We would then like to collaborate with the CodeTalk project maintainers to help transfer the extension source code to them for a potential release to the Visual Studio Marketplace.

An additional investigation was conducted into how to make the VS Code built-in Code Outline feature more accessible. The Code Outline feature contains much of the same data as the Function List, Code Summary, and Current Context features. Code Outline's user experience is not optimized for accessible use-cases around screen-reader spoken word quality and keyboard-only quick navigation through source code symbols. We looked at various was to reorganize the information in Code Outline to better match the CodeTalk data presentations. The screen-reader user interface metadata was also improved to narrate the symbol datatype as well as the line numbers when describing the symbol. We plan to open a issue on the VS Code repository with suggestions for improvements referencing a PR with our experimental changes.

Other feedback for VSCode: