Godot doesn't work with screen readers #58074
Comments
|
Tagging @ndarilek who was working on making Godot accessible to visual impaired (IIRC he is also visual impaired) |
|
There's https://github.com/lightsoutgames/godot-accessibility, but it's a custom TTS based screen reader, and AFAIK it's only for Godot projects not the editor. I'm currently working on system screen reader support for |
|
godot-accessibility works with the Godot UI full stop, that means both the editor and games using it. But I stopped work on it because ultimately it was too hacky to maintain, and I didn't get a whole lot of help. So I'm not sure it has a future. For screen reader support, I'd recommend integrating https://accesskit.dev, possibly as a separate plugin. AccessKit will ultimately work on every platform, including the web. It's Rust-based, so it'll either need in-tree support for Rust in the compilation process, or support for 4.0's GDNative replacement in the Rust bindings. But platform-native accessibility support is complicated enough that both approaches are preferable to hand-coding each platform separately IMO. |
I have seen this library, and it would be nice to use it, but unfortunately it seems like currently it's only implemented for Windows, support for the other platforms is planned. Right now, I'm looking at porting part of the Chromium source (BSD licensed) as a base for the platform specific implementations. |
|
It is Windows-only, but it'll be a lot easier to support other platforms
than it'd be to reinvent AccessKit for Godot specifically. Also, I
suspect you'll get much less accessibility interest on mac than you
would under Windows because macOS has a much smaller share of blind
users. Finally, using AccessKit directly gets you @mwcampbell's decades
of experience as a screen reader developer and access tech professional,
which is going to be hard to match unless you've got similar experience
on the accessibility stack (spoiler alert: I've done
accessibility-related development for a while and *I* don't have that.)
I don't mean to be critical, but doing this mac-only and from scratch
probably isn't a good idea. It's going to think the target audience
right from the start, and I'm afraid it will lead to a negative cycle of
"we did all this effort, but there's been little to no interest to
justify it continuing." OTOH, I know of several blind developers
interested in using something like Godot were it available and
accessible under Windows.
Thanks.
|
My current macOS implementation is just an experiment to assess required changes to the engine, not a plan for the actual thing. So far, I'm just investigating options, and implementations in the other software like Chromium and GUI toolkits. It's always better to know what's going on under the hood before trying to integrate something.
In any case, nothing is decided yet. Probably a generic interface for a plug-ins (both external and built-it) providing access to the node tree updates / dispatching actions, and modification of the engine nodes is the first step. And what's going to be on its other side is another question. |
|
I hope to start working on AccessKit in earnest again in a few weeks. I'd call it alpha right now. If all goes according to plan, the Windows implementation should be beta-quality in a few months, and then I'll start on the Mac implementation. Other platforms will probably have to wait a while, unless other folks start contributing once the design is more locked down. I don't know when I'll work on bindings for languages other than Rust. |
|
Let me just add that accessibility is very hard, and I think that making
a Godot-specific implementation to cross-platform a11y APIs is a big
mistake. Godot, and my work on godot-accessibility, at least partially
inspired AccessKit. While I'm thrilled that Godot may become more
accessible by default, it saddens me to see Godot as a project turn away
from one of the projects it indirectly inspired.
Admittedly the use of Rust poses some challenges. If folks from the
Godot side want to put in some effort toward improving accessibility,
that effort seems better spent implementing a C/C++-compatible FFI for
AccessKit. Building a workable FFI wrapper requires some skill, sure,
but it's a more general skill vs. trying to work around the particulars
of a given accessibility API, whose footguns you'll only know about if
you've spent a while in those particular trenches. And turning away from
a project just because it hasn't had any commits in a couple months, in
favor of a bespoke implementation of that project's same functionality,
seems to trivialize how hard this problem actually is. That's not me
speaking from anger. That's me, as an accessibility expert, giving my
honest assessment on how well this path will likely end up working out
for Godot.
I'm willing to volunteer time to help integrating AccessKit, because
that's a viable path forward and a significant force multiplier. I won't
help with a Godot-specific solution, though, since that'll probably be
very limited and tough to keep updated.
|
Judging by previous attempts to bring Rust support to the Godot, I would say it's highly unlikely that Rust library will be accepted directly into the core. But as a GDExtension it definitely possible, ideally we would use universal Rust bindings (https://github.com/godot-rust/godot-rust), but currently it's not compatible with new GDExtension API. So I guess a reasonable approach would be:
To summarize what we need from the engine itself (correct me if I'm missing something):
I have no intentions to insist on a platform specific implementation. So far, there's a general notion that we should have screen-reader support. I'm willing to spent my time on it and exploring the options (and I'm definitely not an accessibility expert).
In case of dealing with unfinished libraries, I prefer to have as much wiggle room as possible, but it does not mean turning away from it. Also, it's probably better to discuss implementation details in the godotengine/godot-proposals#983, or Contributors Chat instead of doing it this issue. |
|
Here are some updates on AccessKit: A basic macOS adapter is merged. So now two platforms are covered. A Linux adapter is under development. We haven't gotten to mobile platforms or web yet. The Windows adapter supports text editing. This feature in the macOS adapter is currently waiting for review. The Windows and macOS adapters both have the dynamic subclassing hacks that are needed to inject AccessKit support into a window that is owned by some other code. This would be necessary for implementing Godot accessibility in a plugin. AccessKit's own adapter for the Rust winit crate uses this feature. We have a proof-of-concept Unity integration. The link is to an existing open-source Unity-based game that we modified to be accessible using AccessKit. No C or C++ API yet. |
Rust bindings for GDExtension seems to be in active development (unusable at a moment, but it's probably a temporary breakage after the recent GDExtension API changes). With the support for dynamic subclassing, it should be possible to make a plugin entirely in Rust, without any changes to the engine (if subclassing can be done at any moment, even after window is shown). |
|
To be reliable on Windows, subclassing must be done after the window is created but before it is shown. Trying to do it after the window is shown has been the biggest problem with our Unity plugin. |
I have opened PR to add ability to plug in early in the window lifespan (immediately after creation and before destruction) - #72886 (won't make it to 4.0 since Godot is in pre-release feature freeze).
Seems like it's using only 3 C-API function and JSON for the tree update. So with the aforementioned PR, it should be possible to use the same plugin and simple GDExtension wrapper to get windows subclassed in time and generate tree updates from the GDScript. GDExtension wrapper draft - click to expand#include <godot_cpp/core/class_db.hpp>
#include <godot_cpp/core/binder_common.hpp>
#include <godot_cpp/classes/global_constants.hpp>
#include <godot_cpp/classes/display_server.hpp>
#include <godot_cpp/classes/display_server_extension.hpp>
#include <godot_cpp/classes/display_server_extension_manager.hpp>
#include <godot_cpp/variant/utility_functions.hpp>
using namespace godot;
class ExampleAccessKit : public DisplayServerExtension {
GDCLASS(ExampleAccessKit, DisplayServerExtension);
protected:
static void _bind_methods() {
ClassDB::bind_method(D_METHOD("update_tree", "window_id", "update"), &ExampleAccessKit::update_tree);
}
public:
virtual String _get_name() const override {
return String("AccessKit");
}
virtual void _create_window(int32_t p_window_id, int64_t p_native_window_handle) override {
printf("!!!! creating window %d handle %lld\n", p_window_id, p_native_window_handle);
// TODO: call init((HWND)p_native_window_handle);
}
virtual void _destroy_window(int32_t p_window_id, int64_t p_native_window_handle) override {
printf("!!!! closing window %d handle %lld\n", p_window_id, p_native_window_handle);
// TODO: call destroy((HWND)p_native_window_handle);
}
void update_tree(int32_t p_window_id, const String &p_update) {
int64_t native_window_handle = DisplayServer::get_singleton()->window_get_native_handle(DisplayServer::WINDOW_HANDLE, p_window_id);
printf("!!!! update window %d handle %lld\n", p_window_id, native_window_handle);
// TODO: call push_update((HWND)native_window_handle, p_update.utf8().get_data());
}
ExampleAccessKit() {
// TODO: Load accesskit_unity_plugin.dll (or link it statically).
}
~ExampleAccessKit() {
// TODO: Unload accesskit_unity_plugin.dll.
}
};
void initialize_example_module(ModuleInitializationLevel p_level) {
if (p_level != MODULE_INITIALIZATION_LEVEL_SERVERS) {
return;
}
GDREGISTER_CLASS(ExampleAccessKit);
DisplayServerExtensionManager *dseman = DisplayServerExtensionManager::get_singleton();
if (dseman) {
Ref<ExampleAccessKit> de;
de.instantiate();
dseman->add_interface(de);
}
}
void uninitialize_example_module(ModuleInitializationLevel p_level) {
if (p_level != MODULE_INITIALIZATION_LEVEL_SERVERS) {
return;
}
}
extern "C" GDExtensionBool GDE_EXPORT example_library_init(const GDExtensionInterface *p_interface, GDExtensionClassLibraryPtr p_library, GDExtensionInitialization *r_initialization) {
godot::GDExtensionBinding::InitObject init_obj(p_interface, p_library, r_initialization);
init_obj.register_initializer(initialize_example_module);
init_obj.register_terminator(uninitialize_example_module);
init_obj.set_minimum_library_initialization_level(MODULE_INITIALIZATION_LEVEL_SERVERS);
return init_obj.init();
}extends Node
func _ready():
var iface = DisplayServerExtensionManager.find_interface("AccessKit")
iface.update_tree(DisplayServer.MAIN_WINDOW_ID, "json_data_goes_here")I'll try to test it and make a demo a bit later. |
Here's an early draft (based on a proof-of-concept Unity integration, So far I have only tested it with Window 11 Narrator, it's correctly detecting window title, getting tree update from the script, and following focus switching between two buttons. https://github.com/bruvzg/godot_accesskit_demo, depends on #72886 Demo build steps
|
|
I know this wasn't the intent of the original issue, but I just want to point out that Godot 4's TTS functions don't work with screen readers, either. Ideally, they should, instead of using custom text-to-speech generation, because the user experience for blind players is much, much better when using a mature screen reader like NVDA. |
|
An alternate to adding screen reader TTS would be making this plugin support live regions, then writing text to speak to those so the screen reader would speak it automatically, optionally interrupting depending on the assertiveness setting of the region. You can probably get a lot of mileage out of Godot's metadata system here. Since you can associate arbitrary typed data with keys, it might be possible to mirror properties prefixed with something like "accessibility_" over to their AccessKit equivalents directly from the UI nodes they represent. I guess this would have the down side of making them opaque and undocumented from the editor, but it might not be bad for a V1. Really wish I didn't have a pile of things to do for my existing game. Maybe I'll find time to contribute to this sometime soon, would really be neat to have Godot as an option again but godot-accessibility was hacks upon hacks upon hacks. |
While it is related, generic TTS and screen reader support are completely different APIs. TTS by itself probably should be considered as a generic output option, not an accessibility feature.
There's nothing custom, current TTS implementation is using default system APIs (when there is a default system API, which in case of Linux is questionable, since |
This comment was marked as outdated.
This comment was marked as outdated.
|
An update on https://github.com/bruvzg/godot_accesskit_demo Now it is using latest AccessKit release (0.10.0) and working on Windows (tested with Accessibility Insights, Windows 11 Narrator and NVDA), macOS (tested with VoiceOver and Xcode Accessibility Inspector) and Linux (tested with Orca and AT-SPI Browser).
Note: seems like it's reliable only if subclassing is done from |
Is the |
Since using |
Agreed. Glad you're able to do that. I assume you're still using JSON serialization, like the Unity plugin. Do you think your Godot work would benefit from having an AccessKit C API that lets you directly create nodes and tree updates without going through JSON? We've started working on such an API, but I don't have an ETA yet. |
Yes, it's using JSON. At this point. I have not done any real serialization. It's just a few manually entered JSONs in the Godot project scripts.
C API would be more convenient. |
|
While I have nothing constructive/helpful to add, I did just want to say, as a legally blind developer I greatly appreciate that this is an issue being explored 💙 For me, a screenreader is a luxury that I can get by without, but I know not everyone has that luxury. I also know folks with dyslexia who rely on them, so I know it's not just us blind folks who appreciate your hard work. Thanks so much! |
Godot version
Godot_v3.4.2-stable_win64
System information
Windows10
Issue description
Godot doesn't work with screen readers I'm using windows10 with NVDA screen reader, Godot like a blank page i can't read any thing also if i use the mouse i have the same result.can you add screen reader support to help us blind people to create a games
Steps to reproduce
Godot doesn't work with screen readers I'm using windows10 with NVDA screen reader, Godot like a blank page i can't read any thing also if i use the mouse i have the same result.can you add screen reader support to help us blind people to create a games
Minimal reproduction project
Godot doesn't work with screen readers I'm using windows10 with NVDA screen reader, Godot like a blank page i can't read any thing also if i use the mouse i have the same result.can you add screen reader support to help us blind people to create a games
The text was updated successfully, but these errors were encountered: