I’ve acquired GMod recently, mainly to make it accessible for the blind and visually impaired.
This is a quite huge undertaking, and after hours of browsing the official wiki (http://wiki.garrysmod.com), I am really not sure where to start.
Here are the features I’d like to implement. I’d be glad for any pointers, or comments regarding a possible way of implementing them!
Please NOTE: I do not necessarily require code, rather engine functions, or ideas, if it is not possible to realise my original idea. As I know almost nothing about the engine, specific custom or engine functions are really appreciated!
Textual information, where possible, would be sent to a Screen Reader. The simplest solution for this now is to send text to the clipboard. I’m probably going to use Global.SetClipboardText for this.
Extracting game text does not seem to be hard, the console has for example hooks that return the printed text, fire on activate/deactivate events, etc. My main issue here is the UI. I couldn’t find any hook/library function that’d allow me to determine which menu item received focus, and then to get the control’s caption and current value, where applicable (i.e. for a checkbox, combobox, etc).
Furthermore, due to the blind mainly using a keyboard, or a gamepad, some menus are impossible to use, such as the main menu.
At this point, I am not entirely sure whether I should write a custom UI, or just forget about menus and dialogs altogether, since the console can be used to set most, if not all the game options.
If I can replicate menus with ease, just with the added speech/extra sounds/keyboard navigation, chances are high that it’d be a lot easier to use menus rather than typing in commands.
Most items on a map would need a looping sound, so that they could be located easily. This would be especially crucial for pickable items and interactables. Of course this would be hugely dependant on the mod creator, but as long as the built-in ones emit sound, constructing something that is accessible would not be a problem.
I’ve thought of scanning the map around the player, and playing a sound where I found an object. Does EmitSound work for items, or just for entities?
Most of the time, multi-level maps are quite problematic, since you have to look up/down to aim. This is unfortunately quite hard to illustrate using audio only, even if HRTF is available. Thus I think that either AutoAim, or using flat maps while locking the mouse’s Y axis would work. Providing key bindings for LookUp and LookDown seems to be doable as well.
I think there is a hook so I can ForceCenterView when the mouse moves up/down, not sure about autoaim though.
Construction. As GMod is mostly about in-game construction, I am really not sure how to approach this. Perhaps having a mode where the player can scan ahead, and would be read out the name of an object, or switch between possible targets…
Of course any modifications I make would only be useful for the blind or VI, including multiplayer gaming, thus in this case it does not really matter what’s on the screen (i.e. a minigun weapon model while someone’s using a custom-built pistol). As long as the audio signals that it is indeed a pistol, and it does not do 10k damage per second, it is fine!
Thank you very much in advance!