One of the most exciting developments seen in the video game accessibility space over the past few years has been the increase in both big budget and indie games alike aiming to incorporate audio descriptions into their narratives.
From small games like the point and click adventure title Brok the InvestiGator, to big sprawling epics like Spider-Man 2, more and more often we’re seeing video games attempting to be more accessible to partially sighted and sightless blind players by describing on screen visual elements as narrated audio.
However, not all audio descriptions in video games are created equal, and there is currently a pretty notable divide in the way that video games are approaching audio descriptions.
While some video games like Stories of Blossom are aiming to feature full audio descriptions during gameplay, others are focusing on what is being dubbed “cinematic audio descriptions”, a term meant to specify that audio descriptions will be present ONLY during static cutscenes where control has been taken away from the player, and not during gameplay or in engine scenes.
It’s no mystery why this divide exists. Most of the well known examples out there of games that feature full audio descriptions covering gameplay scenes are games where locations and interactions are deliberately limited.
Stories of Blossom and Brok the InvestiGator, the two positive examples I brought up previously, are both side scrolling point and click adventure games. Both games feature distinct background portraits for locations, a set number of interaction points the player can click on, and dialogue built out of segments which can be naturally broken up to make space for audio descriptions. These elements do a lot to make it easier for a developer to incorporate audio descriptions into gameplay – the number of ways a player might be interacting with the game at any one time is finite and pacing is malleable.
By comparison, if we look at a game like The Last of Us: Part 2 Remastered, it’s easy to see why audio descriptions would be more of a challenge to incorporate into gameplay. While cutscenes have predictable pacing, during gameplay the player is set loose in open 3D environments, with a lot more ability to look around in varied directions. Third person shooter sequences, for example, move a lot more quickly and erratically than point and click adventure game interactions, and it would be more difficult to quickly describe gameplay interactions in real time before those gameplay interactions have changed. There is an additional challenge to overcome in providing audio descriptions in fast paced 3D action games outside of scripted cutscene encounters.
While I acknowledge that audio descriptions are more difficult to incorporate into these kinds of games during gameplay sequences, I don’t think the additional challenge involved should be used as an excuse not to push for things to improve. I think a lot of these big budget games already have mechanisms in place which could be used as hooks for narrative audio descriptions during gameplay, and would greatly benefit from making an effort to experiment in this space.
Let’s continue using The Last of Us: Part 2 as an example. During a playthrough of that game, which is completable without sighted assistance due to a series of audio cues and navigation assists, there are several moments where conversations with characters are triggered by walking over invisible trigger points in the world. These locations are funnels in the world which the player is required to cross in order to progress through linear environments, and cause the game to play dialogue while characters walk together. These locations are selected to ensure that combat won’t break out during the length of the conversation, and allow additional information to be conveyed while traversing the world.
It’s easy to imagine gameplay audio descriptions working using a very similar system, with descriptions of what a new environment looks like triggered upon entering a new area. At present, a player in that game could be swimming through deep water in a flooded subway tunnel for example and have no idea because there was not an audio described cutscene to provide that information. A simple conversation style trigger at the entrance to that tunnel causing an audio description line to play describing the new location would go a long way to conveying the tone of a setpiece.
Additionally, while providing moment to moment audio descriptions for combat may be difficult, and could potentially detract from audio cues that are important for combat gameplay, it wouldn’t be too complicated to place a similar trigger near the start of a combat sequence to describe visual information about the enemies who are about to be fought. Something simple like “a group of armed militants is on patrol, they have not seen you, and they have a dog who is trying to follow a scent” would provide context to a combat encounter not currently being provided.
By comparison, looking at a game like Spider-Man 2, that game currently features a phonecall system where the player sometimes receives calls from other characters to discuss plot while swinging around the city. A system like this could be used to trigger audio descriptions relevant to the changing state of the game’s city, such as whether buildings are currently half buried in sand, or if the weather has changed, or the current time of day.
Spider-Man 2 also features a system for scanning the environment around the player to locate nearby areas of interest and interaction highlights. This system could perhaps be adapted to trigger a text to speech readout of nearby interactions, and which directions they’re located in, as well as a general description of whether the player is high up at the top of a building, vs in a park, or near a harbour. These wouldn’t need to be hugely specific to be useful, and could go a long way to providing additional context to players.
There’s one game out there today that I do think does a pretty good job of showcasing what this kind of gameplay implementation of audio descriptions could look like in practice, even if not necessarilly intentionally: Forza Motorsport 2023. While that game’s actual audio descriptions are generally limited to descriptions of pre-race cinematics, one of the game’s blind driving assist options reads out upcoming turns on the racetrack as they’re approached in terms of length, duration, direction, and intensity. These descriptions are triggered as you pass certain invisible markers on a linear track, and when activated give information of the layout of the upcoming turn.
In practice, this is the kind of system that I think could work.
While Forza as a game likely doesn’t have enough time to provide track visual descriptions in real time, mid race, alongside navigation information, other games such as The Last of Us: Part 2 or Spider-Man 2 probably do have time to use this kind of blueprint to describe elements of the environment as a player explores.
I recognise that describing every possible interaction in these kinds of games with audio descriptions is tricky, but these are examples of the kinds of ways audio description could start to be added, little by little, into gameplay sequences in titles that are currently only providing descriptions for their cinematic sequences.
I certainly recognise that this would require an amount of work from developers to implement, and that there’s no great example currently out there to point to for what an ideal implementation of this would look like. This would require a developer to invest money in trying to create something new and untested, and that is not an easy sell. Still, I think it is an important step forward that audio descriptions in games are going to need to eventually take.
Audio Descriptions in games are a fantastic step forward for the industry, and I’m glad we’re seeing big budget games trying to move forward into this space. Even if a game like Spider-Man 2 only currently provides audio descriptions for cinematics, that’s still a huge step forward in terms of making that game more accessible to more people as a complete narrative. I’m not trying to take away from that accomplishment when I say that audio descriptions in games could be better than they are currently, I want to see things improve because I think an already appreciated feature is on the precipice of being able to help support even more players than it already does.
Right now, gameplay specific audio descriptions are mostly the domain of very specific game genres, but I believe that once we see one big budget 3D action game take the plunge into trying to provide audio descriptions in gameplay rather than just cutscenes, it’ll really open up the door to a new era in video game accessibility efforts.