Facebook has long utilized AI to enhance photo descriptions for visually impaired users, but in 2021, the platform is significantly enhancing these capabilities. The company has introduced an updated version of automatic alternative text (AAT) that offers richer and more informative descriptions.
Moving away from heavily supervised AI learning, Facebook is now employing weak supervision, analyzing “billions” of Instagram photos and hashtags. This methodology expands its descriptive capacity from just 100 concepts to over 1,200, encompassing a wide range of subjects, including diverse cuisines and various cultural events, such as weddings that do not feature traditional white dresses.
Moreover, a new object detection system accurately identifies the position and number of people in each photo. Users can now opt for detailed descriptions that highlight the size, position, and nature of objects within the image, in addition to the usual summaries.
While Facebook acknowledges that the descriptions may still contain inaccuracies, such as using "may be" and omitting concepts it cannot confidently identify, this advancement marks a significant improvement in accessibility. It aims to provide visually impaired users with a clearer understanding of the images shared on their feeds.