A tool to create simple language descriptions for museum art.

Museums often provide descriptions of their artworks and exhibits that aim to educate visitors about the background or significance of a piece. However, lengthy or dense descriptions may create challenges for casual visitors or those with cognitive impairments. Our project, MochaSimplifier, aims to language descriptions for museum art in order to alleviate such challenges and make museum content more accessible to all audiences. MochaSimplifier is a web-based platform that allows users to upload an image of a museum text description and generate simpler, more easily understandable versions.

Needfinding

Needfinding is the process of finding gaps in existing systems for a specific audience. This prototype aims to address barriers in museums for individuals with cognitive disabilities. We read the following papers in order to understand their needs:

• Moreno, L., Petrie, H., Martínez, P. et al. Designing user interfaces for content simplification aimed at people with cognitive impairments.Univ Access Inf Soc 23, 99–117 (2024).
• "Simplifying Content for People with Cognitive Disabilities."YouTube, uploaded by IBM Research, Sep 18, 2017.
• Uziel-Karl, Sigal, and Tenne-Rinde, Michal. “Making language accessible for people with cognitive disabilities: Intellectual disability as a test case.” In Handbook of Communication Disorders: Theoretical, Empirical, and Applied Linguistic Perspectives, edited by Elitzur H.G. Dattner and Dorit Ravid, pp. 845–860. Berlin/Boston: De Gruyter Mouton, 2018.
• Hu, Ruimin and Feng, Jinjuan Heidi. "Investigating Information Search by People with Cognitive Disabilities." ACM Transactions on Accessible Computing 7, no. 1 (2015): Article 1, 30 pages.
• Ravelli, Louise J. "Making language accessible: Successful text writing for museum visitors." Linguistics and Education 8, no. 4 (1996): 367–387.
• Garcia Carrizosa, H., Sheehy, K., Rix, J., Seale, J., and Hayhoe, S. "Designing technologies for museums: accessibility and participation issues." Journal of Enabling Technologies 14, no. 1 (2020): 31–39.
• Soares Guedes, Leandro, and Landoni, Monica (Supervisor). PhD Thesis. Università della Svizzera italiana, 2024.
• Soares Guedes, Leandro; Zanardi, Irene; Mastrogiuseppe, Marilina. "Proceedings of the European Conference on Cognitive Ergonomics 2023." ECCE ’23, pp. 1–8.

In our affinity diagram, we organized these needs into six broad categories:

1. Content transformation: Simplify and shorten museum art descriptions, provide context
2. Visual design: Consistent and easy navigation and design
3. Multimedia and alternative modalities: no scrolling, minimize distractions, minimal colors and high contrast, clear functionality, clearly labeled icons, text-to-speech playback
4. Independence: Quickly get definitions for words, customize reading level, frequent use of headings and prompts for guidance
5. Scaffolding: Tutorial, break tasks down into manageable steps
6. Interactions: Links change color when clicked, quickly get definitions for words on hover/tap

Sketching and Storyboarding

Based on our affinity diagram, each member of our team drafted an initial sketch of core features in our application.

Based on our sketches, we created a final storyboard for our prototype:

Prototyping

We created a prototype of our application located on the homepage of this website. Users can input an image, customize the reading level, and view a simplified summary of the text in the uploaded image.