UX Audit using AI data analysis - 2024
Client
Telfaz Dropp
Project
Redesign
Team
3 Designers
Created
2024
My Role
Principal Designer
Duration
3 weeks
Telfaz Dropp wanted us to evaluate Cultural Universe (https://cup.moc.gov.sa) an interactive 3D environment designed for VR / Web / Mobile platforms.
Stakeholder feels there are many problems while interacting with the VR web desktop environment.
Due to the absence of real users, we proposed to conduct heuristics evaluation.
use AI to reduce time
Notion
Used Notion to capture all observations. Its an ideal tool to classify using standard terminologies accross all reviewers.
Captured Databas
Notion helped us to sort the data using more than one criteria (High to low - Desending order)
Sort by "Severity" to prioritise which problems to solve first
Sort by "Design Aspect" most affected area of design aspect
The sources highlight several navigation issues within the Cultural Universe web application:
Lack of Breadcrumb Trail: The absence of a breadcrumb trail in the Founding Village section makes it difficult for users to understand their current location within the site and navigate back to previous sections. [1] This violates the heuristic of recognition rather than recall, as users are forced to rely on their memory instead of clear visual cues.
Difficulty Remembering Visited Sections: Users express difficulty remembering the sections they have visited and the content they have viewed. [2] This suggests that the application's navigation design does not provide adequate support for users to track their progress and revisit areas of interest.
Inability to Rewind Kiosk Content: The inability to rewind content within the kiosk display restricts user control and freedom. [3] This can be frustrating for users who may want to revisit or review specific information presented in the kiosk.
Absence of Current Location Indicator: The lack of a clear indication of the user's current location within the application adds to the navigation challenges. [4] This makes it difficult for users to orient themselves within the virtual environment and plan their exploration effectively.
Unintended Navigation Trigger: At the entrance of the History Walk door, accidentally pressing the "E" key causes users to return to the History Walk, disrupting their intended navigation flow. [5] This unexpected behavior can lead to confusion and frustration.
Mobile Device Navigation Issues: The forward and backward movement functionalities are not working on mobile devices. [6] This severely limits user navigation and restricts access to content within the application.
These navigation issues, combined with other design flaws, contribute to a frustrating user experience.
The source document, a heuristics evaluation of the Cultural Universe web application, identifies several frequently violated heuristics:
Consistency and Standards: This heuristic is violated in several instances, including inconsistent language use, inconsistent text alignment, confusing game controls, and the lack of a clear visual cue for interactive elements. [1-5]
Recognition Rather Than Recall: This heuristic is violated by the absence of a breadcrumb trail, the difficulty in remembering visited sections, the lack of a way to review exploration, and the unclear method of interaction with the virtual environment. [6-9]
User Control and Freedom: This heuristic relates to the users' ability to control their interaction with the system. Violations include the inability to rewind kiosk content, the unintended navigation trigger caused by pressing the "E" key, and the lack of a clear way to review explored content. [8, 10]
Aesthetic & Minimalist Design: This heuristic emphasizes the importance of presenting information in a clear and concise manner, avoiding unnecessary clutter. Violations include excessive text in the Culinary Game popup and settings help section, text-heavy game controls, and the use of a white table color that causes text to blend in, making it difficult to read. [1-4]
Help and Documentation: This heuristic is violated by the lack of clear information on how to use collected game tokens and the absence of clear visual cues for playing the cooking game. [2, 10]
These violations indicate that the Cultural Universe web application needs significant improvements to provide a user-friendly and intuitive experience.
The source document, a heuristics evaluation of the Cultural Universe web application, identifies several frequently violated heuristics:
Unclear Information and Instructions: The absence of clear information on how to use tokens collected in a game detracts from the user experience. [1] Similarly, unclear visual cues on how to play the cooking game create navigation difficulties. [1] Text-heavy game controls make it challenging for users to read, understand, and remember the instructions. [2]
Navigation Issues: The lack of a breadcrumb trail in the Founding Village violates the heuristic of recognition rather than recall, making it difficult for users to navigate the site effectively. [3] Users also experience difficulty remembering their visited sections and the content viewed. [4] The inability to rewind content in the kiosk display restricts user control and freedom. [1] The absence of a current location indicator further adds to the navigation challenges. [2] Accidentally pressing the “E” key at a specific door entrance leads users back to the History Walk, disrupting their intended navigation. [5] On mobile devices, the forward and backward movement functionalities have stopped working. [6]
Inconsistent Design: The presence of Arabic labels within the kiosk despite selecting English as the language for the 3D environment creates inconsistency and violates design standards. [5] Inconsistent text alignment within the Welcome model popup for the History Walk further detracts from the user experience. [7]
Excessive Text and Cognitive Load: The Culinary Game popup presents an overwhelming amount of text, leading to cognitive overload. [5] The settings help section also contains excessive text. [8] The Welcome model popup for the History Walk overwhelms users with too much text. [9]
Visual and Accessibility Issues: The video displayed in the kiosk lacks clarity, impacting the user experience. [4] The start button is not easily noticeable, hindering smooth navigation. [8] Non-interactive elements lack clear visual indicators, making it difficult for users to distinguish them. [9] A spelling error in the Culinary Game kiosk adds to the visual clutter and detracts from the overall professionalism. [8] The table color in the Balaleet-Vermicelli cooking section causes the text to blend in with the background, making it difficult to read. [7] Spacing and alignment inconsistencies in the Palm Tree Game controls further contribute to the visual clutter. [7] Some images are difficult to recognize. [6] The focus cursor sometimes fails to recognize the date when hovering. [6]
Technical Issues: The video playback stutters in the Visual Arts commission section. [6] Changing the graphics settings between low, optimal, and high does not result in any noticeable difference in the environment quality. [6]
Missing Features and Content: The absence of a pre-introduction for each section of the History Walk makes it difficult for users to understand the context and purpose of each section. [10] There is no way for users to review their exploration or see a summary of their learning progress. [10] The game duration is not mentioned. [10] A light effect is missing in the Drum section, creating inconsistency with other sections. [9]
These design elements collectively create a frustrating and confusing user experience, making it difficult for users to fully engage with and comprehend the content provided by the Cultural Universe web application
Notebook LM
Exported the Notion data file to a pdf and fed it to "Notebook LM". Based on the title witin the Notion spreadsheet and the available columns the A! guessed what the subject was and recomended the insights that would give a clear picture of what needs to be corrected and why supported by the data.
Insights
A detailed report with different aspects was deduced from the input (pdf export of the Notion data). The following 3 aspects were rcommended by the NotebookLM.
Figma
The presentation needed to show, the most critical problems supported with pictures and obsrvatuons to communicate and validate the severity and also to give an consolidated overview of the findings as insightes. The insights would help in deciding the plan of action to quickly correct and republish the VR website.
Link - "Problems" section of the Figma presentation
Each problem had a slide in the client presentation. Notice how each of the problem and its data from Notion has been imported into Figma's presentation using a plugin.
Link - "Insights" section of the Figma Presentation.
Based on the insights given by NotebookLM's assesment of the Notion Data, compelling data visualizations like this were created to add to the client presenation.
Plesantly shocked to see AI magic
To be honest we were sckeptic about what AI could do. In reality we used our inteligence with most of the content. We were baffled to see how AI's sugestions and thought process was so close to human inteintelligence.